Proficiency Estimation
Contents |
Proficiency Estimation Type
ConstructMap can produce proficiency estimates using expected a-posteriori (EAP), maximum likelihood (MLE), or plausible values (DPV) estimation algorithms. Each type of proficiency estimate has a different dialog window.
To select an estimation type, select Estimation Tasks - Proficiency Estimation Options, and then use the
EAP: Expected A Posteriori
The EAP method provides a Bayesian analysis of observed events. Conditional response probabilities over the user-defined logit range, with the number of quadrature points also defined by the user, are computed from the item parameter estimates. A multivariate population distribution is then applied to compute multivariate posterior distributions for each respondent. The vector of expected ability estimates (i.e., one for each dimension) for a person n, qnEAP, are computed from
which is approximated by
Users select the integration method (Gaussian Quadrature or Monte Carlo) and associated nodes in the EAP Proficiency Estimation Options dialog window (see Figure 1 above).
MLE: Maximum Likelihood Estimate
ConstructMap produces MLE values from the log likelihood equation,
Users specify the MLE Convergence criteria used in the Newton-Raphson routine when they set the MLE Proficiency Estimation Options. An example is shown in Figure 2. The value entered here will be used by ConstructMap to determine when the improvement in the proficiency estimates during the Newton-Raphson iterations is sufficiently small. Estimation will be considered complete for a case when the improvement from the prior iteration to the current iteration is smaller than this value. The default value is .005. If the model is taking a long time to converge, you may wish to increase this number. When standard errors are large, you may wish to decrease this number.
DPV: Draw Plausible Values
ConstructMap can also produce plausible values from the posterior distributions. This approach makes random draws from the posterior distribution for a respondent with a specific response pattern. Each draw is considered a representative value from the distribution of possible scores for all respondents who generate that specific pattern of responses to the items.
Users can change the number of draws using the Number of Plausible Values setting in the DPV Proficiency Estimation Options dialog window. An example is shown in Figure 1. The value entered here will be used by ConstructMap to determine the number of plausible values to draw from each person’s posterior distribution to arrive at an average. Research suggests that five draws provides both efficiency and accuracy similar to that attained by an infinite number of draws. Large numbers will produce plausible values close to the mean (EAP) while small values will produce more random estimates simulating natural behavior.
Other Settings
Integration Method
When EAP or plausible value (DPV) estimates are requested, the user must select either the Gaussian Quadrature method or the Monte Carlo method of integration. Use the pull-down icon, as shown in Figure 38, to select the method.
Quadrature Points
When the Quadrature integration method is selected, the user can specify the number of quadrature points. In multidimensional models, this is the number of logit values that are used for each dimension as n-tuples are defined for the population. For example, in a three-dimensional model with 15 nodes, probabilities and population distributions are computed for combinations of the variables, yielding 153, or 3,375 total combinations. The default value is 15.
Quadrature Nodes
When the Monte Carlo integration method is selected, the user can specify the total number of quadrature nodes to be used. In multidimensional models, this is the number of n-tuples that will be defined for the population. The theta values for each dimension are selected randomly from a standard normal distribution. The theta matrix is then rotated to align with the stored population means and covariances.
Maximum Logit and Minimum Logit
The logit range displayed in the Proficiency Estimation Options window is captured from the Calibration Options settings. It cannot be changed here.
