Standard-Setting Using the Construct Map

BEAR Center Participants: 
Mark Wilson, Karen Draney, Cathy Kennedy, Deborah Peres, Sevan Tutunciyan, Hiroyuki Yamada

Wilson and Draney (2002) have described a new method, called "construct mapping" for setting standards in achievement testing that takes advantage of the image of a "progress map" based on an item response model analysis of the data. This approach allows the standard setters to give interpretational meaning to potential cut-scores in terms of the responses to both multiple choice and open-ended items. It can also be used as a basis for the "propagation" of the selected cut-scores to new tests using the linking facilities of item response modeling. In the current project, we will investigate the improvement and use of this method in several ways. First, we have written a new software tool that greatly enhances the usefulness of the construct mapping approach. Second, we will use qualitative methods to investigate the best ways to help standard setters use the new tool. Third, we will apply the approach to a more complex scoring procedure involving items scored in multiple (correlated) dimensions.