Challenges and Opportunities of Learning Progressions for the Psychometric Community

TitleChallenges and Opportunities of Learning Progressions for the Psychometric Community
Publication TypeConference Presentation
Year of Publication2013
AuthorsWilson, M
Abstract

In this presentation I discuss both the challenges that are posed to measurement practice by the attempting to develop assessments in the context of a learning progression, and also the opportunities that arise. Note that the manner in which the measurement approach supports the learning progression is referred to as the outcome progression for the learning progression. 

In typical measurement practice, one often concentrates on uni-dimensional constructs. If there are multiple dimensions, then multidimensional approaches are available, such as factor analysis or multidimensional item response modeling. However, learning progressions often include not only multidimensional hypotheses, but also the presence of “links” from one dimension to another. Responding to this challenge, in this presentation, I (a) offer a description of how the BEAR Assessment System (BAS) can be seen as providing a sensible modeling approach for the uni-dimensional case (and, by straightforward extension, for multidimensional cases too), and (b) discuss how this approach can then be expanded to respond to the challenge of hypothesized links between dimensions, thus enriching the armamentarium of psychometric models. 

Thus, the presentation first summarizes the elements of the BAS, emphasizing the central concept of a construct map, and describes how the idea of a construct map can be helpful in the context of a unidimensional learning progression. The presentation uses an example based on the ADM Project described in the introduction. The construct maps form the basis for many iterative interactions among the key players in the development of the outcome progression, structuring the conversations at the following points: (a) the design of items, (b) the coding and scoring of student responses to those items, (c) the iterations between item redesign and item response coding, (d) the development of representations to report the results of the assessments, and (e) professional development for teachers and other users.

The presentation then focuses on some of the more complex ways to see the relationship between a set of construct maps and a learning progression (see Wilson (2009) for a more complete set). This provides the context for a discussion of a new family of psychometric models called structured constructs models (SCMs), which simultaneously model multiple dimensions and also the hypothesized links between them. Alternative formulations of SCMs are presented, along with some examples of results for the ADM data. 

The presentation concludes by (a) discussing some strengths and limitations of this conceptualization and (b) explores the costs/benefits of the wide and deep interactions that this approach implies for (i) measurement and assessment experts, (ii) cognitive development researchers, (iii) curriculum developers, and (iv) teachers.

Notes

annual meeting of the American Educational Research Association, San Francisco

Persons:
Presentation or Publication date: 
Monday, April 29, 2013