ChemQuery/Living by Chemistry (LBC)

Collaborating Institutions: 
Department of Chemistry, UC Berkeley, Lawrence Hall of Science, UC Berkeley, SESAME Graduate Group, UC Berkeley, BEAR Center

Department of Chemistry, UC Berkeley Participants: 
Angelica M. Stacy, Rebecca Krystyniak, Sheryl Mebane

SESAME Graduate Group, UC Berkeley Participants: 
Jennifer Claesgens

BEAR Center Participants: 
Kathleen Scalise, Mark Wilson

Summary from Scalise, Claesgens, Wilson & Stacy (2006):

 

ChemQuery is an assessment system that uses a framework of the key ideas in the discipline, called the Perspectives of Chemists, and criterion-referenced analysis using item response theory (IRT) to map student progress in general chemistry. It currently includes the Perspectives framework, a scale of student progression in understanding, an item bank of open-ended constructed response items, some computer-adaptive selected response items [4,5], a scoring rubric, and item exemplars. Analysis using item response models includes generation of Wright maps, item fits, student progress reports, and reliability and validity evidence. Integral to such criterion-reference measurement is a focus on what is being measured, which is referred to as the construct. The construct is the intention of the assessment, its purpose, and the context in which it is going to be used. This approach to research on student understanding allows us to use quantitative measures to test and refine hypotheses about how learning develops, and to relate that to qualitative evidence about student learning. 

 

The Perspectives construct is built on the theoretical idea that the field of chemistry can be largely grouped into three core conceptions, or scientific models: matter, change, and energy. The purpose in framing the "big ideas" of chemistry is to organize the overarching ideas of the discipline while simultaneously constructing an instrument for measuring the values of these variables for individual students. In addition to this organization of topics, the ChemQuery construct further suggests that learning in each of these three areas, or "strands," is a rich and varied progression, from forms of naive to more complete and consistent understanding of explanatory models of chemistry, and that this progression in understanding represents a set of necessary and distinct areas of understanding enabling higher-order thinking in chemistry. 

 

It may be that the identification of three, or some small discrete number, of "big idea" Perspectives could be a useful point of comparison for this type of assessment across courses in other STEM disciplines. We have been in discussions with physicists, biologists, and mathematicians on what such Perspectives might be in their fields, and are helping other STEM disciplines to explore some possible approaches. As an aside, it is interesting to note that while aspects of the Matter and Energy Perspectives have been identified by physicists as potentially crossing over to their fields fairly directly, the model of Change in our framework is perceived as very much the perspective of chemical and physical change generally described in chemistry objectives and standards. Change in physics might need to encompass both considerably more macroscopic change (for instance, positional change of large objects), and sub-atomic change for particles below the level of electrons, neutrons, and protons. 

 

Developing the ChemQuery progress variables has required studying misconception research and theories of learning such as alternate conceptions and blending of models. This helps describe how students build understanding in chemistry. Implicit to the ChemQuery approach is the assumption that emphasizing deep understanding supports the learning of algorithms and chemical definitions, which is consistent with the Constructivist perspective that the learner builds his or her understanding. Thus, the purpose of the Perspectives construct is to describe a hierarchy of chemistry content that then defines variables to allow us to measure learning outcomes, determines scales for these variables, and constructs instruments for measuring the values of these variables for individual students.