BEAR Center Calendar

BEAR Events Calendar

Fall 2004

|Archive of Past Seminars|Current Seminars|

Step 14

Ordered-Category Attribute Coding Framework for Cognitive Assessments

Tzur Karelitz, Ph.D.

BEAR Center, GSE, UC Berkeley

Some Cognitive Diagnostic Assessment models define skills as binary. Examinees are described as either `skill masters' or `non-masters' and items as either requiring the skill or not. This approach is based on the Q matrix theory (Tatsuoka, 1995), that uses a binary matrix to represent task requirements in terms of underlying skills and knowledge.

I propose an Ordered Category Attribute Coding (OCAC) framework designed to enhance the diagnostic information provided by such models. This approach defines any skill, k, by the M_k steps taken to master it. Consequently, the entries of the categorical Q matrix represent skills' mastery level required by test items and examinees' knowledge patterns represent their location on the learning path of each skill. To illustrate the OCAC approach, consider non-native English speakers who study English. They learn the various tenses, and how to apply them in different settings. For instance, the attribute ``Mastery of past tense'' can be performed at many levels:

None- no ability to use past tense.
Basic- ability to transform a regular verb to past tense.
Moderate- ability to transform a sentence from present simple to past simple, using regular verbs.
Advanced- ability to transform a sentence from any tense to past tense, using regular verbs.
Master- ability to transform a sentence from any tense to past tense, using regular and irregular verbs.

An exam in English tense proficiency can be represented by crossing every tense with these 5 mastery levels. The flexibility of the OCAC framework allows for a more informative, parsimonious and efficient representation of task requirements and examinee knowledge. The levels of required skills can be estimated simultaneously with the examinees knowledge states as well as noise parameters, with high recovery rate of simulated and real data.

Sept 28

Making sense of causal inference in program evaluation and policy
research

Paul W. Holland

Educational Testing Service

Issues of causation and causal inference are often central to both program evaluation and policy research, yet they are vaguely understood even after the 2000 years of analysis started by Aristotle. How can we put our finger on the difference between description and causation? What are the basic issues and how can we sensibly address them in practical work? What
role do the probability models of statistics play in all of this? I will give my answers to these questions in the context of educational research examples.

^

Oct 12

Heuristics for developing classroom science assessments

Kristin M. Bass, Ph.D.

BEAR Center, GSE, UC Berkeley

In this presentation, I will reflect on my experiences developing various types of assessments (e.g., performance assessments, laboratory notebooks, short writing exercises) for elementary hands-on science curricula. I will share the heuristics, or rules of thumb, that I use when creating and evaluating tasks and scoring systems. For instance, I will discuss how I generate tasks that build on existing instructional practices. I will also describe a phenomenon I call the "Scooby-Doo effect" and discuss its relevance to assessment evaluation. My heuristics will be linked to the larger call to articulate methods of assessment development and identify general design principles (National Research Council, 2001).

.

^

Oct 26

Causal Inference and the Effectiveness of Catholic/Christian Mission Secondary Schools in Singapore

Laik Teh, Ph.D.

Nov 9

Teacher Assessment Panel

PJ Hallum, Ph.D. from EVTLD, BEAR Center

Kendyll Stansbury from PACT

Check back here for to view abstract.

^

BEAR Center
Graduate School of Education
University of California, Berkeley
Berkeley, CA 94720