| Sept. 6 |
QME
Meeting
Carolyn Huie Hofstetter, Sophia Rabe-Hesketh, and Mark
Wilson
University of California at Berkeley
top
|
| Sept.
20 | Exploring
Person-Item Interaction with Multidimensional Multilevel Rasch Models
Wen-Chung Wang
Mark Wilson
Abstract:
In standard Rasch models, item responses are assumed to be affected
by a person factor and an item factor. These two factors work independently
on item responses; that is, no person-item interaction is allowed.
If in some cases persons do interact with items, one may either conclude
that the data do not conform to standard Rasch models, or extend standard
Rasch models to take into account such interaction. In this talk,
the latter approach is addressed. We will go through four testing
situations in which person-item interaction occurs and show how to
use multidimensional multilevel Rasch models to deal with the interaction:
(a) testlets in which several items share a common stimulus, (b) constructed-response
items in which raters are involved, (c) rating-scale items in which
persons use their own subjective judgments to select appropriate categories,
and (d) missing data in which some persons intentionally do not respond
to some items.
View
the Powerpoint slides from this presentation.
top
|
| Oct.
4 |
Beyond
Evaluation: Performance Assessments in Teacher Education
Ruth R. Chung, Ph.D.
Abstract:
Teacher performance assessments comprise a new generation of teacher
assessments that have the potential to provide a more authentic and
valid measure of teaching practice than traditional paper-and-pencil
exams. Before teacher performance assessments can gain credibility
and legal defensibility for high stakes licensing or graduation decisions,
however, there are a number of challenges to overcome, including the
cost of implementing and scoring such assessments on a large scale,
and achieving sufficient levels of reliability in scoring. Despite
such challenges, performance assessments may have other benefits that
make them useful beyond the purpose of teacher evaluation.
The Performance Assessment for California Teachers (PACT), an assessment
designed to evaluate the teaching practice of preservice teachers
enrolled in post-baccalaureate credential programs, has been piloted
since the 2002-03 academic year across the UCs, several CSUs, and
several private universities. The results of validity and reliability
studies from the first two years of piloting will be briefly summarized.
However, the focus of this talk will be on findings of my dissertation
study, which examined the impact of the performance assessment on
the learning experiences and teaching practice of preservice teachers
who completed the PACT, as well as the conditions that supported their
learning. This presentation explains how a summative assessment of
teaching could serve as a formative learning experience for teacher
candidates. Also explored are the potential uses of teacher performance
assessment as one measure of the effectiveness of teacher preparation
programs, as a tool for program learning and reform, and as a measurement
instrument for research.
View
the Powerpoint slides from this presentation.
top
|
| Oct.
18 | California
Department of Education's (CDE) Desired Results Developmental Profile
(DRDP):
Instrument development and Item-Response scaling to assess ten dimensions
of development, from birth through middle school
Karen Draney, Stephen Moore,
and Hiro Yamada
University of California at Berkeley
Abstract:
The California Department of Education's Child Development Division
(CDD) sponsors thousands of child care centers across California.
In 1996 CDD created an outcomes based accountability system called
Desired Results for Children and Families. The purpose of the DR System
is to support program improvement in all state-sponsored care centers.
The assessment component of the overall DR System is the Desired Results
Developmental Profile (DRDP). DRDP is an assessment tool for measuring
progress of children on ten dimensions of development, based on teacher
observation. DRDP was first implemented in 2000. Since 2002, the Berkeley
Evaluation and Assessment Research Center (BEAR) has been leading
CDD's initiative to revise DRDP to enhance the psychometric measurement
validity and reliability of DRDP instruments. In fall of 2003, BEAR
conducted a field test of revised DRDP instruments. BEAR then revised
the instruments again and conducted a calibration study in spring
2005. This talk will describe the process by which the DRDP has been
revised by BEAR, and will describe the Item-Respoinse scaling analyses
BEAR is now carrying out to support valid and reliable measurement
across a broad age range.
top
|
Nov. 1
|
Education
and Educational Measurement in Sweden
Per-Erik Lyren, Umeå
University
Abstract:
The presentation will begin with a general overview of the Swedish
educational system, with an emphasis on the system for admission to
higher education. Furthermore, some of the current debate regarding
the educational system will be addressed. The second part of the presentation
will be about graduate studies in Sweden in general and Umeå
University’s graduate program in Educational Measurement in
particular. Topics will include admission, program design, length
of studies, etc. Finally, the last part of the presentation will focus
on the SweSAT (Swedish Scholastic Assessment Test), which is used
for selection to higher education. The SweSAT is probably regarded
as a “dinosaur” to many psychometricians (and others),
since it is neither based on IRT nor computerized. So, a background
on the test, its format, administration, and development during the
recent years will be provided. Also, some thoughts on where the test
might be heading in the future will be shared.
top
|
Nov. 15
|
Probabilistic
Networks for Assessing the Course of Emotions
Frank Rijman
Katholieke Universiteit Leuven, Belgium
Abstract:
The experience sampling methodology (ESM) is a relatively
new data collection technique that results in multiple assessments of
the same person. This way, it becomes possible to model within-person
processes over time. The data I will discuss stem from an ESM study
in which participants were repeatedly questioned about the occurrence
of positive and negative feelings in their daily lives.
The starting point is a latent class model that allows for transitions
between classes over time. This basic model is closely related to the
latent transition (Collins & Wugalter, 1992) and latent markov model
(Langeheine & van de Pol, 1994). However, the latter models currently
allow for a few timepoints only. It is shown how probabilistic network
theory can be used to estimate models that involve many more measurement
occasions. The success of probabilistic networks stems from the fact
that probabilistic networks integrate aspects from both probability
and graph theory. Further advantages of linking up with probabilistic
network theory include that missing data are easily dealt with if the
missingness mechanism is ignorable, and that the basic model is easily
extended. Some extensions will be discussed.
View
the Powerpoint slides from this presentation.
top
|
| Nov. 16 |
Explanatory versus Descriptive Item Response Models
Mark Wilson
University of California, Berkeley
View
the Powerpoint slides from this presentation.
Multidimensionality, the Two-Parameter Logistic Model, and Polytomous Data
View
the Powerpoint slides from this presentation.
Frank Rijman
Katholieke Universiteit Leuven, Belgium
top |
Dec. 4
|
Multiple/mixed
Methods Designs and Strategies
Carolyn Huie Hofstetter
GSE, UC Berkeley
Abstract:
The use of both quantitative methods (e.g., structured surveys,
standardized test data) and qualitative methods (e.g., observations,
unstructured interviews) is increasingly commonplace in educational
evaluation and research, particularly in the context of NCLB. While
the choice of methods is dependent on the research questions at hand,
educational research questions are often complex in nature, thus require
a combination of methods. How to go beyond the presence and analysis
of each type of method in a study, to actually triangulating them,
however, is challenging. This presentation will discuss what we mean
by multiple/mixed methods in educational research, provide analytic
strategies/considerations, and present selected examples.
top
|
