Mark Wilson, Xiaohui Zheng & Leah Walker,UC Berkeley

Latent Growth Item Response Models

Amy Dray,
UCLA/UC Berkeley

Understanding ignorance: A study of social development and diversity in children’s comprehension of a story of discrimination

Betsy Feldman,
UC Davis/UC Berkeley

Best Practices for the Analysis of Problem Behaviors

Anders Skrondal,
London School of Economics

Sun-Joo Cho, UC Berkeley &
Allan S. Cohen, University of Georgia

Veronica Santelices,
Measurement Center, Catholic University of Chile

The Evolution of Gender Gap in Beijing's Middle Schools:
Are boys left behind?

Fang Lai, New York University

Gender equity in academic progress is one of the key components of the equity of education. Recent U.S. evidence shows that girls outperform boys in the overall performance, doing especially better in reading and writing, but slightly lag behind in math and science. With the increasing convergence in primary and secondary education policies between US and China, it would be interesting to compare the educational performance of these two countries. This paper is among the first attempts to provide rigorous empirical evidences of the gender achievement gap in China. Using census and administrative data on a cohort of 7,235 students who entered middle school in Beijing’s Eastern City District in 1999 and graduated in 2002, this paper looks at the evolution of the gender gap in student performance over the middle school period. We find that within each school, girls have higher test scores in all subjects for most part of the middle school period, and that the within-school gender gap favoring girls is bigger than that during the primary school, ranging from 0.17 to 0.38 standardized scores in the average scores across subjects. However, this gap has been steadily closed up over the three years of middle school period, especially in two science subjects, physics and chemistry, until a comeback of girls’ performance on the Middle School Graduation Examinations, which is then followed by a sudden drop in the girl-dominant gender gap on the High School Entrance Exams taken right after the former. The magnitudes of gender gaps decrease over the distribution of the performance in both raw and value-added measures. We relate these patterns of gender achievement gaps over the years to the differences in cognitive and non-cognitive skills, parental care received, and schooling experiences between boys and girls by adding the relevant mediators to the model and using Oaxaca decomposition. The conclusions raise concerns about boys’ performing under their potential in a test-oriented system, which is detrimental to the continuation of their education beyond the compulsory stage.

Latent Growth Item Response Models

Mark Wilson, Xiaohui Zheng & Leah Walker,
UC Berkeley

In this presentation, we will review models for analyzing longitudinal data based on two different but related paradigms: hierarchical linear modeling (HLM), and item response modeling.  The current “standard” approach uses HLM to analyze and model such data.  This has been an important methodology for analyzing large-scale longitudinal data sets over the last twenty or so years.  Despite this success, there are limitations to this approach, especially as it relates to issues concerning possible measurement complications, such as differential item functioning and multidimensionality.  This is partly because the HLM model is typically expressed using the total score as the outcome variable (or, sometimes, as the estimate of the outcome using a measurement model)—in either case, the items themselves remain outside of the formulation of the HLM analysis.  One recent approach to address this is to incorporate probit or logit links into HLM-type programs, thus allowing a direct representation of the item in the statistical model (e.g., see the work of Kamata and colleagues).  Unfortunately, this still does not allow for the full use of the sophisticated tools of item response modeling, as the HLM software is typically not designed with these possibilities in mind.  Our strategy is essentially the reverse of this—to incorporate HLM-type models directly into on item response framework.  In order to do this, we show how, using a multidimensional perspective, the HLM-formulation can be expressed directly into the argument of the item response function, following a similar strategy as used by Willet and Seyer (1995) for factor analysis: We call this a Latent Growth Item Response Model (LG-IRM).  In order to investigate the success of this strategy, we have carried out a small simulation study and an empirical analysis using NELS data.  Both show that the HLM results are essentially replicated in the LG-IRM analysis, although some technical complexities remain to be resolved.  We conclude the paper by outlining the next steps that we see as being taken to fulfill the promise of this formulation, and speculate as to the possible advantages that will this be gained.

top

Understanding ignorance: A study of social development and diversity in children’s comprehension of a story of discrimination

Amy J. Dray

Recent research in reading comprehension has emphasized the difficulties children face in understanding informational texts. But fiction is not always easy to understand either. As classrooms increasingly incorporate non-Eurocentric literature into their reading programs, and curricula use this literature to help students better understand important social issues such as racism, questions remain as to how children comprehend stories that deal with socially complex topics. In cases where deeply comprehending a story means deeply understanding the social situation depicted in it, reading comprehension may require more than just reading skill. For some stories, knowledge of social issues may play a role in comprehension.
           
In this seminar, I will present results from a study that examined relationships between reading comprehension, social development, and social context for 5th grade students who read two fictional narratives. Repeated-measure hierarchical linear modeling techniques were used to evaluate differences in comprehension of two stories. The results suggested that social understanding was a predictor of comprehension for both stories but that the effect was moderated by reading skill. Social understanding did not play a role for proficient readers, but poor readers with strong social knowledge comprehended the stories as well as their better-reading peers. Analyses also suggested students from diverse communities better understood both stories, controlling for socioeconomic status. I will conclude the talk with a discussion of the implications of the study and future directions for the research.

top

Best Practices for the Analysis of Problem Behaviors

Betsy Feldman, UC Davis/UC Berkeley

This talk will address the statistical modeling of problem-behaviors over time, using adolescent alcohol use as a motivating example. Like most problem behaviors, adolescent drinking is not straightforward to measure or model. To begin with, exact measurements of drinking over long periods of time are difficult or impossible to get. The alcohol-use data researchers collect are usually categorical summaries of the behavior, with a limited number of categories, and are often highly skewed with a pileup in the category that indicates an absence of behavior. In the first part of the talk, I use adolescent drinking data, collected from 7th grade through 12th grade, to illustrate some appropriate models for longitudinal, categorical data: hierarchical generalized linear modeling (HGLM), growth mixture analysis (GMA), latent class growth analysis (LCGA), and latent class analysis (LCA). Methods for model assessment and selection will also be discussed.

Longitudinal models of problem behaviors are frequently used in a prevention setting to identify youths at risk for later poor outcomes, such as psychological disorders, but using trajectories (or trajectory classes, in the case of mixture models) to predict binary outcomes can pose some specification and interpretational problems. In the second part of the talk, I use models from Part 1 to predict problematic drinking at age 27. I demonstrate methods for specifying and assessing prediction models, and to evaluate the quality of prediction, utilize measures such as sensitivity, specificity, and ROC curves, which are commonly found in epidemiological cross-sectional studies but rarely used in longitudinal analyses. Finally, the talk concludes with a brief description of my current work and thoughts about future directions.

Download the Best Practices powerpoint presentation.

Download the Best Practices reading list.

top

Panel Data Econometrics Using Structural Equation Modeling

Anders Skrondal, London School of Economics

Panel or longitudinal data are often used for "causal" inference with observational data in the social sciences because each subject can serve as their own control. However, when multilevel or random effects models are used a potential problem is that covariates may be correlated with the random effects, producing inconsistent estimates of "causal effects." Econometricians have developed useful methods for handling this kind of endogeneity which are not well known outside econometrics. We will demonstrate that we can alternatively use structural equation modeling (SEM) to obtain identical or very similar estimates to those produced by the econometric methods. It turns out that the approach using SEM has many advantages. The methods will be illustrated using data from the Panel Study of Income Dynamics on the returns to schooling.

top

A Multilevel Mixture IRT Model for DIF Analysis

Sun-Joo Cho, UC Berkeley & Allan S. Cohen, University of Georgia

The usual methodology for detection of differential item functioning (DIF) is to examine differences among manifest groups formed by such characteristics as gender, ethnicity, age, etc. Unfortunately, membership in a manifest group is often only modestly related to the actual cause(s) of DIF. Mixture item response theory (IRT) models have been suggested as an alternative methodology to identifying groups formed along the nuisance dimension(s)
assumed to be the actual cause(s) of DIF. A multilevel mixture IRT model (MMixIRTM) is described that enables simultaneous detection of DIF at both examinee- and school-levels. The MMixIRTM can be viewed as a combination of an IRT model, an unrestricted latent class model, and a multilevel model. Three perspectives on this model will be presented: First, the MMixIRTM can be formed by incorporating mixtures into a multilevel IRT model; second, the MMixIRTM can be formed by incorporating a multilevel structure into a mixture IRT model; and third, the model can be formed by including an IRT model in a multilevel unrestricted latent class model. A fully Bayesian estimation of the MMixIRTM will be described including analysis of label switching, use of priors, and model selection strategies along with a discussion of scale linkage. A simulation study and a real data example will be
presented.

top

Bias in the SAT?
Results from differential item functioning and predictive validity analyses

Veronica Santelices, Measurement Center, Catholic University of Chile

See Photos from this seminar.

top