Tuesday, November 26, 2024
2:00 - 4:00 PM (PST) at Berkeley Way West 1207 and via Zoom
Open to GSE faculty, students, and community.
Request a zoom link from convenors@bear.berkeley.edu
About the Session:
This special session features multiple presentations exploring cutting-edge AI applications in educational measurement and psychometrics. These talks will delve into topics like autograding, decision-tree approaches, holistic and analytic methods for training autograders, and handling data imbalances. The session will provide insights into leveraging AI to enhance accuracy, fairness, and scalability in assessment practices.
Context: Autograding with AI (Presenter: Perman Gochyyev)
A concise introduction on how AI technologies are being integrated into the field of educational measurement, with a focus on autograding.
Exploring analytic vs. holistic approaches for autograder development (Presenters: Will Cai, Yukie Toyama, Xingyao Xiao)
Comparison of training autograders on analytic codes that merge into holistic scores versus training them directly on holistic scores.
Leveraging decision tree in autograder development (Presenters: Yunting Liu, Alexis Fernandez, Tila Tran, Yukie Toyama)
Exploration of a method that trains an autograder to make sequential decisions at boundaries of adjacent scores, helping to uncover the black box of the autograder's grading process, and identify areas where human intervention may be necessary.
Imbalance Issues in Training Data (Presenters: Yunting Liu, Xutao Feng)
Analysis of imbalances in items: Market item (low score prevalence) vs. Open-source item (Chemistry, high score prevalence). Addressing challenges and proposing solutions for balanced AI model performance.vides estimates of standardized treatment effect sizes corrected for attenuation due to measurement error.
About the presenters:
Will Cai is a fourth-year undergraduate student at UC Berkeley, majoring in Math and Computer Science. His research interests include Machine Learning, Natural Language Processing, and their real-world applications.
Alexis Fernandez is a third-year undergraduate student at UC Berkeley studying Engineering Mathematics and Statistics, with an emphasis in computer science. She’s been working at the BEAR center over a year, contributing to autograder development for college-readiness assessments.
Xutao Feng is a fourth-year undergraduate student at UC Berkeley, and his majors are Computer Science and Data Science. He has been working for the BEAR center for over a year, contributing to autograder development for a college-readiness assessment in computational thinking.
Perman Gochyyev is currently a Statistical Project Leader at Sanofi, Perman has a Ph.D. in Quantitative Methods and Evaluation from UC Berkeley. His expertise spans biostatistics, psychometrics, and causal inference, with a focus on IRT models and latent variable modeling.
Yunting Liu is a third-year doctoral student at UC Berkeley in the Social Research Methodology cluster and a researcher at the BEAR Center. Her research focuses on latent variable modeling, longitudinal analysis, and Generative AI applications in education.
Yukie Toyama is a research specialist at the BEAR center. Her research focuses on the use of learning progressions and explanatory item response models to advance understanding about student learning in literacy and STEM fields.
Tila Tran is an undergraduate Statistics major at UC Berkeley and research apprentice for URAP. Her research interests encompass measurement invariance, applications of Generative AI, and developing predictive models to inform decision-making.
Xingyao Xiao is a PhD candidate at UC Berkeley and a researcher at the BEAR Center. Her research focuses on leveraging Bayesian statistical methods and psychometric models to enhance the interpretability and fairness of assessment results, making them more meaningful and actionable in educational contexts.