Social Evaluative Reasoning in the Workplace: Validation of an Assessment of Soft Skill Proficiency for Secondary Students in Special Education

While assessments of general social proficiency constructs are numerous in the ASD literature, and in the special education literature more broadly, investigations of the latent structure(s) of vocational-specific social proficiency constructs have yet to receive serious empirical investigation. Furthermore, trends in this literature point to a number of measurement-related issues that include: (1) item/construct incongruity in the development of instruments; (2) inconsistent representations of the role of context in instruments; (3) relatively outdated approaches to instrument design and analysis of item response data, and; (4) lack of evidence for the internal structure of measurement variables in arguments for validity. Finally, the notion of learning progressions in the assessment of social proficiency, and potential models of cognition that underpin them, are not generally considered or empirically tested in this literature.

This study extends this body of research by addressing the measurement-related issues introduced above. In addition, I propose and test an explicit model of cognition for the measurement of Social Evaluative Reasoning (SER) in the workplace that takes advantage of the principles of sound educational measurement embedded in Wilson’s (2005) item response modeling approach to constructing measures. SER is a soft skill construct defined as context-specific critical thinking involving the appraisal of the effectiveness and appropriateness of employee behavior as it occurs in response to a variety of customer social cues in entry-level employment sectors heavy in soft skill demand. Proficiency in SER ability is conceptualized in an antecedent (A)-behavior (B)-consequence (C) formulation. More specifically, within a workplace setting, given (A): some amount of available social information, was (B): a target employee’s behavioral response, (C): appropriate given the situational context of the workplace and the organizing and directing forces it imposes on employee behavior?

Scenarios in a comic strip format were developed to represent workplace settings heavy in soft skill demand. Control for the social complexity of the scenarios was based on the manipulation of two factors: 1) type, frequency, and co-occurrence of basic and complex emotions (viz., social perceptual units (SPUs), and; 2) whether or not the target employee correctly resolved the situation depicted in the scenario (correct versus incorrect resolution). Analysis of the item response data utilized both descriptive and item explanatory extensions of the Rasch model. In this presentation, results will be discussed in the context of the five strands of the AERA/APA/NCME (2014) testing standards for validity evidence and the practical implications of this research will also be considered.

Dr. Jerred Jolin is a 2018 graduate of the Joint Doctoral Program in Special Education, University of California at Berkeley and San Francisco State University. His research interests lay at the intersection of post-secondary transition, vocational development, and educational measurement and assessment. He has worked professionally in the employment services sector in a number of capacities, supporting young adults with a variety of disabilities in obtaining employment. More recently he was a part-time faculty member at Foothill College where he taught vocational development courses in the Tools for Transition and Work program. Currently his a Postdoctoral Scholar at the Berkeley Evaluation and Assessment Research Center, where he working on an NMSI-funded project to develop assessments of college readiness in algebraic and statistical thinking.

Tuesday, September 4, 2018 - 2:00pm
2121 Berkeley Way
PDF icon Slide1-31.6 MB
PDF icon Slide2-31.48 MB
PDF icon Slide3-31.57 MB