PISA stats: An Examination of Different Scale Usage Correction Procedures to Enhance Cross-Cultural Data Comparability

An Examination of Different Scale Usage Correction Procedures to Enhance Cross-Cultural Data Comparability. Jia He, Joanne M. Chung, Fons J. R. van de Vijver. Journal of Cross-Cultural Psychology, October 13, 2020. https://doi.org/10.1177/0022022120960815

Abstract: This study aims to examine different scale usage correction procedures that are meant to enhance the cross-cultural comparability of Likert scale data. Specifically, we examined a priori study design (i.e., anchoring vignettes and overclaiming) and post hoc statistical procedures (i.e., ipsatization and extreme response style correction) in data from the 2012 Programme for International Student Assessment across 64 countries. We analyzed both original item responses and corrected item scores from two targeted scales in an integrative fashion by using multilevel confirmatory factor analysis and multilevel regressions. Results indicate that mean levels and structural relations varied across the correction procedures, although the psychological meaning of the constructs examined did not change. Furthermore, scores were least affected by these procedures for females who did not repeat a grade and students with higher math achievement. We discuss the implications of our findings and offer recommendations for researchers who are considering scale usage correction procedures.

Keywords: anchoring vignettes, overclaiming, response styles, score standardization, cross-cultural comparisons