Testing cross-national construct equivalence in international surveys. Applications on international civic and citizenship education survey data

Data collected in international studies enables researchers, educators, and policy makers to compare educational systems regarding several aspects such as student achievement but also different attitudes and beliefs. Such findings are often included in international reports in the form of league tables comparing country averages on different measures and are subject of important country comparisons and subsequent decisions. Nevertheless the cross-cultural generalizability of attitudinal measures and the possibility of country comparisons cannot be always reached and statistical tests of measurement invariance (MI) must be carried out to ensure meaningful country comparisons and related conclusions. This dissertation aims to address the issue of MI of attitudinal measures. A case is made for valid country comparisons of measures collected in cross-national surveys by documenting and illustrating with examples the required tests of measurement invariance (MI). After a comprehensive account of the theoretical groundings of MI in a multiple-group confirmatory analysis (MG-CFA) framework, three nested and consecutively more constraining levels of invariance - configural invariance, metric invariance, and scalar invariance – are discussed and explored. More specifically, by testing a set of three increasingly constrained models measuring the latent concept, we estimate whether model structure in the groups, factor loadings and intercepts are equivalent. Consequently, we establish if comparisons that are made on the latent variable are significant across groups (countries). In agreement with the theory, it is assumed that in order to ensure the highest level of cross-cultural comparability (e.g. comparing country averages), MI testing must confirm the highest level of MI, scalar invariance. We approach the research topic taking as example the measure of students’ attitudes toward equal rights for immigrants collected in the International Civic and Citizenship Education Study – ICCS conducted by the International Association for the Evaluation of Educational Achievement – IEA in 2009. The methodology is applied both to all European countries and to sub-groups of students such as the non-immigrant/native students in these countries as well as students with an immigrant background. The estimation takes into account the specific properties of data. The results are discussed both within the sample and sub-samples setting and show that the required level of scalar invariance is not always reached. In particular, in the studied countries, higher levels of construct equivalence seem to be achieved only for the sub-sample of students with an immigrant background. Conclusions and implications for further research and also for reporting and interpreting current research findings are drawn.