Multivariate multilevel model for the analysis of PIRLS & TIMMS data

The paper analyses the Italian subsample of the TIMMS&PIRLS 2011 Combined International Database, containing the results of the assessment on Reading, Mathematics and Science for a sample of 4th grade pupils, alongside with several variables about the pupils, their teachers and their schools. This is the first time that TIMMS and PIRLS survey are conducted on the same sample, thus allowing to jointly analyse the achievement in three subjects. Indeed, the Official Reports analyse the three results separately, without exploiting relationships among them. Even research papers usually analyse the results subject by subject. We propose a multivariate multilevel model to explore the determinants of the achievements in the three subjects, focusing on the characteristics of the teachers. The model considers the three scores on reading, math and science as a joint outcome, measured at pupil level, with pupils nested within classes. The classes are nested in 202 schools, with only 37 schools having two classes. Thus, the school is not considered as a further hierarchical level with its own random effect. However, the school characteristics are included in the model as covariates. This multivariate analysis allows us to decompose the residual variances and covariances among the three scores into a pupil-level component and a class-level component. Moreover, the multivariate model allows us to test whether the covariates have different effects on the three responses. The five plausible values for each subject convey the uncertainty inherent in test scores. We account for this source of variability by combining the results through multiple imputation procedures. Preliminary results show that the three scores are highly correlated, in particular at class level. Moreover, the proportion of variability of scores at class level is relevant (from 0.20 for reading to 0.30 for science), thus calling for an analysis of contextual factors and teacher effects.