Trying to Raise (Low) Math Achievement and to Promote (Rigorous) Policy Evaluation in Italy: Evidence From a Large-Scale Randomized Trial

Background: Italy is a country showing low math achievement, especiallyin the Southern regions. Moreover, national student assessments are recentand rigorous policy evaluation is lacking. This study presents the results ofone of the first randomized controlled trials implemented in Italian schoolsin order to measure the effects of a professional development (PD)program for teachers on student math achievement. The program wasalready at scale when it was being evaluated. Objective: Assessing theeffects of a PD program for math teachers on their students' achievementand making suggestions for future policy evaluations. Design: A large-scaleclustered randomized control trial has been conducted. It involves 175 lower secondary schools (sixth - eighth grade) in four among the Italian lowestperforming regions. Alongside national standard math assessments, theproject collected a wide amount of information. Subjects: Math in lowersecondary schools. Measures: Math achievement as measured by standardizedtests provided by the National Education Assessment Institute (IstitutoNazionale per la Valutazione del Sistema di Istruzione e Formazione);teacher and student practices and attitudes collected through questionnaires.Results: Findings suggest that the program had no significant impact on mathscores during the first year (when the program was held). Nonetheless someheterogeneity was detected, as the treatment does seem "to work" withmiddle-aged teachers. Moreover, effects on teaching practice and studentattitudes appear. Conclusion: Some effects attributable to the interventionhave been detected. Moreover, this project shows that a rigorous approachto evaluation is feasible also in a context lacking attention towards evidencebasedpolicies, such the Italian school system.