Background: A recent stream of literature recognizes the impact of good/poor implementation on the effectiveness of programs. However, implementation is often disregarded in randomized controlled trials (RCTs) because they are run on a small scale. Replicated RCTs, although rare, provide a unique opportunity to study the relevance of implementation for program effectiveness. Objectives: Evaluating the effectiveness of an at-scale professional development program for lower secondary school math teachers through two repeated RCTs. Research Design: The program lasts a full school year and provides innovative methods for teaching math. The evaluation was conducted on two cohorts of teachers in the 2009/10 and 2010/11 school years. The program and RCTs were held at scale. Participating teachers and their classes were followed for 3 years. Impact is estimated by comparing the math scores of treatment and control students. Subjects: The evaluation involved 195 teachers and their 3940 students (first cohort) and 146 teachers and their 2858 students (second cohort). Measures: The key outcome is students’ math achievement, measured through standardized assessment. Results: In the first wave, the program did not impact on students’ achievement, while in the second wave, a positive, persistent, and not insignificant effect was found. After excluding other sources of change, different findings across waves are interpreted in the light of improvements in the program implementation, such as enrollment procedure, teacher collaboration, and integration of innovation in daily teaching. Conclusions: Repeated assessment of interventions already at-scale provides the opportunity to better identify and correct sources of weak implementation, potentially improving effectiveness.

Abbiati, G., Argentin, G., Caputo, A., Pennisi, A. (2022). Repetita Iuvant? A Repeated Randomized Controlled Trial on the Effectiveness of an At-Scale Teacher Professional Development Program. EVALUATION REVIEW, 46(5), 578-597 [10.1177/0193841X211055354].

Repetita Iuvant? A Repeated Randomized Controlled Trial on the Effectiveness of an At-Scale Teacher Professional Development Program

Argentin, Gianluca;
2022

Abstract

Background: A recent stream of literature recognizes the impact of good/poor implementation on the effectiveness of programs. However, implementation is often disregarded in randomized controlled trials (RCTs) because they are run on a small scale. Replicated RCTs, although rare, provide a unique opportunity to study the relevance of implementation for program effectiveness. Objectives: Evaluating the effectiveness of an at-scale professional development program for lower secondary school math teachers through two repeated RCTs. Research Design: The program lasts a full school year and provides innovative methods for teaching math. The evaluation was conducted on two cohorts of teachers in the 2009/10 and 2010/11 school years. The program and RCTs were held at scale. Participating teachers and their classes were followed for 3 years. Impact is estimated by comparing the math scores of treatment and control students. Subjects: The evaluation involved 195 teachers and their 3940 students (first cohort) and 146 teachers and their 2858 students (second cohort). Measures: The key outcome is students’ math achievement, measured through standardized assessment. Results: In the first wave, the program did not impact on students’ achievement, while in the second wave, a positive, persistent, and not insignificant effect was found. After excluding other sources of change, different findings across waves are interpreted in the light of improvements in the program implementation, such as enrollment procedure, teacher collaboration, and integration of innovation in daily teaching. Conclusions: Repeated assessment of interventions already at-scale provides the opportunity to better identify and correct sources of weak implementation, potentially improving effectiveness.
Articolo in rivista - Articolo scientifico
implementation analysis; mathematics; repeated randomized controlled trial; student achievement; teacher professional development;
English
26-nov-2021
2022
46
5
578
597
reserved
Abbiati, G., Argentin, G., Caputo, A., Pennisi, A. (2022). Repetita Iuvant? A Repeated Randomized Controlled Trial on the Effectiveness of an At-Scale Teacher Professional Development Program. EVALUATION REVIEW, 46(5), 578-597 [10.1177/0193841X211055354].
File in questo prodotto:
File Dimensione Formato  
Abbiati-2022-Evaluation Review-VoR.pdf

Solo gestori archivio

Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Licenza: Tutti i diritti riservati
Dimensione 530.03 kB
Formato Adobe PDF
530.03 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/362161
Citazioni
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 1
Social impact