Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.

Aarts, A., Anderson, J., Anderson, C., Attridge, P., Attwood, A., Axt, J., et al. (2015). Estimating the reproducibility of psychological science. SCIENCE, 349(6251) [10.1126/science.aac4716].

Estimating the reproducibility of psychological science

COSTANTINI, GIULIO
Membro del Collaboration Group
;
PERUGINI, MARCO
Membro del Collaboration Group
;
2015

Abstract

Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.
Articolo in rivista - Articolo scientifico
Reproducibility; Replication
English
2015
349
6251
aac4716
reserved
Aarts, A., Anderson, J., Anderson, C., Attridge, P., Attwood, A., Axt, J., et al. (2015). Estimating the reproducibility of psychological science. SCIENCE, 349(6251) [10.1126/science.aac4716].
File in questo prodotto:
File Dimensione Formato  
aac4716.full.pdf

Solo gestori archivio

Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Dimensione 661.1 kB
Formato Adobe PDF
661.1 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/89972
Citazioni
  • Scopus 5221
  • ???jsp.display-item.citation.isi??? 3658
Social impact