This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.

Delios, A., Giulia Clemente, E., Wu, T., Tan, H., Wang, Y., Gordon, M., et al. (2022). Examining the generalizability of research findings from archival data. PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 119(30) [10.1073/pnas.2120377119].

Examining the generalizability of research findings from archival data

Cristina Zogmaister
Membro del Collaboration Group
2022

Abstract

This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.
Articolo in rivista - Articolo scientifico
archival data; context sensitivity; generalizability; reproducibility; research reliability;
English
19-lug-2022
2022
119
30
e2120377119
open
Delios, A., Giulia Clemente, E., Wu, T., Tan, H., Wang, Y., Gordon, M., et al. (2022). Examining the generalizability of research findings from archival data. PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 119(30) [10.1073/pnas.2120377119].
File in questo prodotto:
File Dimensione Formato  
Delios-2022-PNAS-Collaboration list members.pdf

accesso aperto

Descrizione: Collaboration list members
Tipologia di allegato: Other attachments
Dimensione 1.79 MB
Formato Adobe PDF
1.79 MB Adobe PDF Visualizza/Apri
Delios-2022-PNAS-VoR2.pdf

accesso aperto

Descrizione: Research article
Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Dimensione 843.41 kB
Formato Adobe PDF
843.41 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/396185
Citazioni
  • Scopus 12
  • ???jsp.display-item.citation.isi??? 14
Social impact