An essential first step in planning a confirmatory or a replication study is to determine the sample size necessary to draw statistically reliable inferences using power analysis. A key problem, however, is that what is available is the sample-size estimate of the effect size, and its use can lead to severely underpowered studies when the effect size is overestimated. As a potential remedy, we introduce safeguard power analysis, which uses the uncertainty in the estimate of the effect size to achieve a better likelihood of correctly identifying the population effect size. Using a lower-bound estimate of the effect size, in turn, allows researchers to calculate a sample size for a replication study that helps protect it from being underpowered. We show that in most common instances, compared with nominal power, safeguard power is higher whereas standard power is lower. We additionally recommend the use of safeguard power analysis to evaluate the strength of the evidence provided by the original study.

Perugini, M., Gallucci, M., Costantini, G. (2014). Safeguard Power as a Protection Against Imprecise Power Estimates. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 9(3), 319-332 [10.1177/1745691614528519].

Safeguard Power as a Protection Against Imprecise Power Estimates

PERUGINI, MARCO
;
GALLUCCI, MARCELLO;COSTANTINI, GIULIO
2014

Abstract

An essential first step in planning a confirmatory or a replication study is to determine the sample size necessary to draw statistically reliable inferences using power analysis. A key problem, however, is that what is available is the sample-size estimate of the effect size, and its use can lead to severely underpowered studies when the effect size is overestimated. As a potential remedy, we introduce safeguard power analysis, which uses the uncertainty in the estimate of the effect size to achieve a better likelihood of correctly identifying the population effect size. Using a lower-bound estimate of the effect size, in turn, allows researchers to calculate a sample size for a replication study that helps protect it from being underpowered. We show that in most common instances, compared with nominal power, safeguard power is higher whereas standard power is lower. We additionally recommend the use of safeguard power analysis to evaluate the strength of the evidence provided by the original study.
Articolo in rivista - Articolo scientifico
power analysis, precision, confidence intervals, experimental methods
English
2014
9
3
319
332
none
Perugini, M., Gallucci, M., Costantini, G. (2014). Safeguard Power as a Protection Against Imprecise Power Estimates. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 9(3), 319-332 [10.1177/1745691614528519].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/51784
Citazioni
  • Scopus 122
  • ???jsp.display-item.citation.isi??? 117
Social impact