Motivated by the practice of exploratory research, we formulate an approach to multiple testing that reverses the conventional roles of the user and the multiple testing procedure. Traditionally, the user chooses the error criterion, and the procedure the resulting rejected set. Instead, we propose to let the user choose the rejected set freely, and to let the multiple testing procedure return a confidence statement on the number of false rejections incurred. In our approach, such confidence statements are simultaneous for all choices of the rejected set, so that post hoc selection of the rejected set does not compromise their validity. The proposed reversal of roles requires nothing more than a review of the familiar closed testing procedure, but with a focus on the non-consonant rejections that this procedure makes. We suggest several shortcuts to avoid the computational problems associated with closed testing. © Institute of Mathematical Statistics, 2011.

Goeman, J., Solari, A. (2011). Multiple Testing for Exploratory Research. STATISTICAL SCIENCE, 26(4), 584-597 [10.1214/11-STS356].

Multiple Testing for Exploratory Research

SOLARI, ALDO
2011

Abstract

Motivated by the practice of exploratory research, we formulate an approach to multiple testing that reverses the conventional roles of the user and the multiple testing procedure. Traditionally, the user chooses the error criterion, and the procedure the resulting rejected set. Instead, we propose to let the user choose the rejected set freely, and to let the multiple testing procedure return a confidence statement on the number of false rejections incurred. In our approach, such confidence statements are simultaneous for all choices of the rejected set, so that post hoc selection of the rejected set does not compromise their validity. The proposed reversal of roles requires nothing more than a review of the familiar closed testing procedure, but with a focus on the non-consonant rejections that this procedure makes. We suggest several shortcuts to avoid the computational problems associated with closed testing. © Institute of Mathematical Statistics, 2011.
Articolo in rivista - Articolo scientifico
Closed testing, confidence set, false discovery proportion
English
584
597
14
Goeman, J., Solari, A. (2011). Multiple Testing for Exploratory Research. STATISTICAL SCIENCE, 26(4), 584-597 [10.1214/11-STS356].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/38572
Citazioni
  • Scopus 77
  • ???jsp.display-item.citation.isi??? 74
Social impact