We consider the class of all multiple testing methods controlling tail probabilities of the false discovery proportion, either for one random set or simultaneously for many such sets. This class encompasses methods controlling familywise error rate, generalized familywise error rate, false discovery exceedance, joint error rate, simultaneous control of all false discovery proportions, and others, as well as gene set testing in genomics and cluster inference in neuroimaging. We show that all such methods are either equivalent to a closed testing procedure, or are uniformly improved by one. Moreover, we show that a closed testing method is admissible if and only if all its local tests are admissible. This implies that, when designing methods, it is sufficient to restrict attention to closed testing. We demonstrate the practical usefulness of this design principle by obtaining more informative inferences from the method of higher criticism, and by constructing a uniform improvement of a recently proposed method.

Goeman, J., Hemerik, J., Solari, A. (2021). Only closed testing procedures are admissible for controlling false discovery proportions. ANNALS OF STATISTICS, 49(2), 1218-1238 [10.1214/20-AOS1999].

Only closed testing procedures are admissible for controlling false discovery proportions

Solari A.
2021

Abstract

We consider the class of all multiple testing methods controlling tail probabilities of the false discovery proportion, either for one random set or simultaneously for many such sets. This class encompasses methods controlling familywise error rate, generalized familywise error rate, false discovery exceedance, joint error rate, simultaneous control of all false discovery proportions, and others, as well as gene set testing in genomics and cluster inference in neuroimaging. We show that all such methods are either equivalent to a closed testing procedure, or are uniformly improved by one. Moreover, we show that a closed testing method is admissible if and only if all its local tests are admissible. This implies that, when designing methods, it is sufficient to restrict attention to closed testing. We demonstrate the practical usefulness of this design principle by obtaining more informative inferences from the method of higher criticism, and by constructing a uniform improvement of a recently proposed method.
Articolo in rivista - Articolo scientifico
Familywise error rate; Higher criticism; Multiple testing; Selective inference; Simultaneous inference;
English
2-apr-2021
2021
49
2
1218
1238
none
Goeman, J., Hemerik, J., Solari, A. (2021). Only closed testing procedures are admissible for controlling false discovery proportions. ANNALS OF STATISTICS, 49(2), 1218-1238 [10.1214/20-AOS1999].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/320528
Citazioni
  • Scopus 13
  • ???jsp.display-item.citation.isi??? 16
Social impact