Institute of Mathematical Statistics Lecture Notes - Monograph Series

Power and sample size comparisons of stepwise FWE and FDR controlling test procedures in the normal many-one case

Manfred Horn and Charles W. Dunnett

Full-text: Open access


Our paper compares the any-pair power, all-pairs power and per-pair power of several test procedures that control either the familywise error rate (FWE) or the false discovery rate (FDR). Our investigations are restricted to one-sided many-one comparisons of normal distributions. Some of the investigated procedures make use of this known special structure and some do not. The numbers $k$ of hypotheses we considered were 5, 10 and 100. The statements with small and large $k$ were quite similar. We found that all methods except one do not essentially differ in their any-pair powers. The most remarkable differences between the test procedures can be observed concerning their all-pairs powers.

We also compared the sample sizes that are necessary for the different procedures to ensure a specified power and investigated their dependency on the number $k$ of hypotheses. With specified per-pair or any-pair power, the FWE controlling methods need similar sample sizes as the FDR controlling methods. With specified all-pairs power, the sample sizes for four of the six FDR controlling procedures studied increase with $k$ at noticeably lower rates than for the FWE controlling procedures.

Chapter information

Y. Benjamini, F. Bretz and S. Sarkar, eds., Recent Developments in Multiple Comparison Procedures (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2004), 48-64

First available in Project Euclid: 28 November 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F03: Hypothesis testing
Secondary: 62J15: Paired and multiple comparisons

multiple comparison procedures false discovery rate familywise error rate any-pair power all-pairs power per-pair power average power sample size comparisons with control

Copyright © 2004, Institute of Mathematical Statistics


Horn, Manfred; Dunnett, Charles W. Power and sample size comparisons of stepwise FWE and FDR controlling test procedures in the normal many-one case. Recent Developments in Multiple Comparison Procedures, 48--64, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2004. doi:10.1214/lnms/1196285625.

Export citation