The Annals of Statistics

Asymptotic comparison of step-down and step-up multiple test procedures based on exchangeable test statistics

H. Finner and M. Roters

Full-text: Open access


In this paper interest is focused on some theoretical investigations concerning the comparison of two popular multiple test procedures, so-called step-down and step-up procedures, in terms of their defining critical values. Such procedures can be applied, for example, to multiple comparisons with a control. In the definition of the critical values for these procedures order statistics play a central role. For $k \epsilon N_0$ fixed we consider the joint cumulative distribution function (cdf) $P(Y_{1:n} \leq c_1,\dots, Y_{n-k:n} \leq c_{n-k})$ of the first $n - k$ order statistics and the cdf$P(Y_{n-k:n} \leq c_{n-k})$ of the ($k + 1$)th largest order statistic $Y_{n-k:n}$ of $n$ random variables $Y_1,\dots, Y_n$ belonging to a sequence of exchangeable real-valued random variables. Our interest is focused on the asymptotic behavior of these cdf's and their interrelation if $n$ tends to $\infty$. It turns out that they sometimes behave completely differently compared with the iid case treated in Finner and Roters so that positive results are only possible under additional assumptions concerning the underlying distribution. We consider different sets of assumptions which then allow analogous results for the exchangeable case. Recently, Dalal and Mallows derived a result concerning the monotonicity of a certain set of critical values in connection with the joint cdf of order statistics in the iid case. We give a counterexample for the exchangeable case underlining the difficulties occurring in this situation. As an application we consider the comparison of certain step-down and step-up procedures in multiple comparisons with a control. The results of this paper yield a more theoretical explanation of the superiority of the step-up procedure which has been observed earlier by comparing tables of critical values. As a by-product we are able to quantify the tightness of the Bonferroni inequality in connection with maximum statistics.

Article information

Ann. Statist., Volume 26, Number 2 (1998), 505-524.

First available in Project Euclid: 31 July 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G30: Order statistics; empirical distribution functions 62J15: Paired and multiple comparisons
Secondary: 60F99: None of the above, but in this section 62F05: Asymptotic properties of tests

Bonferroni inequality comparisons with a control exchangeable random variables extreme order statistic familywise error rate joint distribution of order statistics many-one problem maximum statistic multiple comparisons multiple level step-down test step-up test


Finner, H.; Roters, M. Asymptotic comparison of step-down and step-up multiple test procedures based on exchangeable test statistics. Ann. Statist. 26 (1998), no. 2, 505--524. doi:10.1214/aos/1028144847.

Export citation


  • Dalal, S. R. and Mallows, C. L. (1992). Buying with exact confidence. Ann. Appl. Probab. 2 752-765.
  • Dunnett, C. W. and Tamhane, A. C. (1992). A step-up multiple test procedure. J. Amer. Statist. Assoc. 87 162-170.
  • Finner, H. and Giani, G. (1996). Duality between multiple testing and selecting. J. Statist. Plann. Inference 54 201-227.
  • Finner, H., Hay ter, A. J. and Roters, M. (1993). On the joint distribution function of order statistics with reference to step-up multiple test procedures. Technical Report 93-19, Univ. Trier.
  • Finner, H. and Roters, M. (1994). On the limit behaviour of the joint distribution function of order statistics. Ann. Inst. Statist. Math. 46 343-349.
  • Finner, H. and Roters, M. (1997). Asy mptotic comparison of the critical values of step-down and step-up multiple comparison procedures. To appear in J. Statist. Plann. Inference.
  • Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75 800-802.
  • Leadbetter, M. R., Lindgren, G. and Rootz´en, H. (1983). Extremes and Related Properties of Random Sequences and Processes. Springer, New York.
  • Marcus, R., Peritz, E. and Gabriel, K. R. (1976). On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63 655-660.
  • Rom, D. M. (1990). A sequentially rejective test procedure based on a modified Bonferroni inequality. Biometrika 77 663-665.
  • Sarkar, S. K. and Chang, C.-K. (1997). The Simes method for multiple hy pothesis testing with positively dependent test statistics. J. Amer. Statist. Assoc. 92 1601-1608.
  • Tong, Y. L. (1990). The Multivariate Normal Distribution. Springer, New York.
  • Welsch, R. E. (1977). Stepwise multiple comparison procedures. J. Amer. Statist. Assoc. 72 566-575.