Simple Stepwise Tests of Hypotheses and Multiple Comparisons

Abstract

Holm's (1979) step-down and Hochberg's (1988) step-up procedures for tests of multiple hypotheses are simple to apply and are widely used. Holm's procedure controls the familywise error rate (FWE), while Hochberg's is more powerful. This paper investigates a step-down procedure (labelled CS) of Seneta & Chen (1997) which is a sharpening of Holm's, takes into account the degree of association between test statistics, and also controls the FWE. Computation for the CS procedure may be minimized by using the procedure as an adjustment to Holm's. The computational steps are detailed, and the adjustment is then illustrated by an application to a text-book example of multiple comparisons, in which step-wise procedures are shown to perform better than the usual Tukey T-comparison. Simulation investigations in a standard comparison with a control setting show that the CS step-down procedure is more powerful than Hochberg's step-up procedure and the procedure of Simes (1986), especially in regard to error rate, and not much less powerful than an optimal, but very specific, step-up procedure of Dunnett & Tamhane (1992).