A new multiple testing method in the dependent case

Abstract

The most popular multiple testing procedures are stepwise procedures based on P-values for individual test statistics. Included among these are the false discovery rate (FDR) controlling procedures of Benjamini–Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289–300] and their offsprings. Even for models that entail dependent data, P-values based on marginal distributions are used. Unlike such methods, the new method takes dependency into account at all stages. Furthermore, the P-value procedures often lack an intuitive convexity property, which is needed for admissibility. Still further, the new methodology is computationally feasible. If the number of tests is large and the proportion of true alternatives is less than say 25 percent, simulations demonstrate a clear preference for the new methodology. Applications are detailed for models such as testing treatments against control (or any intraclass correlation model), testing for change points and testing means when correlation is successive.