Higher criticism: $p$-values and criticism

Abstract

This paper compares the higher criticism statistic (Donoho and Jin [ Ann. Statist. 32 (2004) 962–994]), a modification of the higher criticism statistic also suggested by Donoho and Jin, and two statistics of the Berk–Jones [ Z. Wahrsch. Verw. Gebiete 47 (1979) 47–59] type. New approximations to the significance levels of the statistics are derived, and their accuracy is studied by simulations. By numerical examples it is shown that over a broad range of sample sizes the Berk–Jones statistics have a better power function than the higher criticism statistics to detect sparse mixtures. The applications suggested by Meinshausen and Rice [ Ann. Statist. 34 (2006) 373–393], to find lower confidence bounds for the number of false hypotheses, and by Jeng, Cai and Li [ Biometrika 100 (2013) 157–172], to detect copy number variants, are also studied.