Open Access
February 2013 An algorithm to compute the power of Monte Carlo tests with guaranteed precision
Axel Gandy, Patrick Rubin-Delanchy
Ann. Statist. 41(1): 125-142 (February 2013). DOI: 10.1214/12-AOS1076


This article presents an algorithm that generates a conservative confidence interval of a specified length and coverage probability for the power of a Monte Carlo test (such as a bootstrap or permutation test). It is the first method that achieves this aim for almost any Monte Carlo test. Previous research has focused on obtaining as accurate a result as possible for a fixed computational effort, without providing a guaranteed precision in the above sense. The algorithm we propose does not have a fixed effort and runs until a confidence interval with a user-specified length and coverage probability can be constructed. We show that the expected effort required by the algorithm is finite in most cases of practical interest, including situations where the distribution of the $p$-value is absolutely continuous or discrete with finite support. The algorithm is implemented in the R-package simctest, available on CRAN.


Download Citation

Axel Gandy. Patrick Rubin-Delanchy. "An algorithm to compute the power of Monte Carlo tests with guaranteed precision." Ann. Statist. 41 (1) 125 - 142, February 2013.


Published: February 2013
First available in Project Euclid: 5 March 2013

zbMATH: 1347.62011
MathSciNet: MR3059412
Digital Object Identifier: 10.1214/12-AOS1076

Primary: 62-04 , 62L12
Secondary: 62F40 , 62L15

Keywords: algorithm , Monte Carlo testing , power , significance test

Rights: Copyright © 2013 Institute of Mathematical Statistics

Vol.41 • No. 1 • February 2013
Back to Top