## Statistical Science

### Recent Developments in Bootstrap Methodology

#### Abstract

Ever since its introduction, the bootstrap has provided both a powerful set of solutions for practical statisticians, and a rich source of theoretical and methodological problems for statistics. In this article, some recent developments in bootstrap methodology are reviewed and discussed. After a brief introduction to the bootstrap, we consider the following topics at varying levels of detail: the use of bootstrapping for highly accurate parametric inference; theoretical properties of nonparametric bootstrapping with unequal probabilities; subsampling and the m out of n bootstrap; bootstrap failures and remedies for superefficient estimators; recent topics in significance testing; bootstrap improvements of unstable classifiers and resampling for dependent data. The treatment is telegraphic rather than exhaustive.

#### Article information

Source
Statist. Sci., Volume 18, Issue 2 (2003), 141-157.

Dates
First available in Project Euclid: 19 September 2003

https://projecteuclid.org/euclid.ss/1063994969

Digital Object Identifier
doi:10.1214/ss/1063994969

Mathematical Reviews number (MathSciNet)
MR2026076

Zentralblatt MATH identifier
1331.62179

#### Citation

Davison, A. C.; Hinkley, D. V.; Young, G. A. Recent Developments in Bootstrap Methodology. Statist. Sci. 18 (2003), no. 2, 141--157. doi:10.1214/ss/1063994969. https://projecteuclid.org/euclid.ss/1063994969

#### References

• Baggerly, K. A. (1998). Empirical likelihood as a goodness-of-fit measure. Biometrika 85 535--547.
• Barndorff-Nielsen, O. E. (1980). Conditionality resolutions. Biometrika 67 293--310.
• Barndorff-Nielsen, O. E. (1983). On a formula for the distribution of the maximum likelihood estimator. Biometrika 70 343--365.
• Barndorff-Nielsen, O. E. (1986). Inference on full or partial parameters based on the standardized signed log likelihood ratio. Biometrika 73 307--322.
• Barndorff-Nielsen, O. E. and Cox, D. R. (1979). Edgeworth and saddle-point approximations with statistical applications (with discussion). J. Roy. Statist. Soc. Ser. B 41 279--312.
• Barndorff-Nielsen, O. E. and Cox, D. R. (1994). Inference and Asymptotics. Chapman and Hall, London.
• Beran R. J. (1986). Simulated power functions. Ann. Statist. 14 151--173.
• Beran, R. J. (1987). Prepivoting to reduce level error of confidence sets. Biometrika 74 457--468.
• Beran, R. J. (1988). Prepivoting test statistics: A bootstrap view of asymptotic refinements. J. Amer. Statist. Assoc. 83 687--697.
• Beran, R. J. (1995). Stein confidence sets and the bootstrap. Statist. Sinica 5 109--127.
• Beran, R. J. (1997). Diagnosing bootstrap success. Ann. Inst. Statist. Math. 49 l--24.
• Beran, R. J. (2003). The impact of the bootstrap on statistical algorithms and theory. Statist. Sci. 18 175--184 (this issue).
• Bickel, P. J. and Freedman, D. A. (1981). Some asymptotic theory for the bootstrap. Ann. Statist. 9 1196--1217.
• Bickel, P. J. and Ghosh, J. K. (1990). A decomposition for the likelihood ratio statistic and the Bartlett correction--- a Bayesian argument. Ann. Statist. 18 1070--1090.
• Bickel, P. J., Götze, F. and van Zwet, W. R. (1997). Resampling fewer than $n$ observations: Gains, losses, and remedies for losses. Statist. Sinica 7 1--32.
• Booth, J. G. and Hobert, J. P. (1998). Standard errors of prediction in generalized linear mixed models. J. Amer. Statist. Assoc. 93 262--272.
• Brazzale, A. R. (2000). Practical small-sample parametric inference. Ph.D. dissertation, Dept. Mathematics, Swiss Federal Institute of Technology, Lausanne.
• Breiman, L. (1996a). Heuristics of instability and stabilization in model selection. Ann. Statist. 24 2350--2383.
• Breiman, L. (1996b). Bagging predictors. Machine Learning 24 123--140.
• Bretagnolle, J. (1983). Lois limites du bootstrap de certaines fonctionelles. Ann. Inst. H. Poincaré Probab. Statist. 19 281--296.
• Brumback, B. A. and Rice, J. A. (1998). Smoothing spline models for the analysis of nested and crossed samples of curves (with discussion). J. Amer. Statist. Assoc. 93 961--994.
• Bühlmann, P. (2002a). Bootstraps for time series. Statist. Sci. 17 52--72.
• Bühlmann, P. (2002b). Sieve bootstrap with variable-length Markov chains for stationary categorical time series (with discussion). J. Amer. Statist. Assoc. 97 443--471.
• Bühlmann, P. and Yu, B. (2002). Analyzing bagging. Ann. Statist. 30 927--961.
• Canty, A. J., Davison, A. C., Hinkley, D. V. and Ventura, V. (2002). Bootstrap diagnostics. Preprint, Institute of Mathematics, Swiss Federal Institute of Technology, Lausanne.
• Carpenter, J. (1999). Test inversion bootstrap confidence intervals. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 159--172.
• Corcoran, S. A. (1998). Bartlett adjustment of empirical discrepancy statistics. Biometrika 85 967--972.
• Cox, D. R. (1980). Local ancillarity. Biometrika 67 279--286.
• Davison, A. C. (2003). Statistical Models. Cambridge Univ. Press.
• Davison, A. C. and Hinkley, D. V. (1997). Bootstrap Methods and Their Application. Cambridge Univ. Press.
• Delgado, M. A. and González Manteiga, W. (2001). Significance testing in nonparametric regression based on the bootstrap. Ann. Statist. 29 1469--1507.
• DiCiccio, T. J. and Efron, B. (1992). More accurate confidence intervals in exponential families. Biometrika 79 231--245.
• DiCiccio, T. J. and Efron, B. (1996). Bootstrap confidence intervals (with discussion). Statist. Sci. 11 189--228.
• DiCiccio, T. J., Martin, M. A. and Stern, S. E. (2001). Simple and accurate one-sided inference from signed roots of likelihood ratios. Canad. J. Statist. 29 67--76.
• DiCiccio, T. J. and Romano, J. P. (1990). Nonparametric confidence limits by resampling methods and least favorable families. Internat. Statist. Rev. 58 59--76.
• DiCiccio, T. J. and Romano, J. P. (1995). On bootstrap procedures for second-order accurate confidence limits in parametric models. Statist. Sinica 5 141--160.
• Durbin, J. (1980). Approximations for densities of sufficient estimators. Biometrika 67 311--333.
• Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Ann. Statist. 7 1--26.
• Efron, B. (1983). Estimating the error rate of a prediction rule: Improvement on cross-validation. J. Amer. Statist. Assoc. 78 316--331.
• Efron, B. (1986). How biased is the apparent error rate of a prediction rule? J. Amer. Statist. Assoc. 81 461--470.
• Efron, B. (1987). Better bootstrap confidence intervals (with discussion). J. Amer. Statist. Assoc. 82 171--200.
• Efron, B. and Hinkley, D. V. (1978). Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information (with discussion). Biometrika 65 457--487.
• Efron, B. and Tibshirani, R. J. (1993). An Introduction to the Bootstrap. Chapman and Hall, New York.
• Efron, B. and Tibshirani, R. J. (1997). Improvements on cross-validation: The $.632+$ bootstrap method. J. Amer. Statist. Assoc. 92 548--560.
• Efron, B. and Tibshirani, R. J. (1998). The problem of regions. Ann. Statist. 26 1687--1718.
• Fan, J. and Lin, S. (1998). Test of significance when data are curves. J. Amer. Statist. Assoc. 93 1007--1021.
• Freund, Y. and Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. System Sci. 55 119--139.
• Garthwaite, P. H. and Buckland, S. T. (1992). Generating Monte Carlo confidence intervals by the Robbins--Monro process. Appl. Statist. 41 159--171.
• Hall, P. (1985). Resampling a coverage pattern. Stochastic Process. Appl. 20 231--246.
• Hall, P. (1986). On the bootstrap and confidence intervals. Ann. Statist. 14 1431--1452.
• Hall, P. (1992). The Bootstrap and Edgeworth Expansion. Springer, New York.
• Hall, P. and Presnell, B. (1999a). Intentionally biased bootstrap methods. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 143--158.
• Hall, P. and Presnell, B. (1999b). Biased bootstrap methods for reducing the effects of contamination. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 661--680.
• Hall, P. and Presnell, B. (1999c). Density estimation under constraints. J. Comput. Graph. Statist. 8 259--277.
• Hall, P. and Wilson, S. R. (1991). Two guidelines for bootstrap hypothesis testing. Biometrics 47 757--762.
• Härdle, W. (1989). Resampling for inference from curves. Bull. Inst. Internat. Statist. 53 53--64.
• Härdle, W. (1990). Applied Nonparametric Regression. Cambridge Univ. Press.
• Hesterberg, T. C. (1999). Bootstrap tilting confidence intervals and hypothesis tests. In Computer Science and Statistics: Proc. 31st Symposium on the Interface 389--393. Interface Foundation of North America, Inc., Fairfax Station, VA.
• Hinkley, D. V. (1980). Likelihood as approximate pivotal distribution. Biometrika 67 287--292.
• Huang, H. (2002). Scenario generation for multivariate series data using the nearest neighbor bootstrap. Ph.D. dissertation, Dept. Decision Sciences and Engineering, Rensselaer Polytechnic Institute, Troy, New York.
• Lahiri, S. N. (2003). On the impact of bootstrap in survey sampling and small-area estimation. Statist. Sci. 18 199--210 (this issue).
• Lawless, J. (1982). Statistical Models and Methods for Lifetime Data. Wiley, New York.
• Lee, S. M. S. and Young, G. A. (2003). Prepivoting by weighted bootstrap iteration. Biometrika 90 393--410.
• Lee, Y. D. and Lahiri, S. N. (2002). Least squares variogram fitting by spatial subsampling. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 837--854.
• Liu, R. Y. and Singh, K. (1997). Notions of limiting $P$ values based on data depth and bootstrap. J. Amer. Statist. Assoc. 92 266--277.
• Mammen, E. (1993). Bootstrap and wild bootstrap for high-dimensional linear models. Ann. Statist. 21 255--285.
• Martin, M. A. (1990). On bootstrap iteration for coverage correction in confidence intervals. J. Amer. Statist. Assoc. 85 1105--1118.
• McCullagh, P. (2000). Resampling and exchangeable arrays. Bernoulli 6 285--301.
• Newton, M. A. and Geyer, C. J. (1994). Bootstrap recycling: A Monte Carlo alternative to the nested bootstrap. J. Amer. Statist. Assoc. 89 905--912.
• Owen, A. B. (1988). Empirical likelihood ratio confidence intervals for a single functional. Biometrika 75 237--249.
• Owen, A. B. (2001). Empirical Likelihood. Chapman and Hall/CRC, Boca Raton, FL.
• Politis, D. N. (2003). The impact of bootstrap methods on time series analysis. Statist. Sci. 18 219--230 (this issue).
• Politis, D. N., Paparoditis, E. and Romano, J. P. (1999). Resampling marked point processes. In Multivariate Analysis, Design of Experiments, and Survey Sampling (S. Ghosh, ed.) 163--185. Dekker, New York.
• Politis, D. N., Romano, J. P. and Wolf, M. (1999). Subsampling. Springer, New York.
• Putter, H. and van Zwet, W. R. (1996). Resampling: Consistency of substitution estimators. Ann. Statist. 24 2297--2318.
• Rajagopalan, B. and Lall, U. (1999). A $k$-nearest-neighbor simulator for daily precipitation and other weather variables. Water Resources Res. 35 3089--3101.
• Samworth, R. J. (2003). A note on methods of restoring consistency to the bootstrap. Biometrika. To appear.
• Schapire, R. E., Freund, Y., Bartlett, P. and Lee, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Ann. Statist. 26 1651--1686.
• Severini, T. A. (2000). Likelihood Methods in Statistics. Clarendon, Oxford.
• Shao, J. (2003). Impact of the bootstrap on sample surveys. Statist. Sci. 18 191--198 (this issue).
• Shao, J. and Tu, D. (1995). The Jackknife and Bootstrap. Springer, New York.
• Singh, K. (1981). On the asymptotic accuracy of Efron's bootstrap. Ann. Statist. 91 1187--1195.
• Stute, W., González Manteiga, W. and Presedo Quindimil, M. (1998). Bootstrap approximations in model checks for regression. J. Amer. Statist. Assoc. 93 141--149.
• Ventura, V. (2002). Non-parametric bootstrap recycling. Statist. Comput. 12 261--273.
• Wang, Y. D. and Wahba, G. (1995). Bootstrap confidence intervals for smoothing splines and their comparison to Bayesian confidence intervals. J. Statist. Comput. Simulation 51 263--279.
• Wu, C.-F. J. (1986). Jackknife, bootstrap and other resampling methods in regression analysis (with discussion). Ann. Statist. 14 1261--1350.