The Annals of Statistics

Bootstrap consistency for general semiparametric M-estimation

Guang Cheng and Jianhua Z. Huang

Full-text: Open access


Consider M-estimation in a semiparametric model that is characterized by a Euclidean parameter of interest and an infinite-dimensional nuisance parameter. As a general purpose approach to statistical inferences, the bootstrap has found wide applications in semiparametric M-estimation and, because of its simplicity, provides an attractive alternative to the inference approach based on the asymptotic distribution theory. The purpose of this paper is to provide theoretical justifications for the use of bootstrap as a semiparametric inferential tool. We show that, under general conditions, the bootstrap is asymptotically consistent in estimating the distribution of the M-estimate of Euclidean parameter; that is, the bootstrap distribution asymptotically imitates the distribution of the M-estimate. We also show that the bootstrap confidence set has the asymptotically correct coverage probability. These general conclusions hold, in particular, when the nuisance parameter is not estimable at root-n rate, and apply to a broad class of bootstrap methods with exchangeable bootstrap weights. This paper provides a first general theoretical study of the bootstrap in semiparametric models.

Article information

Ann. Statist., Volume 38, Number 5 (2010), 2884-2915.

First available in Project Euclid: 20 July 2010

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F40: Bootstrap, jackknife and other resampling methods
Secondary: 62G20: Asymptotic properties

Bootstrap consistency bootstrap confidence set semiparametric model M-estimation


Cheng, Guang; Huang, Jianhua Z. Bootstrap consistency for general semiparametric M -estimation. Ann. Statist. 38 (2010), no. 5, 2884--2915. doi:10.1214/10-AOS809.

Export citation


  • [1] Barbe, P. and Bertail, P. (1995). The Weighted Bootstrap. Lecture Notes in Statistics 98. Springer, New York.
  • [2] Banerjee, M., Mukherjee, D. and Mishra, S. (2009). Semiparametric binary regression models under shape constraints with an application to Indian schooling data. J. Econometrics 149 101–117.
  • [3] Bickel, P. J. and Freedman, D. A. (1981). Some asymptotic theory for the bootstrap. Ann. Statist. 9 1196–1217.
  • [4] Bickel, P. J., Klaassen, C. A. J., Ritov, Y. and Wellner, J. A. (1998). Efficient and Adaptive Estimation for Semiparametric Models. Springer, New York.
  • [5] Chatterjee, S. and Bose, A. (2005). Generalized bootstrap for estimating equations. Ann. Statist. 33 414–436.
  • [6] Chen, X. and Pouzo, D. (2009). Efficient estimation of semiparametric conditional moment models with possibly nonsmooth residuals. J. Econometrics 152 46–60.
  • [7] Cheng, G. (2008). Semiparametric additive isotonic regression. J. Statist. Plann. Inference 100 345–362.
  • [8] Cheng, G. and Kosorok, M. (2008). Higher order semiparametric frequentist inference with the profile sampler. Ann. Statist. 36 1786–1818.
  • [9] Cheng, G. and Kosorok, M. (2008). General frequentist properties of the posterior profile distribution. Ann. Statist. 36 1819–1853.
  • [10] Delecroix, M., Hristache, M. and Patilea, V. (2006). On semiparametric M-estimation in single-index regression. J. Statist. Plann. Inference 136 730–769.
  • [11] Dixon, J., Kosorok, M. and Lee, B. L. (2005). Functional inference in semiparametric models using the piggyback bootstrap. Ann. Inst. Statist. Math. 57 255–277.
  • [12] Dudley, R. M. (1984). A Course on Empirical Processes. Lecture Notes in Math. 1097 2–142. Springer, Berlin.
  • [13] Efron, B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans. SIAM, Philadelphia.
  • [14] Efron, B. and Tibshirani, R. (1986). Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Statist. Sci. 1 54–75.
  • [15] Grenander, U. (1981). Abstract Inference. Wiley, New York.
  • [16] Gelman, A., Carlin, J., Stern, H. and Rubin, D. (2003). Bayesian Data Analysis, 2nd ed. Chapman and Hall, London.
  • [17] Gine, E. and Zinn, J. (1990). Bootstrapping general empirical functions. Ann. Probab. 18 851–869.
  • [18] Hall, P. (1992). The Bootstrap and Edgeworth Expansion. Springer, New York.
  • [19] Hardle, W., Huet, S., Mammen, E. and Sperlich, S. (2004). Bootstrap inference in semiparametric generalized additive models. Econometric Theory 20 265–300.
  • [20] Huang, J. (1999). Efficient estimation of the partly linear Cox model. Ann. Statist. 27 1536–1563.
  • [21] Kosorok, M., Lee, B. L. and Fine, J. P. (2004). Robust inference for univariate proportional hazards frailty regression models. Ann. Statist. 32 1448–1491.
  • [22] Kosorok, M. (2008). Introduction to Empirical Processes and Semiparametric Inference. Springer, New York.
  • [23] Kosorok, M. (2008). Boostrapping the Grenander estimator. In Beyond Parametrics in Interdisciplinary Research: Festschrift in Honor of Professor Pranab K. Sen. IMS Collections 1 282–292. IMS, Beachwood, OH.
  • [24] Lee, B. L., Kosorok, M. R. and Fine, J. P. (2005). The profile sampler. J. Amer. Statist. Assoc. 100 960–969.
  • [25] Lee, S. M. S. and Pun, M. C. (2006). On m out of n bootstrapping for nonstandard M-estimation with nuisance parameters. J. Amer. Statist. Assoc. 101 1185–1197.
  • [26] Liang, H., Härdle, W. and Sommerfeld, V. (2000). Bootstrap approximations in a partially linear regression model. J. Statist. Plann. Inference 91 413–426.
  • [27] Lo, A. Y. (1993). A Bayesian bootstrap for censored data. Ann. Statist. 21 100–123.
  • [28] Ma, S. and Kosorok, M. (2005). Robust semiparametric M-estimation and the weighted bootstrap. J. Multivariate Anal. 96 190–217.
  • [29] Ma, S. and Kosorok, M. (2005). Penalized log-likelihood estimation for partly linear transformation models with current status data. Ann. Statist. 33 2256–2290.
  • [30] Mason, D. and Newton, M. (1992). A rank statistic approach to the consistency of a general bootstrap. Ann. Statist. 20 1611–1624.
  • [31] Murphy, S. A. and van der Vaart, A. W. (1999). Observed information in semiparametric models. Bernoulli 5 381–412.
  • [32] Murphy, S. A. and van der Vaart, A. W. (2000). On profile likelihood. J. Amer. Statist. Assoc. 95 1461–1474.
  • [33] Praestgaard, J. and Wellner, J. (1993). Exchangeably weighted bootstraps of the general empirical process. Ann. Probab. 21 2053–2086.
  • [34] Rubin, D. (1981). The Bayesian bootstrap. Ann. Statist. 9 130–134.
  • [35] Sen, B., Banerjee, M. and Woodroofe, M. B. (2010). Inconsistency of bootstrap: The Grenander estimator. Ann. Statist. 38 1953–1977.
  • [36] Singh, K. (1981). On the asymptotic accuracy of Efron’s bootstrap. Ann. Statist. 9 1187–1195.
  • [37] Strawderman, R. (2006). A regression model for dependent gap times. Int. J. Biostat. 2 Article 1, 34 pp. (electronic).
  • [38] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
  • [39] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Univ. Press, Cambridge.
  • [40] van de Geer, S. (2000). Empirical Processes in M-Estimation. Cambridge Univ. Press, Cambridge.
  • [41] Wellner, J. A. and Zhan, Y. (1996). Bootstrapping Z-estimators. Technical Report 308, Univ. Washington.
  • [42] Wellner, J. A. and Zhang, Y. (2007). Two likelihood-based semiparametric estimation methods for panel count data with covariates. Ann. Statist. 35 2106–2142.
  • [43] Young, J. G., Jewell, N. P. and Samuels, S. J. (2008). Regression analysis of a disease onset distribution using diagnosis data. Biometrics 64 20–28.
  • [44] Zeng, D. L. and Lin, D. Y. (2007). Maximum likelihood estimation in semiparametric models with censored data (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 69 507–564.
  • [45] Zhang, C. M. and Yu, T. (2008). Semiparametric detection of significant activation for brain FMRI. Ann. Statist. 36 1693–1725.