The Annals of Statistics

A general Bahadur representation of M-estimators and its application to linear regression with nonstochastic designs

Xuming He and Qi-Man Shao

Full-text: Open access


We obtain strong Bahadur representations for a general class of M-estimators that satisfies $\Sigma_i \psi (x_i, \theta) = o(\delta_n)$, where the $x_i$'s are independent but not necessarily identically distributed random variables. The results apply readily to M-estimators of regression with nonstochastic designs. More specifically, we consider the minimum $L_p$ distance estimators, bounded influence GM-estimators and regression quantiles. Under appropriate design conditions, the error ratesobtained for the first-order approximations are sharp in these cases. We also provide weaker and more easily verifiable conditions that suffice for an error rate that is suboptimal but strong enough for deriving the asymptotic distribution of M-estimators in a wide variety of problems.

Article information

Ann. Statist., Volume 24, Number 6 (1996), 2608-2630.

First available in Project Euclid: 16 September 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F12: Asymptotic properties of estimators 62J05: Linear regression
Secondary: 60F15: Strong theorems 62F35: Robustness and adaptive procedures

Asymptotic approximation Bahadur representation $M$-estimator linear regression minimum $L_p$-distance estimators


He, Xuming; Shao, Qi-Man. A general Bahadur representation of M -estimators and its application to linear regression with nonstochastic designs. Ann. Statist. 24 (1996), no. 6, 2608--2630. doi:10.1214/aos/1032181172.

Export citation


  • Babu, G. J. (1989). Strong representation for LAD estimators in linear models. Probab. Theory Related Fields 83 547-558.
  • Bahadur, R. R. (1966). A note on quantiles in large samples. Ann. Math. Statist. 37 577-581.
  • Bai, Z. D., Rao, C. R. and Wu, Y. (1992). M-estimation of multivariate linear regression parameters under a convex discrepancy function. Statist. Sinica 2 237-254.
  • Bloomfield, P. and Steiger, W. L. (1983). Least Absolute Deviation: Theory, Applications and Algorithms. Birkh¨auser, Boston.
  • Bose, A. (1996). Bahadur representation of M-estimates based on U functionals. Technical Report 96-2, Dept. Statistics, Purdue Univ.
  • Carroll, R. J. (1978). On almost sure expansions for M-estimates. Ann. Statist. 6 314-318.
  • Chen, X. (1993). On the law of the iterated logarithm for independent Banach space valued random variables. Ann. Probab. 21 1991-2011.
  • Freedman, D. (1975). On tail probability for martingales. Ann. Probab. 3 100-118.
  • Haberman, S. J. (1989). Convexity and estimation. Ann. Statist. 17 1631-1661.
  • Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J. and Stahel, W. A. (1986). Robust Statistics: The Approach Based on Influence Functions. Wiley, New York.
  • He, X. and Shao, Q. M. (1994). A strong Bahadur representation theorem of M-estimators. Research Report 590, Dept. Mathematics, National Univ. Singapore.
  • He, X. and Wang, G. (1995). Law of the iterated logarithm and invariance principle for Mestimators. Proc. Amer. Math. Soc. 123 563-573.
  • H ¨ossjer, O. (1994). Rank-based estimates in the linear model with high breakdown point. J. Amer. Statist. Assoc. 89 149-158.
  • Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proc. Fifth Berkeley Sy mp. Math. Statist. Probab. 1 221-233. Univ. California Press, Berkeley.
  • Jure ckov´a, J. (1985). Representation of M-estimators with the second-order asy mptotic distribution. Statist. Decisions 3 263-276.
  • Jure ckov´a, J. and Sen, P. K. (1987). A second-order asy mptotic distributional representation of M-estimators with discontinuous score functions. Ann. Probab. 15 814-823.
  • Kiefer, J. (1967). On Bahadur representation of sample quantiles. Ann. Math. Statist. 38 1323- 1342.
  • Koenker, R. and Bassett, G. (1978). Regression quantiles. Econometrica 46 33-50.
  • Liese, F. and Vajda, I. (1994). Consistency of M-estimates in general regression models. J. Multivariate Anal. 50 93-114.
  • Martinsek, A. (1989). Almost sure expansion for M-estimators and S-estimators in regression. Technical Report 25, Dept. Statistics, Univ. Illinois.
  • Niemiro, W. (1992). Asy mptotics for M-estimators defined by convex minimization. Ann. Statist. 20 1514-1533.
  • Pollard, D. (1991). Asy mptotics for least absolute deviation regression estimators. Econometric Theory 7 186-199.
  • Portnoy, S. and Koenker, R. (1989). Adaptive L-estimation for linear models. Ann. Statist. 17 362-381.
  • Serfling, R. J. (1980). Approximation Theorems of Mathematical Statistics. Wiley, New York.
  • Sp¨ath, H. (1991). Mathematical Algorithms for Linear Regression. Academic Press, New York.
  • Watson, G. A. (1980). Approximation Theory and Numerical Methods. Wiley, New York.
  • Welsh, A. (1989). On M-process and M-estimation. Ann. Statist. 17 337-362.
  • Wittmann, R. (1987). Sufficient moment and truncated moment conditions for the law of the iterated logarithm. Probab. Theory Related Fields 75 509-530.
  • Yohai, V. J. and Maronna, R. A (1979). Asy mptotic behavior of M-estimators for the linear model. Ann. Statist. 7 258-268.