The Annals of Statistics

M-estimation of linear models with dependent errors

Wei Biao Wu

Full-text: Open access


We study asymptotic properties of M-estimates of regression parameters in linear models in which errors are dependent. Weak and strong Bahadur representations of the M-estimates are derived and a central limit theorem is established. The results are applied to linear models with errors being short-range dependent linear processes, heavy-tailed linear processes and some widely used nonlinear time series.

Article information

Ann. Statist., Volume 35, Number 2 (2007), 495-521.

First available in Project Euclid: 5 July 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J05: Linear regression
Secondary: 60F05: Central limit and other weak theorems

Robust estimation linear model dependence nonlinear time series


Wu, Wei Biao. M -estimation of linear models with dependent errors. Ann. Statist. 35 (2007), no. 2, 495--521. doi:10.1214/009053606000001406.

Export citation


  • Andersen, P. K. and Gill, R. D. (1982). Cox's regression model for counting processes: A large sample study. Ann. Statist. 10 1100--1120.
  • Arcones, M. A. (1996). The Bahadur--Kiefer representation of $L\sb p$ regression estimators. Econometric Theory 12 257--283.
  • Arcones, M. A. (1998). Second order representations of the least absolute deviation regression estimator. Ann. Inst. Statist. Math. 50 87--117.
  • Babu, G. J. (1989). Strong representations for LAD estimators in linear models. Probab. Theory Related Fields 83 547--558.
  • Babu, G. J. and Singh, K. (1978). On deviations between empirical and quantile processes for mixing random variables. J. Multivariate Anal. 8 532--549.
  • Bahadur, R. R. (1966). A note on quantiles in large samples. Ann. Math. Statist. 37 577--580.
  • Bai, Z. D., Rao, C. R. and Wu, Y. (1992). $M$-estimation of multivariate linear regression parameters under a convex discrepancy function. Statist. Sinica 2 237--254.
  • Bassett, G. and Koenker, R. (1978). Asymptotic theory of least absolute error regression. J. Amer. Statist. Assoc. 73 618--622.
  • Berlinet, A., Liese, F. and Vajda, I. (2000). Necessary and sufficient conditions for consistency of $M$-estimates in regression models with general errors. J. Statist. Plann. Inference 89 243--267.
  • Bloomfield, P. and Steiger, W. L. (1983). Least Absolute Deviations. Theory, Applications and Algorithms. Birkhäuser, Boston.
  • Carroll, R. J. (1978). On almost sure expansions for $M$-estimates. Ann. Statist. 6 314--318.
  • Chen, X. R., Bai, Z. D., Zhao, L. and Wu, Y. (1990). Asymptotic normality of minimum $L\sb 1$-norm estimates in linear models. Sci. China Ser. A 33 1311--1328.
  • Cui, H., He, X. and Ng, K. W. (2004). $M$-estimation for linear models with spatially-correlated errors. Statist. Probab. Lett. 66 383--393.
  • Davis, R. A., Knight, K. and Liu, J. (1992). $M$-estimation for autoregressions with infinite variance. Stochastic Process. Appl. 40 145--180.
  • Davis, R. A. and Wu, W. (1997). $M$-estimation for linear regression with infinite variance. Probab. Math. Statist. 17 1--20.
  • Diaconis, P. and Freedman, D. (1999). Iterated random functions. SIAM Rev. 41 45--76.
  • Doukhan, P. (1994). Mixing. Properties and Examples. Lecture Notes in Statist. 85. Springer, New York.
  • Eicker, F. (1963). Asymptotic normality and consistency of the least squares estimators for families of linear regressions. Ann. Math. Statist. 34 447--456.
  • Freedman, D. A. (1975). On tail probabilities for martingales. Ann. Probab. 3 100--118.
  • Gastwirth, J. L. and Rubin, H. (1975). The behavior of robust estimators on dependent data. Ann. Statist. 3 1070--1100.
  • Gleser, L. J. (1965). On the asymptotic theory of fixed-size sequential confidence bounds for linear regression parameters. Ann. Math. Statist. 36 463--467.
  • Gorodetskii, V. V. (1977). On the strong mixing property for linear sequences. Theory Probab. Appl. 22 411--412.
  • Haberman, S. J. (1989). Concavity and estimation. Ann. Statist. 17 1631--1661.
  • Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J. and Stahel, W. A. (1986). Robust Statistics. The Approach Based on Influence Functions. Wiley, New York.
  • Hannan, E. J. (1973). Central limit theorems for time series regression. Z. Wahrsch. Verw. Gebiete 26 157--170.
  • He, X. and Shao, Q.-M. (1996). A general Bahadur representation of $M$-estimators and its application to linear regression with nonstochastic designs. Ann. Statist. 24 2608--2630.
  • Hesse, C. H. (1990). A Bahadur-type representation for empirical quantiles of a large class of stationary, possibly infinite-variance, linear processes. Ann. Statist. 18 1188--1202.
  • Hsing, T. (1999). On the asymptotic distributions of partial sums of functionals of infinite-variance moving averages. Ann. Probab. 27 1579--1599.
  • Huber, P. J. (1973). Robust regression: Asymptotics, conjectures and Monte Carlo. Ann. Statist. 1 799--821.
  • Huber, P. J. (1981). Robust Statistics. Wiley, New York.
  • Ibragimov, I. A. and Linnik, Yu. V. (1971). Independent and Stationary Sequences of Random Variables. Wolters-Noordhoff, Groningen.
  • Jurečková, J. and Sen, P. K. (1996). Robust Statistical Procedures. Wiley, New York.
  • Kallianpur, G. (1983). Some ramifications of Wiener's ideas on nonlinear prediction. In Norbert Wiener, Collected Works (P. Masani, ed.) 3 402--424. MIT Press, Cambridge, MA.
  • Koenker, R. (2005). Quantile Regression. Cambridge Univ. Press.
  • Koenker, R. and Bassett, G. (1978). Regression quantiles. Econometrica 46 33--50.
  • Koul, H. L. (1977). Behavior of robust estimators in the regression model with dependent errors. Ann. Statist. 5 681--699.
  • Koul, H. L. and Surgailis, D. (2000). Second-order behavior of $M$-estimators in linear regression with long-memory errors. J. Statist. Plann. Inference 91 399--412.
  • Lee, C.-H. and Martin, R. D. (1986). Ordinary and proper location $M$-estimates for autoregressive-moving average models. Biometrika 73 679--686.
  • Niemiro, W. (1992). Asymptotics for $M$-estimators defined by convex minimization. Ann. Statist. 20 1514--1533.
  • Pham, T. D. and Tran, L. T. (1985). Some mixing properties of time series models. Stochastic Process. Appl. 19 297--303.
  • Phillips, P. C. B. (1991). A shortcut to LAD estimator asymptotics. Econometric Theory 7 450--463.
  • Pollard, D. (1991). Asymptotics for least absolute deviation regression estimators. Econometric Theory 7 186--199.
  • Portnoy, S. L. (1977). Robust estimation in dependent situations. Ann. Statist. 5 22--43.
  • Portnoy, S. L. (1979). Further remarks on robust estimation in dependent situations. Ann. Statist. 7 224--231.
  • Prakasa Rao, B. L. S. (1981). Asymptotic behavior of $M$-estimators for the linear model with dependent errors. Bull. Inst. Math. Acad. Sinica 9 367--375.
  • Priestley, M. B. (1988). Nonlinear and Nonstationary Time Series Analysis. Academic Press, London.
  • Rao, C. R. and Zhao, L. C. (1992). Linear representations of $M$-estimates in linear models. Canad. J. Statist. 20 359--368.
  • Rockafellar, R. T. (1970). Convex Analysis. Princeton Univ. Press.
  • Ronner, A. E. (1984). Asymptotic normality of $p$-norm estimators in multiple regression. Z. Wahrsch. Verw. Gebiete 66 613--620.
  • Rosenblatt, M. (1959). Stationary processes as shifts of functions of independent random variables. J. Math. Mech. 8 665--681.
  • Rosenblatt, M. (1971). Markov Processes. Structure and Asymptotic Behavior. Springer, New York.
  • Solomyak, B. M. (1995). On the random series $\sum \pm \lambda^n$ (an Erdös problem). Ann. of Math. (2) 142 611--625.
  • Stine, R. A. (1997). Nonlinear time series. Encyclopedia of Statistical Sciences 430--437. Wiley, New York.
  • Surgailis, D. (2002). Stable limits of empirical processes of moving averages with infinite variance. Stochastic Process. Appl. 100 255--274.
  • Tong, H. (1990). Nonlinear Time Series. A Dynamical System Approach. Oxford Univ. Press.
  • Tsay, R. S. (2005). Analysis of Financial Time Series, 2nd ed. Wiley, Hoboken, NJ.
  • Welsh, A. H. (1986). Bahadur representations for robust scale estimators based on regression residuals. Ann. Statist. 14 1246--1251.
  • Welsh, A. H. (1989). On $M$-processes and $M$-estimation. Ann. Statist. 17 337--361.
  • Wiener, N. (1958). Nonlinear Problems in Random Theory. MIT Press, Cambridge, MA.
  • Withers, C. S. (1981). Conditions for linear process to be strongly mixing. Z. Wahrsch. Verw. Gebiete 57 477--480.
  • Wu, W. B. (2003). Additive functionals of infinite-variance moving averages. Statist. Sinica 13 1259--1267.
  • Wu, W. B. (2005). On the Bahadur representation of sample quantiles for dependent sequences. Ann. Statist. 33 1934--1963
  • Wu, W. B. (2005). Nonlinear system theory: Another look at dependence. Proc. Natl. Acad. Sci. USA 102 14,150--14,154.
  • Wu, W. B. (2006). $M$-estimation of linear models with dependent errors. Available at
  • Wu, W. B. and Shao, X. (2004). Limit theorems for iterated random functions. J. Appl. Probab. 41 425--436.
  • Yohai, V. J. (1974). Robust estimation in the linear model. Ann. Statist. 2 562--567.
  • Yohai, V. J. and Maronna, R. A. (1979). Asymptotic behavior of $M$-estimators for the linear model. Ann. Statist. 7 258--268.
  • Zeckhauser, R. and Thompson, M. (1970). Linear regression with non-normal error terms. Review Economics and Statistics 52 280--286.
  • Zhao, L. (2000). Some contributions to $M$-estimation in linear models. J. Statist. Plann. Inference 88 189--203.