The Annals of Statistics
- Ann. Statist.
- Volume 35, Number 2 (2007), 495-521.
M-estimation of linear models with dependent errors
Full-text: Open access
Abstract
We study asymptotic properties of M-estimates of regression parameters in linear models in which errors are dependent. Weak and strong Bahadur representations of the M-estimates are derived and a central limit theorem is established. The results are applied to linear models with errors being short-range dependent linear processes, heavy-tailed linear processes and some widely used nonlinear time series.
Article information
Source
Ann. Statist., Volume 35, Number 2 (2007), 495-521.
Dates
First available in Project Euclid: 5 July 2007
Permanent link to this document
https://projecteuclid.org/euclid.aos/1183667282
Digital Object Identifier
doi:10.1214/009053606000001406
Mathematical Reviews number (MathSciNet)
MR2336857
Zentralblatt MATH identifier
1117.62070
Subjects
Primary: 62J05: Linear regression
Secondary: 60F05: Central limit and other weak theorems
Keywords
Robust estimation linear model dependence nonlinear time series
Citation
Wu, Wei Biao. M -estimation of linear models with dependent errors. Ann. Statist. 35 (2007), no. 2, 495--521. doi:10.1214/009053606000001406. https://projecteuclid.org/euclid.aos/1183667282
References
- Andersen, P. K. and Gill, R. D. (1982). Cox's regression model for counting processes: A large sample study. Ann. Statist. 10 1100--1120. Mathematical Reviews (MathSciNet): MR0673646
Digital Object Identifier: doi:10.1214/aos/1176345976
Project Euclid: euclid.aos/1176345976
Zentralblatt MATH: 0526.62026 - Arcones, M. A. (1996). The Bahadur--Kiefer representation of $L\sb p$ regression estimators. Econometric Theory 12 257--283.
- Arcones, M. A. (1998). Second order representations of the least absolute deviation regression estimator. Ann. Inst. Statist. Math. 50 87--117. Mathematical Reviews (MathSciNet): MR1614207
Digital Object Identifier: doi:10.1023/A:1003401414732
Zentralblatt MATH: 0928.62047 - Babu, G. J. (1989). Strong representations for LAD estimators in linear models. Probab. Theory Related Fields 83 547--558. Mathematical Reviews (MathSciNet): MR1022629
Digital Object Identifier: doi:10.1007/BF01845702
Zentralblatt MATH: 0665.62033 - Babu, G. J. and Singh, K. (1978). On deviations between empirical and quantile processes for mixing random variables. J. Multivariate Anal. 8 532--549. Mathematical Reviews (MathSciNet): MR0520961
Digital Object Identifier: doi:10.1016/0047-259X(78)90031-3
Zentralblatt MATH: 0395.62039 - Bahadur, R. R. (1966). A note on quantiles in large samples. Ann. Math. Statist. 37 577--580. Mathematical Reviews (MathSciNet): MR0189095
Digital Object Identifier: doi:10.1214/aoms/1177699450
Project Euclid: euclid.aoms/1177699450 - Bai, Z. D., Rao, C. R. and Wu, Y. (1992). $M$-estimation of multivariate linear regression parameters under a convex discrepancy function. Statist. Sinica 2 237--254. Mathematical Reviews (MathSciNet): MR1152307
- Bassett, G. and Koenker, R. (1978). Asymptotic theory of least absolute error regression. J. Amer. Statist. Assoc. 73 618--622. Mathematical Reviews (MathSciNet): MR0514166
Digital Object Identifier: doi:10.2307/2286611
Zentralblatt MATH: 0391.62046 - Berlinet, A., Liese, F. and Vajda, I. (2000). Necessary and sufficient conditions for consistency of $M$-estimates in regression models with general errors. J. Statist. Plann. Inference 89 243--267. Mathematical Reviews (MathSciNet): MR1794424
Digital Object Identifier: doi:10.1016/S0378-3758(99)00218-9
Zentralblatt MATH: 0997.62050 - Bloomfield, P. and Steiger, W. L. (1983). Least Absolute Deviations. Theory, Applications and Algorithms. Birkhäuser, Boston.
- Carroll, R. J. (1978). On almost sure expansions for $M$-estimates. Ann. Statist. 6 314--318. Mathematical Reviews (MathSciNet): MR0464470
Digital Object Identifier: doi:10.1214/aos/1176344126
Project Euclid: euclid.aos/1176344126
Zentralblatt MATH: 0376.62034 - Chen, X. R., Bai, Z. D., Zhao, L. and Wu, Y. (1990). Asymptotic normality of minimum $L\sb 1$-norm estimates in linear models. Sci. China Ser. A 33 1311--1328. Mathematical Reviews (MathSciNet): MR1084956
- Cui, H., He, X. and Ng, K. W. (2004). $M$-estimation for linear models with spatially-correlated errors. Statist. Probab. Lett. 66 383--393. Mathematical Reviews (MathSciNet): MR2045131
- Davis, R. A., Knight, K. and Liu, J. (1992). $M$-estimation for autoregressions with infinite variance. Stochastic Process. Appl. 40 145--180. Mathematical Reviews (MathSciNet): MR1145464
Digital Object Identifier: doi:10.1016/0304-4149(92)90142-D - Davis, R. A. and Wu, W. (1997). $M$-estimation for linear regression with infinite variance. Probab. Math. Statist. 17 1--20. Mathematical Reviews (MathSciNet): MR1455606
- Diaconis, P. and Freedman, D. (1999). Iterated random functions. SIAM Rev. 41 45--76. Mathematical Reviews (MathSciNet): MR1669737
Digital Object Identifier: doi:10.1137/S0036144598338446
JSTOR: links.jstor.org
Zentralblatt MATH: 0926.60056 - Doukhan, P. (1994). Mixing. Properties and Examples. Lecture Notes in Statist. 85. Springer, New York. Mathematical Reviews (MathSciNet): MR1312160
- Eicker, F. (1963). Asymptotic normality and consistency of the least squares estimators for families of linear regressions. Ann. Math. Statist. 34 447--456. Mathematical Reviews (MathSciNet): MR0148177
Digital Object Identifier: doi:10.1214/aoms/1177704156
Project Euclid: euclid.aoms/1177704156 - Freedman, D. A. (1975). On tail probabilities for martingales. Ann. Probab. 3 100--118. Mathematical Reviews (MathSciNet): MR0380971
Digital Object Identifier: doi:10.1214/aop/1176996452
Zentralblatt MATH: 0313.60037
JSTOR: links.jstor.org - Gastwirth, J. L. and Rubin, H. (1975). The behavior of robust estimators on dependent data. Ann. Statist. 3 1070--1100. Mathematical Reviews (MathSciNet): MR0395039
Digital Object Identifier: doi:10.1214/aos/1176343241
Project Euclid: euclid.aos/1176343241
Zentralblatt MATH: 0359.62042 - Gleser, L. J. (1965). On the asymptotic theory of fixed-size sequential confidence bounds for linear regression parameters. Ann. Math. Statist. 36 463--467. Mathematical Reviews (MathSciNet): MR0172424
Digital Object Identifier: doi:10.1214/aoms/1177700157
Project Euclid: euclid.aoms/1177700157 - Gorodetskii, V. V. (1977). On the strong mixing property for linear sequences. Theory Probab. Appl. 22 411--412. Mathematical Reviews (MathSciNet): MR0438450
- Haberman, S. J. (1989). Concavity and estimation. Ann. Statist. 17 1631--1661. Mathematical Reviews (MathSciNet): MR1026303
Digital Object Identifier: doi:10.1214/aos/1176347385
Project Euclid: euclid.aos/1176347385
Zentralblatt MATH: 0699.62027 - Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J. and Stahel, W. A. (1986). Robust Statistics. The Approach Based on Influence Functions. Wiley, New York.
- Hannan, E. J. (1973). Central limit theorems for time series regression. Z. Wahrsch. Verw. Gebiete 26 157--170. Mathematical Reviews (MathSciNet): MR0331683
Digital Object Identifier: doi:10.1007/BF00533484
Zentralblatt MATH: 0246.62086 - He, X. and Shao, Q.-M. (1996). A general Bahadur representation of $M$-estimators and its application to linear regression with nonstochastic designs. Ann. Statist. 24 2608--2630. Mathematical Reviews (MathSciNet): MR1425971
Digital Object Identifier: doi:10.1214/aos/1032181172
Project Euclid: euclid.aos/1032181172
Zentralblatt MATH: 0867.62012 - Hesse, C. H. (1990). A Bahadur-type representation for empirical quantiles of a large class of stationary, possibly infinite-variance, linear processes. Ann. Statist. 18 1188--1202. Mathematical Reviews (MathSciNet): MR1062705
Digital Object Identifier: doi:10.1214/aos/1176347746
Project Euclid: euclid.aos/1176347746
Zentralblatt MATH: 0712.62042 - Hsing, T. (1999). On the asymptotic distributions of partial sums of functionals of infinite-variance moving averages. Ann. Probab. 27 1579--1599. Mathematical Reviews (MathSciNet): MR1733161
Digital Object Identifier: doi:10.1214/aop/1022677460
Project Euclid: euclid.aop/1022677460
Zentralblatt MATH: 0961.60038 - Huber, P. J. (1973). Robust regression: Asymptotics, conjectures and Monte Carlo. Ann. Statist. 1 799--821. Mathematical Reviews (MathSciNet): MR0356373
Digital Object Identifier: doi:10.1214/aos/1176342503
Project Euclid: euclid.aos/1176342503
Zentralblatt MATH: 0289.62033 - Huber, P. J. (1981). Robust Statistics. Wiley, New York.
- Ibragimov, I. A. and Linnik, Yu. V. (1971). Independent and Stationary Sequences of Random Variables. Wolters-Noordhoff, Groningen.
- Jurečková, J. and Sen, P. K. (1996). Robust Statistical Procedures. Wiley, New York. Mathematical Reviews (MathSciNet): MR1387346
- Kallianpur, G. (1983). Some ramifications of Wiener's ideas on nonlinear prediction. In Norbert Wiener, Collected Works (P. Masani, ed.) 3 402--424. MIT Press, Cambridge, MA.
- Koenker, R. (2005). Quantile Regression. Cambridge Univ. Press. Mathematical Reviews (MathSciNet): MR2268657
- Koenker, R. and Bassett, G. (1978). Regression quantiles. Econometrica 46 33--50. Mathematical Reviews (MathSciNet): MR0474644
Digital Object Identifier: doi:10.2307/1913643
JSTOR: links.jstor.org - Koul, H. L. (1977). Behavior of robust estimators in the regression model with dependent errors. Ann. Statist. 5 681--699. Mathematical Reviews (MathSciNet): MR0443213
Digital Object Identifier: doi:10.1214/aos/1176343892
Project Euclid: euclid.aos/1176343892
Zentralblatt MATH: 0358.62032 - Koul, H. L. and Surgailis, D. (2000). Second-order behavior of $M$-estimators in linear regression with long-memory errors. J. Statist. Plann. Inference 91 399--412. Mathematical Reviews (MathSciNet): MR1814792
Digital Object Identifier: doi:10.1016/S0378-3758(00)00190-7
Zentralblatt MATH: 0967.62040 - Lee, C.-H. and Martin, R. D. (1986). Ordinary and proper location $M$-estimates for autoregressive-moving average models. Biometrika 73 679--686.
- Niemiro, W. (1992). Asymptotics for $M$-estimators defined by convex minimization. Ann. Statist. 20 1514--1533. Mathematical Reviews (MathSciNet): MR1186263
Digital Object Identifier: doi:10.1214/aos/1176348782
Project Euclid: euclid.aos/1176348782
Zentralblatt MATH: 0786.62040 - Pham, T. D. and Tran, L. T. (1985). Some mixing properties of time series models. Stochastic Process. Appl. 19 297--303. Mathematical Reviews (MathSciNet): MR0787587
Digital Object Identifier: doi:10.1016/0304-4149(85)90031-6
Zentralblatt MATH: 0564.62068 - Phillips, P. C. B. (1991). A shortcut to LAD estimator asymptotics. Econometric Theory 7 450--463.
- Pollard, D. (1991). Asymptotics for least absolute deviation regression estimators. Econometric Theory 7 186--199.
- Portnoy, S. L. (1977). Robust estimation in dependent situations. Ann. Statist. 5 22--43. Mathematical Reviews (MathSciNet): MR0445716
Digital Object Identifier: doi:10.1214/aos/1176343738
Project Euclid: euclid.aos/1176343738 - Portnoy, S. L. (1979). Further remarks on robust estimation in dependent situations. Ann. Statist. 7 224--231. Mathematical Reviews (MathSciNet): MR0515697
Digital Object Identifier: doi:10.1214/aos/1176344568
Project Euclid: euclid.aos/1176344568
Zentralblatt MATH: 0399.62038 - Prakasa Rao, B. L. S. (1981). Asymptotic behavior of $M$-estimators for the linear model with dependent errors. Bull. Inst. Math. Acad. Sinica 9 367--375. Mathematical Reviews (MathSciNet): MR0635317
- Priestley, M. B. (1988). Nonlinear and Nonstationary Time Series Analysis. Academic Press, London. Mathematical Reviews (MathSciNet): MR0991969
- Rao, C. R. and Zhao, L. C. (1992). Linear representations of $M$-estimates in linear models. Canad. J. Statist. 20 359--368. Mathematical Reviews (MathSciNet): MR1208349
Digital Object Identifier: doi:10.2307/3315607
JSTOR: links.jstor.org - Rockafellar, R. T. (1970). Convex Analysis. Princeton Univ. Press.
- Ronner, A. E. (1984). Asymptotic normality of $p$-norm estimators in multiple regression. Z. Wahrsch. Verw. Gebiete 66 613--620. Mathematical Reviews (MathSciNet): MR0753816
Digital Object Identifier: doi:10.1007/BF00531893
Zentralblatt MATH: 0525.62033 - Rosenblatt, M. (1959). Stationary processes as shifts of functions of independent random variables. J. Math. Mech. 8 665--681.
- Rosenblatt, M. (1971). Markov Processes. Structure and Asymptotic Behavior. Springer, New York.
- Solomyak, B. M. (1995). On the random series $\sum \pm \lambda^n$ (an Erdös problem). Ann. of Math. (2) 142 611--625. Mathematical Reviews (MathSciNet): MR1356783
Digital Object Identifier: doi:10.2307/2118556
JSTOR: links.jstor.org - Stine, R. A. (1997). Nonlinear time series. Encyclopedia of Statistical Sciences 430--437. Wiley, New York.
- Surgailis, D. (2002). Stable limits of empirical processes of moving averages with infinite variance. Stochastic Process. Appl. 100 255--274. Mathematical Reviews (MathSciNet): MR1919616
Digital Object Identifier: doi:10.1016/S0304-4149(02)00103-5
Zentralblatt MATH: 1059.60044 - Tong, H. (1990). Nonlinear Time Series. A Dynamical System Approach. Oxford Univ. Press. Mathematical Reviews (MathSciNet): MR1079320
- Tsay, R. S. (2005). Analysis of Financial Time Series, 2nd ed. Wiley, Hoboken, NJ.
- Welsh, A. H. (1986). Bahadur representations for robust scale estimators based on regression residuals. Ann. Statist. 14 1246--1251. Mathematical Reviews (MathSciNet): MR0856820
Digital Object Identifier: doi:10.1214/aos/1176350064
Project Euclid: euclid.aos/1176350064
Zentralblatt MATH: 0604.62028 - Welsh, A. H. (1989). On $M$-processes and $M$-estimation. Ann. Statist. 17 337--361. Mathematical Reviews (MathSciNet): MR0981455
Digital Object Identifier: doi:10.1214/aos/1176347021
Project Euclid: euclid.aos/1176347021
Zentralblatt MATH: 0701.62074 - Wiener, N. (1958). Nonlinear Problems in Random Theory. MIT Press, Cambridge, MA.
- Withers, C. S. (1981). Conditions for linear process to be strongly mixing. Z. Wahrsch. Verw. Gebiete 57 477--480. Mathematical Reviews (MathSciNet): MR0631371
Digital Object Identifier: doi:10.1007/BF01025869
Zentralblatt MATH: 0465.60032 - Wu, W. B. (2003). Additive functionals of infinite-variance moving averages. Statist. Sinica 13 1259--1267.
- Wu, W. B. (2005). On the Bahadur representation of sample quantiles for dependent sequences. Ann. Statist. 33 1934--1963 Mathematical Reviews (MathSciNet): MR2166566
Digital Object Identifier: doi:10.1214/009053605000000291
Project Euclid: euclid.aos/1123250233
Zentralblatt MATH: 1080.62024 - Wu, W. B. (2005). Nonlinear system theory: Another look at dependence. Proc. Natl. Acad. Sci. USA 102 14,150--14,154. Mathematical Reviews (MathSciNet): MR2172215
Digital Object Identifier: doi:10.1073/pnas.0506715102
Zentralblatt MATH: 1135.62075 - Wu, W. B. (2006). $M$-estimation of linear models with dependent errors. Available at arxiv.org/math/0412268.
- Wu, W. B. and Shao, X. (2004). Limit theorems for iterated random functions. J. Appl. Probab. 41 425--436. Mathematical Reviews (MathSciNet): MR2052582
Digital Object Identifier: doi:10.1239/jap/1082999076
Project Euclid: euclid.jap/1082999076
Zentralblatt MATH: 1046.60024 - Yohai, V. J. (1974). Robust estimation in the linear model. Ann. Statist. 2 562--567. Mathematical Reviews (MathSciNet): MR0365875
Digital Object Identifier: doi:10.1214/aos/1176342717
Project Euclid: euclid.aos/1176342717
Zentralblatt MATH: 0286.62043 - Yohai, V. J. and Maronna, R. A. (1979). Asymptotic behavior of $M$-estimators for the linear model. Ann. Statist. 7 258--268. Mathematical Reviews (MathSciNet): MR0520237
Digital Object Identifier: doi:10.1214/aos/1176344610
Project Euclid: euclid.aos/1176344610
Zentralblatt MATH: 0408.62027 - Zeckhauser, R. and Thompson, M. (1970). Linear regression with non-normal error terms. Review Economics and Statistics 52 280--286.
- Zhao, L. (2000). Some contributions to $M$-estimation in linear models. J. Statist. Plann. Inference 88 189--203. Mathematical Reviews (MathSciNet): MR1792040
Digital Object Identifier: doi:10.1016/S0378-3758(00)00078-1
Zentralblatt MATH: 0951.62058
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Heavy tail modeling and teletraffic data: special invited paper
Resnick, Sidney I., The Annals of Statistics, 1997 - Invariance principles for linear processes with application to isotonic regression
Dedecker, Jérôme, Merlevède, Florence, and Peligrad, Magda, Bernoulli, 2011 - Multivariate generalized linear-statistics of short range dependent data
Fischer, Svenja, Fried, Roland, and Wendler, Martin, Electronic Journal of Statistics, 2016
- Heavy tail modeling and teletraffic data: special invited paper
Resnick, Sidney I., The Annals of Statistics, 1997 - Invariance principles for linear processes with application to isotonic regression
Dedecker, Jérôme, Merlevède, Florence, and Peligrad, Magda, Bernoulli, 2011 - Multivariate generalized linear-statistics of short range dependent data
Fischer, Svenja, Fried, Roland, and Wendler, Martin, Electronic Journal of Statistics, 2016 - Performance bounds for parameter estimates of high-dimensional linear models with correlated errors
Wu, Wei-Biao and Wu, Ying Nian, Electronic Journal of Statistics, 2016 - Large-sample inference for nonparametric regression with dependent errors
Robinson, P. M., The Annals of Statistics, 1997 - Time series regression with long-range dependence
Robinson, P. M. and Hidalgo, F. J., The Annals of Statistics, 1997 - Bahadur Representations for Robust Scale Estimators Based on Regression Residuals
Welsh, A. H., The Annals of Statistics, 1986 - Quantile processes for semi and nonparametric regression
Chao, Shih-Kang, Volgushev, Stanislav, and Cheng, Guang, Electronic Journal of Statistics, 2017 - Strong invariance principles for sequential Bahadur–Kiefer and Vervaat error processes of long-range dependent sequences
Csörgő, Miklós, Szyszkowicz, Barbara, and Wang, Lihong, The Annals of Statistics, 2006 - Estimation of the Parameters of Linear Time Series Models Subject to Nonlinear Restrictions
Nagaraj, Neerchal K. and Fuller, Wayne A., The Annals of Statistics, 1991