The Annals of Statistics

Simultaneous confidence bands for Yule–Walker estimators and order selection

Moritz Jirak

Full-text: Open access


Let {Xk, k ∈ ℤ} be an autoregressive process of order q. Various estimators for the order q and the parameters Θq = (θ1, …, θq)T are known; the order is usually determined with Akaike’s criterion or related modifications, whereas Yule–Walker, Burger or maximum likelihood estimators are used for the parameters Θq. In this paper, we establish simultaneous confidence bands for the Yule–Walker estimators θ̂i; more precisely, it is shown that the limiting distribution of max1≤idn|θ̂iθi| is the Gumbel-type distribution eez, where q ∈ {0, …, dn} and $d_{n}=\mathcal{O}(n^{\delta})$, δ > 0. This allows to modify some of the currently used criteria (AIC, BIC, HQC, SIC), but also yields a new class of consistent estimators for the order q. These estimators seem to have some potential, since they outperform most of the previously mentioned criteria in a small simulation study. In particular, if some of the parameters {θi}1≤idn are zero or close to zero, a significant improvement can be observed. As a byproduct, it is shown that BIC, HQC and SIC are consistent for q ∈ {0, …, dn} where $d_{n}=\mathcal{O}(n^{\delta})$.

Article information

Ann. Statist., Volume 40, Number 1 (2012), 494-528.

First available in Project Euclid: 7 May 2012

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60M10 62F05: Asymptotic properties of tests
Secondary: 62F10: Point estimation 62F12: Asymptotic properties of estimators

Autoregressive process Yule–Walker estimators extreme value distribution order selection AIC


Jirak, Moritz. Simultaneous confidence bands for Yule–Walker estimators and order selection. Ann. Statist. 40 (2012), no. 1, 494--528. doi:10.1214/11-AOS963.

Export citation


  • [1] Akaike, H. (1969). Fitting autoregressive models for prediction. Ann. Inst. Statist. Math. 21 243–247.
  • [2] Akaike, H. (1977). On entropy maximization principle. In Applications of Statistics (Proc. Sympos., Wright State Univ., Dayton, Ohio, 1976) 27–41. North-Holland, Amsterdam.
  • [3] An, H. Z., Chen, Z. G. and Hannan, E. J. (1982). Autocorrelation, autoregression and autoregressive approximation. Ann. Statist. 10 926–936.
  • [4] Anderson, T. W. (1971). The Statistical Analysis of Time Series. Wiley, New York.
  • [5] Balkema, A. A. and de Haan, L. (1990). A convergence rate in extreme-value theory. J. Appl. Probab. 27 577–585.
  • [6] Barron, A., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301–413.
  • [7] Berk, K. N. (1974). Consistent autoregressive spectral estimates. Ann. Statist. 2 489–502. Collection of articles dedicated to Jerzy Neyman on his 80th birthday.
  • [8] Berkes, I., Gombay, E. and Horváth, L. (2009). Testing for changes in the covariance structure of linear processes. J. Statist. Plann. Inference 139 2044–2063.
  • [9] Berman, S. M. (1964). Limit theorems for the maximum term in stationary sequences. Ann. Math. Statist. 35 502–516.
  • [10] Berthet, P. and Mason, D. M. (2006). Revisiting two strong approximation results of Dudley and Philipp. In High Dimensional Probability. Institute of Mathematical Statistics Lecture Notes—Monograph Series 51 155–172. IMS, Beachwood, OH.
  • [11] Bhansali, R. J. (1991). Consistent recursive estimation of the order of an autoregressive moving average process. International Statistical Review/Revue Internationale de Statistique 59 81–96.
  • [12] Bickel, P. J. and Gel, Y. R. (2011). Banded regularization of autocovariance matrices in application to parameter estimation and forecasting of time series. J. R. Stat. Soc. Ser. B Stat. Methodol. 73 711–728.
  • [13] Bollerslev, T. (1986). Generalized autoregressive conditional heteroskedasticity. J. Econometrics 31 307–327.
  • [14] Bollerslev, T., Engle, R. F. and Nelson, D. B. (1994). Arch models. In Handbook of Econometrics, Vol. IV. Handbooks in Economics 2 2959–3038. North-Holland, Amsterdam.
  • [15] Box, G. E. P., Jenkins, G. M. and Reinsel, G. C. (2008). Time Series Analysis: Forecasting and Control, 4th ed. Wiley, Hoboken, NJ.
  • [16] Brockwell, P. J., Dahlhaus, R. and Trindade, A. A. (2005). Modified Burg algorithms for multivariate subset autoregression. Statist. Sinica 15 197–213.
  • [17] Brockwell, P. J. and Davis, R. A. (1991). Time Series: Theory and Methods, 2nd ed. Springer, New York.
  • [18] Deo, C. M. (1972). Some limit theorems for maxima of absolute values of Gaussian sequences. Sankhyā Ser. A 34 289–292.
  • [19] Engle, R. (2002). Dynamic conditional correlation: A simple class of multivariate generalized autoregressive conditional heteroskedasticity models. J. Bus. Econom. Statist. 20 339–350.
  • [20] Foster, D. P. and George, E. I. (1994). The risk inflation criterion for multiple regression. Ann. Statist. 22 1947–1975.
  • [21] Galbraith, R. F. and Galbraith, J. I. (1974). On the inverses of some patterned matrices arising in the theory of stationary time series. J. Appl. Probab. 11 63–71.
  • [22] Gouriéroux, C. (1997). ARCH Models and Financial Applications. Springer, New York.
  • [23] Hannan, E. J. (1970). Multiple Time Series. Wiley, Sydney.
  • [24] Hannan, E. J. (1980). The estimation of the order of an ARMA process. Ann. Statist. 8 1071–1081.
  • [25] Hannan, E. J. and Quinn, B. G. (1979). The determination of the order of an autoregression. J. Roy. Statist. Soc. Ser. B 41 190–195.
  • [26] Ing, C.-K. and Wei, C.-Z. (2003). On same-realization prediction in an infinite-order autoregressive process. J. Multivariate Anal. 85 130–155.
  • [27] Ing, C.-K. and Wei, C.-Z. (2005). Order selection for same-realization predictions in autoregressive processes. Ann. Statist. 33 2423–2474.
  • [28] Leeb, H. and Pötscher, B. M. (2008). Model selection. In Handbook of Financial Time Series. Springer, New York.
  • [29] Lütkepohl, H. (2005). New Introduction to Multiple Time Series Analysis. Springer, Berlin.
  • [30] Mallows, C. L. (1964). Choosing variables in a linear regression: A graphical aid. Presented at the Central Regional Meeting of the Institute of Mathematical Statistics, Manhattan, KS 5.
  • [31] Mallows, C. L. (1966). Choosing a subset regression. Presented at the Joint Statistical Meeting, Los Angeles, CA.
  • [32] McClave, J. (1975). Subset autoregression. Technometrics 17 213–220.
  • [33] McLeod, A. I. and Zhang, Y. (2008). Improved subset autoregression: With R package. Journal of Statistical Software 28 1–28.
  • [34] Nakatsuka, T. (1978). Regions of autocorrelation coefficients in AR(p) and EX(p) processes. Ann. Inst. Statist. Math. 30 315–319.
  • [35] Omey, E. (1989). On the rate of convergence in extreme value theory. In Stability Problems for Stochastic Models (Sukhumi, 1987). Lecture Notes in Math. 1412 270–279. Springer, Berlin.
  • [36] Parzen, E. (1974). Some recent advances in time series modeling. IEEE Trans. Automat. Control AC-19 723–730. System identification and time-series analysis.
  • [37] Quenouille, M. H. (1947). A large-sample test for the goodness of fit of autoregressive schemes. J. Roy. Statist. Soc. (N.S.) 110 123–129.
  • [38] Rissanen, J. (1978). Modeling by shortest data description. Automatica 14 465–471.
  • [39] Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461–464.
  • [40] Shao, J. (1997). An asymptotic theory for linear model selection. Statist. Sinica 7 221–264. With comments and a rejoinder by the author.
  • [41] Shibata, R. (1976). Selection of the order of an autoregressive model by Akaike’s information criterion. Biometrika 63 117–126.
  • [42] Tong, H. (1977). Some comments on the canadian lynx data. J. Roy. Statist. Soc. Ser. A 140 432–436.
  • [43] Walker, A. M. (1952). Some properties of the asymptotic power functions of goodness-of-fit tests for linear autoregressive schemes. J. Roy. Statist. Soc. Ser. B. 14 117–134.
  • [44] Walker, G. (1931). On periodicity in series of related terms. Monthly Weather Review 59 277–278.
  • [45] Whittle, P. (1952). Tests of fit in time series. Biometrika 39 309–318.
  • [46] Whittle, P. (1963). On the fitting of multivariate autoregressions, and the approximate canonical factorization of a spectral density matrix. Biometrika 50 129–134.
  • [47] Wu, W. B. (2009). An asymptotic theory for sample covariances of Bernoulli shifts. Stochastic Process. Appl. 119 453–467.
  • [48] Wu, W. B. and Xiao, H. (2011). Asymptotic inference of autocovariances of stationary processes. Available at arXiv:1105.3423.
  • [49] Yule, U. G. (1927). On a method of investigating periodicities in disturbed series, with special reference to wolfer’s sunspot numbers. Phil. Trans. R. Soc. Lond. A 226 267–298.