The Annals of Statistics

Aggregation of predictors for nonstationary sub-linear processes and online adaptive forecasting of time varying autoregressive processes

Christophe Giraud, François Roueff, and Andres Sanchez-Perez

Full-text: Open access

Enhanced PDF (411 KB)

Abstract

In this work, we study the problem of aggregating a finite number of predictors for nonstationary sub-linear processes. We provide oracle inequalities relying essentially on three ingredients: (1) a uniform bound of the $\ell^{1}$ norm of the time varying sub-linear coefficients, (2) a Lipschitz assumption on the predictors and (3) moment conditions on the noise appearing in the linear representation. Two kinds of aggregations are considered giving rise to different moment conditions on the noise and more or less sharp oracle inequalities. We apply this approach for deriving an adaptive predictor for locally stationary time varying autoregressive (TVAR) processes. It is obtained by aggregating a finite number of well chosen predictors, each of them enjoying an optimal minimax convergence rate under specific smoothness conditions on the TVAR coefficients. We show that the obtained aggregated predictor achieves a minimax rate while adapting to the unknown smoothness. To prove this result, a lower bound is established for the minimax rate of the prediction risk for the TVAR process. Numerical experiments complete this study. An important feature of this approach is that the aggregated predictor can be computed recursively and is thus applicable in an online prediction context.

Article information

Source
Ann. Statist., Volume 43, Number 6 (2015), 2412-2450.

Dates
Received: May 2014
Revised: May 2015
First available in Project Euclid: 7 October 2015

Permanent link to this document
https://projecteuclid.org/euclid.aos/1444222080

Digital Object Identifier
doi:10.1214/15-AOS1345

Mathematical Reviews number (MathSciNet)
MR3405599

Zentralblatt MATH identifier
1327.62478

Subjects
Primary: 62M20: Prediction [See also 60G25]; filtering [See also 60G35, 93E10, 93E11] 62G99: None of the above, but in this section 62M10: Time series, auto-correlation, regression, etc. [See also 91B84] 68W27: Online algorithms

Keywords
Nonstationary time series exponential weighted aggregation online learning time varying autoregressive processes adaptive prediction

Citation

Giraud, Christophe; Roueff, François; Sanchez-Perez, Andres. Aggregation of predictors for nonstationary sub-linear processes and online adaptive forecasting of time varying autoregressive processes. Ann. Statist. 43 (2015), no. 6, 2412--2450. doi:10.1214/15-AOS1345. https://projecteuclid.org/euclid.aos/1444222080


Export citation

References

  • Alquier, P. and Wintenberger, O. (2012). Model selection for weakly dependent time series forecasting. Bernoulli 18 883–913.
    Mathematical Reviews (MathSciNet): MR2948906
    Digital Object Identifier: doi:10.3150/11-BEJ359
    Project Euclid: euclid.bj/1340887007
  • Anava, O., Hazan, E., Mannor, S. and Shamir, O. (2013). Online learning for time series prediction. Preprint. Available at arXiv:1302.6927.
    arXiv: 1302.6927
  • Arkoun, O. (2011). Sequential adaptive estimators in nonparametric autoregressive models. Sequential Anal. 30 229–247.
    Mathematical Reviews (MathSciNet): MR2801140
  • Audibert, J.-Y. (2009). Fast learning rates in statistical inference through aggregation. Ann. Statist. 37 1591–1646.
    Mathematical Reviews (MathSciNet): MR2533466
    Zentralblatt MATH: 05582005
    Digital Object Identifier: doi:10.1214/08-AOS623
    Project Euclid: euclid.aos/1245332827
  • Brockwell, P. J. and Davis, R. A. (2006). Time Series: Theory and Methods. Springer, New York. Reprint of the second (1991) edition.
    Mathematical Reviews (MathSciNet): MR2839251
  • Catoni, O. (1997). A mixture approach to universal model selection. Technical report, École Normale Supérieure.
  • Catoni, O. (2004). Statistical Learning Theory and Stochastic Optimization. Lecture Notes in Math. 1851. Springer, Berlin.
    Mathematical Reviews (MathSciNet): MR2163920
    Zentralblatt MATH: 1076.93002
  • Cesa-Bianchi, N. and Lugosi, G. (2006). Prediction, Learning, and Games. Cambridge Univ. Press, Cambridge.
    Mathematical Reviews (MathSciNet): MR2409394
    Zentralblatt MATH: 1114.91001
  • Dahlhaus, R. (1996). On the Kullback–Leibler information divergence of locally stationary processes. Stochastic Process. Appl. 62 139–168.
    Mathematical Reviews (MathSciNet): MR1388767
    Zentralblatt MATH: 0849.60032
    Digital Object Identifier: doi:10.1016/0304-4149(95)00090-9
  • Dahlhaus, R. (2009). Local inference for locally stationary time series based on the empirical spectral measure. J. Econometrics 151 101–112.
    Mathematical Reviews (MathSciNet): MR2559818
    Digital Object Identifier: doi:10.1016/j.jeconom.2009.03.002
  • Dahlhaus, R. and Polonik, W. (2006). Nonparametric quasi-maximum likelihood estimation for Gaussian locally stationary processes. Ann. Statist. 34 2790–2824.
    Mathematical Reviews (MathSciNet): MR2329468
    Zentralblatt MATH: 1114.62034
    Digital Object Identifier: doi:10.1214/009053606000000867
    Project Euclid: euclid.aos/1179935065
  • Dahlhaus, R. and Polonik, W. (2009). Empirical spectral processes for locally stationary time series. Bernoulli 15 1–39.
    Mathematical Reviews (MathSciNet): MR2546797
    Digital Object Identifier: doi:10.3150/08-BEJ137
    Project Euclid: euclid.bj/1233669881
  • Dalalyan, A. S. and Tsybakov, A. B. (2008). Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity. Mach. Learn. 72 39–61.
  • Doukhan, P. and Wintenberger, O. (2008). Weakly dependent chains with infinite memory. Stochastic Process. Appl. 118 1997–2013.
    Mathematical Reviews (MathSciNet): MR2462284
    Zentralblatt MATH: 1166.60031
    Digital Object Identifier: doi:10.1016/j.spa.2007.12.004
  • Gerchinovitz, S. (2011). Prediction of individual sequences and prediction in the statistical framework: Some links around sparse regression and aggregation techniques. Ph.D. thesis, Univ. Paris Sud-Paris XI.
  • Giraud, C., Roueff, F. and Sanchez-Perez, A. (2015). Supplement to “Aggregation of predictors for non stationary sub-linear processes and online adaptive forecasting of time varying autoregressive processes.” DOI:10.1214/15-AOS1345SUPP.
  • Grenier, Y. (1983). Time-dependent ARMA modeling of nonstationary signals. IEEE Transactions on ASSP 31 899–911.
  • Juditsky, A. and Nemirovski, A. (2000). Functional aggregation for nonparametric regression. Ann. Statist. 28 681–712.
    Mathematical Reviews (MathSciNet): MR1792783
    Zentralblatt MATH: 1105.62338
    Digital Object Identifier: doi:10.1214/aos/1015951994
    Project Euclid: euclid.aos/1015951994
  • Künsch, H. R. (1995). A note on causal solutions for locally stationary AR-processes. Unpublished preprint, ETH Zürich.
  • Lepskiĭ, O. V. (1990). A problem of adaptive estimation in Gaussian white noise. Teor. Veroyatn. Primen. 35 459–470.
    Mathematical Reviews (MathSciNet): MR1091202
  • Leung, G. and Barron, A. R. (2006). Information theory and mixing least-squares regressions. IEEE Trans. Inform. Theory 52 3396–3410.
    Mathematical Reviews (MathSciNet): MR2242356
    Digital Object Identifier: doi:10.1109/TIT.2006.878172
  • Massart, P. (2007). Concentration Inequalities and Model Selection. Lecture Notes in Math. 1896. Springer, Berlin.
    Mathematical Reviews (MathSciNet): MR2319879
    Zentralblatt MATH: 1170.60006
  • Moulines, E., Priouret, P. and Roueff, F. (2005). On recursive estimation for time varying autoregressive processes. Ann. Statist. 33 2610–2654.
    Mathematical Reviews (MathSciNet): MR2253097
    Zentralblatt MATH: 1084.62089
    Digital Object Identifier: doi:10.1214/009053605000000624
    Project Euclid: euclid.aos/1140191668
  • Rigollet, P. and Tsybakov, A. B. (2012). Sparse estimation by exponential weighting. Statist. Sci. 27 558–575.
    Mathematical Reviews (MathSciNet): MR3025134
    Digital Object Identifier: doi:10.1214/12-STS393
    Project Euclid: euclid.ss/1356098556
  • Sancetta, A. (2010). Recursive forecast combination for dependent heterogeneous data. Econometric Theory 26 598–631.
    Mathematical Reviews (MathSciNet): MR2600575
    Zentralblatt MATH: 1189.62186
    Digital Object Identifier: doi:10.1017/S0266466609100105
  • Stoltz, G. (2011). Contributions to the sequential prediction of arbitrary sequences: Applications to the theory of repeated games and empirical studies of the performance of the aggregation of experts. Habilitation à diriger des recherches, Univ. Paris Sud-Paris XI.
  • Tong, H. and Lim, K. S. (1980). Threshold autoregression, limit cycles and cyclical data. J. Roy. Statist. Soc. Ser. B 42 245–292.
  • Tsybakov, A. B. (2003). Optimal rates of aggregation. In Learning Theory and Kernel Machines (B. Schölkopf and M. K. Warmuth, eds.). Lecture Notes in Computer Science 2777 303–313. Springer, Berlin.
  • Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation. Springer, New York.
    Mathematical Reviews (MathSciNet): MR2724359
  • Vovk, V. G. (1990). Aggregating strategies. In Proc. Third Workshop on Computational Learning Theory 371–383. Morgan Kaufmann, San Mateo, CA.
  • Wang, Z., Paterlini, S., Gao, F. and Yang, Y. (2014). Adaptive minimax regression estimation over sparse $\ell_{q}$-hulls. J. Mach. Learn. Res. 15 1675–1711.
    Mathematical Reviews (MathSciNet): MR3225246
    Zentralblatt MATH: 06378126
  • Yang, Y. (2000a). Combining different procedures for adaptive regression. J. Multivariate Anal. 74 135–161.
    Mathematical Reviews (MathSciNet): MR1790617
    Zentralblatt MATH: 0964.62032
    Digital Object Identifier: doi:10.1006/jmva.1999.1884
  • Yang, Y. (2000b). Mixing strategies for density estimation. Ann. Statist. 28 75–87.
    Mathematical Reviews (MathSciNet): MR1762904
    Zentralblatt MATH: 1106.62322
    Digital Object Identifier: doi:10.1214/aos/1016120365
    Project Euclid: euclid.aos/1016120365
  • Yang, Y. (2004). Combining forecasting procedures: Some theoretical results. Econometric Theory 20 176–222.
    Mathematical Reviews (MathSciNet): MR2028357
    Zentralblatt MATH: 1046.62101
    Digital Object Identifier: doi:10.1017/S0266466604201086

Supplemental materials

  • Supplementary material for: Aggregation of predictors for nonstationary sub-linear processes and online adaptive forecasting of time varying autoregressive processes. We explain how to build nonadaptive minimax predictors which can be used in the aggregation step. The document also contains some technical proofs and provides additional results with improved aggregation rates.
    Digital Object Identifier: doi:10.1214/15-AOS1345SUPP
    Supplemental PDF (171 KB)

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more