The Annals of Statistics

Time-varying nonlinear regression models: Nonparametric estimation and model selection

Ting Zhang and Wei Biao Wu

Full-text: Open access


This paper considers a general class of nonparametric time series regression models where the regression function can be time-dependent. We establish an asymptotic theory for estimates of the time-varying regression functions. For this general class of models, an important issue in practice is to address the necessity of modeling the regression function as nonlinear and time-varying. To tackle this, we propose an information criterion and prove its selection consistency property. The results are applied to the U.S. Treasury interest rate data.

Article information

Ann. Statist., Volume 43, Number 2 (2015), 741-768.

First available in Project Euclid: 3 March 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation 62G08: Nonparametric regression
Secondary: 62G20: Asymptotic properties

Information criterion local linear estimation nonparametric model selection nonstationary processes time-varying nonlinear regression models


Zhang, Ting; Wu, Wei Biao. Time-varying nonlinear regression models: Nonparametric estimation and model selection. Ann. Statist. 43 (2015), no. 2, 741--768. doi:10.1214/14-AOS1299.

Export citation


  • Aït-Sahalia, Y. (1996). Testing continuous-time models of the spot interest rate. Rev. Finan. Stud. 9 385–426.
  • Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory (Tsahkadsor, 1971) (B. N. Petrov and F. Csaski, eds.) 267–281. Akadémiai Kiadó, Budapest.
  • Andrews, D. W. K. (1993). Tests for parameter instability and structural change with unknown change point. Econometrica 61 821–856.
  • Andrews, D. W. K. (1995). Nonparametric kernel estimation for semiparametric models. Econometric Theory 11 560–596.
  • Azzalini, A. and Bowman, A. (1993). On the use of nonparametric regression for checking linear relationships. J. R. Stat. Soc. Ser. B. Stat. Methodol. 55 549–557.
  • Black, F. and Scholes, M. (1973). The pricing of options and corporate liabilities. J. Polit. Economy 81 637–654.
  • Bosq, D. (1996). Nonparametric Statistics for Stochastic Processes: Estimation and Prediction. Lecture Notes in Statistics 110. Springer, New York.
  • Brown, R. L., Durbin, J. and Evans, J. M. (1975). Techniques for testing the constancy of regression relationships over time. J. R. Stat. Soc. Ser. B. Stat. Methodol. 37 149–192.
  • Cai, Z. (2007). Trending time-varying coefficient time series models with serially correlated errors. J. Econometrics 136 163–188.
  • Castellana, J. V. and Leadbetter, M. R. (1986). On smoothed probability density estimation for stationary processes. Stochastic Process. Appl. 21 179–193.
  • Chan, K. C., Karolyi, A. G., Longstaff, F. A. and Sanders, A. B. (1992). An empirical comparison of alternative models of the short-term interest rate. J. Finance 47 1209–1227.
  • Chow, G. C. (1960). Tests of equality between sets of coefficients in two linear regressions. Econometrica 28 591–605.
  • Courtadon, G. (1982). The pricing of options on default-free bonds. J. Finan. Quant. Anal. 17 75–100.
  • Cox, J. C., Ingersoll, J. E. Jr. and Ross, S. A. (1985). A theory of the term structure of interest rates. Econometrica 53 385–407.
  • Davis, R. A., Huang, D. W. and Yao, Y.-C. (1995). Testing for a change in the parameter values and order of an autoregressive model. Ann. Statist. 23 282–304.
  • Dette, H. (1999). A consistent test for the functional form of a regression based on a difference of variance estimators. Ann. Statist. 27 1012–1040.
  • Draghicescu, D., Guillas, S. and Wu, W. B. (2009). Quantile curve estimation and visualization for nonstationary time series. J. Comput. Graph. Statist. 18 1–20.
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
  • Fan, J. and Yao, Q. (2003). Nonlinear Time Series: Nonparametric and Parametric Methods. Springer, New York.
  • Fan, J. and Zhang, J.-T. (2000a). Two-step estimation of functional linear models with applications to longitudinal data. J. R. Stat. Soc. Ser. B. Stat. Methodol. 62 303–322.
  • Fan, J. and Zhang, W. (2000b). Simultaneous confidence bands and hypothesis testing in varying-coefficient models. Scand. J. Stat. 27 715–731.
  • Fan, J., Zhang, C. and Zhang, J. (2001). Generalized likelihood ratio statistics and Wilks phenomenon. Ann. Statist. 29 153–193.
  • Freedman, D. A. (1975). On tail probabilities for martingales. Ann. Probab. 3 100–118.
  • Gao, J. and Gijbels, I. (2008). Bandwidth selection in nonparametric kernel testing. J. Amer. Statist. Assoc. 103 1584–1594.
  • González Manteiga, W. and Cao, R. (1993). Testing the hypothesis of a general linear model using nonparametric regression estimation. TEST 2 161–188.
  • Györfi, L., Härdle, W., Sarda, P. and Vieu, P. (1989). Nonparametric Curve Estimation from Time Series. Lecture Notes in Statistics 60. Springer, Berlin.
  • Hall, P., Müller, H.-G. and Wu, P.-S. (2006). Real-time density and mode estimation with application to time-dynamic mode tracking. J. Comput. Graph. Statist. 15 82–100.
  • Hansen, B. E. (2008). Uniform convergence rates for kernel estimation with dependent data. Econometric Theory 24 726–748.
  • Härdle, W., Hall, P. and Marron, J. S. (1988). How far are automatically chosen regression smoothing parameters from their optimum? J. Amer. Statist. Assoc. 83 86–101.
  • Härdle, W. and Mammen, E. (1993). Comparing nonparametric versus parametric regression fits. Ann. Statist. 21 1926–1947.
  • Härdle, W. and Marron, J. S. (1985). Optimal bandwidth selection in nonparametric regression function estimation. Ann. Statist. 13 1465–1481.
  • He, C., Teräsvirta, T. and González, A. (2009). Testing parameter constancy in stationary vector autoregressive models against continuous change. Econometric Rev. 28 225–245.
  • Hoover, D. R., Rice, J. A., Wu, C. O. and Yang, L.-P. (1998). Nonparametric smoothing estimates of time-varying coefficient models with longitudinal data. Biometrika 85 809–822.
  • Huang, J. Z., Wu, C. O. and Zhou, L. (2004). Polynomial spline estimation and inference for varying coefficient models with longitudinal data. Statist. Sinica 14 763–788.
  • Hurvich, C. M., Simonoff, J. S. and Tsai, C.-L. (1998). Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J. R. Stat. Soc. Ser. B. Stat. Methodol. 60 271–293.
  • Leybourne, S. J. and McCabe, B. P. M. (1989). On the distribution of some test statistics for coefficient constancy. Biometrika 76 169–177.
  • Li, Q. and Racine, J. S. (2007). Nonparametric Econometrics: Theory and Practice. Princeton Univ. Press, Princeton, NJ.
  • Lin, C.-F. J. and Teräsvirta, T. (1999). Testing parameter constancy in linear models against stochastic stationary parameters. J. Econometrics 90 193–213.
  • Liu, W. and Wu, W. B. (2010). Simultaneous nonparametric inference of time series. Ann. Statist. 38 2388–2421.
  • Liu, W., Xiao, H. and Wu, W. B. (2013). Probability and moment inequalities under dependence. Statist. Sinica 23 1257–1272.
  • Mallows, C. L. (1973). Some comments on $C_p$. Technometrics 15 661–675.
  • Masry, E. (1996). Multivariate local polynomial regression for time series: Uniform strong consistency and rates. J. Time Series Anal. 17 571–599.
  • Merton, R. C. (1974). On the pricing of corporate debt: The risk structure of interest rates. J. Finan. Econ. 3 125–144.
  • Nabeya, S. and Tanaka, K. (1988). Asymptotic theory of a test for the constancy of regression coefficients against the random walk alternative. Ann. Statist. 16 218–235.
  • Neumann, M. H. (1998). Strong approximation of density estimators from weakly dependent observations by density estimators from independent observations. Ann. Statist. 26 2014–2048.
  • Neumann, M. H. and Kreiss, J.-P. (1998). Regression-type inference in nonparametric autoregression. Ann. Statist. 26 1570–1613.
  • Nyblom, J. (1989). Testing for the constancy of parameters over time. J. Amer. Statist. Assoc. 84 223–230.
  • Opsomer, J., Wang, Y. and Yang, Y. (2001). Nonparametric regression with correlated errors. Statist. Sci. 16 134–153.
  • Orbe, S., Ferreira, E. and Rodriguez-Poo, J. (2005). Nonparametric estimation of time varying parameters under shape restrictions. J. Econometrics 126 53–77.
  • Orbe, S., Ferreira, E. and Rodriguez-Poo, J. (2006). On the estimation and testing of time varying constraints in econometric models. Statist. Sinica 16 1313–1333.
  • Park, B. U. and Marron, J. S. (1990). Comparison of data-driven bandwidth selectors. J. Amer. Statist. Assoc. 85 66–72.
  • Peligrad, M. (1992). Properties of uniform consistency of the kernel estimators of density and of regression functions under dependence assumptions. Stochastics Stochastics Rep. 40 147–168.
  • Ploberger, W., Krämer, W. and Kontrus, K. (1989). A new test for structural stability in the linear regression model. J. Econometrics 40 307–318.
  • Priestley, M. B. and Chao, M. T. (1972). Non-parametric function fitting. J. R. Stat. Soc. Ser. B. Stat. Methodol. 34 385–392.
  • Ramsay, J. O. and Silverman, B. W. (2005). Functional Data Analysis, 2nd ed. Springer, New York.
  • Robinson, P. M. (1983). Nonparametric estimators for time series. J. Time Series Anal. 4 185–207.
  • Ruppert, D., Sheather, S. J. and Wand, M. P. (1995). An effective bandwidth selector for local least squares regression. J. Amer. Statist. Assoc. 90 1257–1270.
  • Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461–464.
  • Shao, J. (1997). An asymptotic theory for linear model selection. Statist. Sinica 7 221–264.
  • Shao, X. and Wu, W. B. (2007). Asymptotic spectral theory for nonlinear time series. Ann. Statist. 35 1773–1801.
  • Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman & Hall, London.
  • Stanton, R. (1997). A nonparametric model of term structure dynamics and the market price of interest rate risk. J. Finance 52 1973–2002.
  • Tibshirani, R. J. and Tibshirani, R. (2009). A bias correction for the minimum error rate in cross-validation. Ann. Appl. Stat. 3 822–829.
  • Tjøstheim, D. (1994). Non-linear time series: A selective review. Scand. J. Stat. 21 97–130.
  • Vasicek, O. (1977). An equilibrium characterization of the term structure. J. Finan. Econ. 5 177–188.
  • Vogt, M. (2012). Nonparametric regression for locally stationary time series. Ann. Statist. 40 2601–2633.
  • Wand, M. P. and Jones, M. C. (1995). Kernel Smoothing. Monographs on Statistics and Applied Probability 60. Chapman & Hall, London.
  • Wang, Y. D. (1998). Smoothing spline models with correlated random errors. J. Amer. Statist. Assoc. 93 341–348.
  • Wu, W. B. (2005). Nonlinear system theory: Another look at dependence. Proc. Natl. Acad. Sci. USA 102 14150–14154 (electronic).
  • Wu, W. B., Huang, Y. and Huang, Y. (2010). Kernel estimation for time series: An asymptotic theory. Stochastic Process. Appl. 120 2412–2431.
  • Xia, Y. (1998). Bias-corrected confidence bands in nonparametric regression. J. R. Stat. Soc. Ser. B. Stat. Methodol. 60 797–811.
  • Yang, Y. (2005). Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation. Biometrika 92 937–950.
  • Yu, B. (1993). Density estimation in the $L^\infty$-norm for dependent data with applications to the Gibbs sampler. Ann. Statist. 21 711–735.
  • Zhang, C. and Dette, H. (2004). A power comparison between nonparametric regression tests. Statist. Probab. Lett. 66 289–301.
  • Zhang, T. and Wu, W. B. (2011). Testing parametric assumptions of trends of a nonstationary time series. Biometrika 98 599–614.
  • Zhang, T. and Wu, W. B. (2012). Inference of time-varying regression models. Ann. Statist. 40 1376–1402.
  • Zhang, T. and Wu, W. B. (2015). Supplement to “Time-varying nonlinear regression models: Nonparametric estimation and model selection.” DOI:10.1214/14-AOS1299SUPP.
  • Zhao, Z. (2008). Parametric and nonparametric models and methods in financial econometrics. Stat. Surv. 2 1–42.
  • Zheng, J. X. (1996). A consistent test of functional form via nonparametric estimation techniques. J. Econometrics 75 263–289.
  • Zhou, Z. and Wu, W. B. (2010). Simultaneous inference of linear models with time varying coefficients. J. R. Stat. Soc. Ser. B. Stat. Methodol. 72 513–531.

Supplemental materials