The Annals of Statistics

Methodology and convergence rates for functional linear regression

Peter Hall and Joel L. Horowitz

Full-text: Open access


In functional linear regression, the slope “parameter” is a function. Therefore, in a nonparametric context, it is determined by an infinite number of unknowns. Its estimation involves solving an ill-posed problem and has points of contact with a range of methodologies, including statistical smoothing and deconvolution. The standard approach to estimating the slope function is based explicitly on functional principal components analysis and, consequently, on spectral decomposition in terms of eigenvalues and eigenfunctions. We discuss this approach in detail and show that in certain circumstances, optimal convergence rates are achieved by the PCA technique. An alternative approach based on quadratic regularisation is suggested and shown to have advantages from some points of view.

Article information

Ann. Statist., Volume 35, Number 1 (2007), 70-91.

First available in Project Euclid: 6 June 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J05: Linear regression
Secondary: 62G20: Asymptotic properties

Deconvolution dimension reduction eigenfunction eigenvalue linear operator minimax optimality nonparametric principal components analysis smoothing quadratic regularisation


Hall, Peter; Horowitz, Joel L. Methodology and convergence rates for functional linear regression. Ann. Statist. 35 (2007), no. 1, 70--91. doi:10.1214/009053606000000957.

Export citation


  • Bhatia, R., Davis, C. and McIntosh, A. (1983). Perturbation of spectral subspaces and solution of linear operator equations. Linear Algebra Appl. 52/53 45–67.
  • Blundell, R. and Powell, J. L. (2003). Endogeneity in nonparametric and semiparametric regression models. In Advances in Economics and Econometrics: Theory and Applications (M. Dewatripont, L. P. Hansen and S. J. Turnovsky, eds.) 2 312–357. Cambridge Univ. Press.
  • Cardot, H. and Sarda, P. (2003). Linear regression models for functional data. Available at
  • Carroll, R. J., Ruppert, D. and Stefanski, L. A. (1995). Measurement Error in Nonlinear Models. Chapman and Hall, London.
  • Cavalier, L., Golubev, G. K., Picard, D. and Tsybakov, A. B. (2002). Oracle inequalities for inverse problems. Ann. Statist. 30 843–874.
  • Chiou, J.-M., Müller, H.-G. and Wang, J.-L. (2003). Functional quasi-likelihood regression models with smooth random effects. J. R. Stat. Soc. Ser. B Stat. Methodol. 65 405–423.
  • Cuevas, A., Febrero, M. and Fraiman, R. (2002). Linear functional regression: The case of fixed design and functional response. Canad. J. Statist. 30 285–300.
  • Darolles, S., Florens, J.-P. and Renault, E. (2002). Nonparametric instrumental regression. Working paper, GREMAQ, Univ. Social Science, Toulouse.
  • Delaigle, A. and Gijbels, I. (2002). Estimation of integrated squared density derivatives from a contaminated sample. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 869–886.
  • Donoho, D. L. (1995). Nonlinear solution of linear inverse problems by wavelet-vaguelette decomposition. Appl. Comput. Harmon. Anal. 2 101–126.
  • Efromovich, S. and Koltchinskii, V. (2001). On inverse problems with unknown operators. IEEE Trans. Inform. Theory 47 2876–2894.
  • Fan, J. (1991). On the optimal rates of convergence for nonparametric deconvolution problems. Ann. Statist. 19 1257–1272.
  • Fan, J. (1993). Adaptively local one-dimensional subproblems with application to a deconvolution problem. Ann. Statist. 21 600–610.
  • Ferraty, F. and Vieu, P. (2000). Dimension fractale et estimation de la régression dans des espaces vectoriels semi-normés. C. R. Acad. Sci. Paris Sér. I Math. 330 139–142.
  • Hall, P. and Horowitz, J. L. (2005). Nonparametric methods for inference in the presence of instrumental variables. Ann. Statist. 33 2904–2929.
  • Horowitz, J. L. and Markatou, M. (1996). Semiparametric estimation of regression models for panel data. Rev. Econom. Stud. 63 145–168.
  • Johnstone, I. M. (1999). Wavelet shrinkage for correlated data and inverse problems: Adaptivity results. Statist. Sinica 9 51–83.
  • Kundur, D. and Hatzinakos, D. (1998). A novel blind deconvolution scheme for image restoration using recursive filtering. IEEE Trans. Signal Processing 46 375–390.
  • Li, T. and Hsiao, C. (2004). Robust estimation of generalized linear models with measurement errors. J. Econometrics 118 51–65.
  • Natterer, F. (1984). Error bounds for Tikhonov regularization in Hilbert scales. Applicable Anal. 18 29–37.
  • Neubauer, A. (1989). Tikhonov regularisation for nonlinear ill-posed problems: Optimal convergence rates and finite-dimensional approximation. Inverse Problems 5 541–557.
  • Newey, W. K. and Powell, J. L. (2003). Instrumental variable estimation of nonparametric models. Econometrica 71 1565–1578.
  • Newey, W. K., Powell, J. L. and Vella, F. (1999). Nonparametric estimation of triangular simultaneous equations models. Econometrica 67 565–603.
  • Ramsay, J. O. and Silverman, B. W. (1997). Functional Data Analysis. Springer, New York.
  • Ramsay, J. O. and Silverman, B. W. (2002). Applied Functional Data Analysis: Methods and Case Studies. Springer, New York.
  • Shenk, J. S. and Westerhaus, M. O. (1991). Population definition, sample selection, and calibration procedures for near infrared reflectance spectroscopy. Crop Science 31 469–474.
  • Staniswalis, J. G. and Lee, J. J. (1998). Nonparametric regression analysis of longitudinal data. J. Amer. Statist. Assoc. 93 1403–1418.
  • Stefanski, L. and Carroll, R. J. (1990). Deconvoluting kernel density estimators. Statistics 21 169–184.
  • Tikhonov, A. N. (1963). On the solution of incorrectly put problems and the regularisation method. In Outlines Joint Sympos. Partial Differential Equations (Novosibirsk, 1963) 261–265. Acad. Sci. USSR Siberian Branch, Moscow.
  • van Rooij, A. C. M. and Ruymgaart, F. H. (1996). Asymptotic minimax rates for abstract linear estimators. J. Statist. Plann. Inference 53 389–402.
  • Wesley, I. J., Uthayakumaran, S., Anderssen, R. S., Cornish, G. B., Bekes, F., Osborne, B. G. and Skerritt, J. H. (1999). A curve-fitting approach to the near infrared reflectance measurement of wheat flour proteins which influence dough quality. J. Near Infrared Spectroscopy 7 229–240.
  • Yang, Y., Galatsanos, N. P. and Stark, H. (1994). Projection-based blind deconvolution. J. Optical Soc. Amer. Ser. A 11, 2401–2409.