The Annals of Statistics

Nonparametric regression, confidence regions and regularization

P. L. Davies, A. Kovac, and M. Meise

Full-text: Open access


In this paper we offer a unified approach to the problem of nonparametric regression on the unit interval. It is based on a universal, honest and nonasymptotic confidence region $\mathcal{A}_{n}$ which is defined by a set of linear inequalities involving the values of the functions at the design points. Interest will typically center on certain simplest functions in $\mathcal{A}_{n}$ where simplicity can be defined in terms of shape (number of local extremes, intervals of convexity/concavity) or smoothness (bounds on derivatives) or a combination of both. Once some form of regularization has been decided upon the confidence region can be used to provide honest nonasymptotic confidence bounds which are less informative but conceptually much simpler.

Article information

Ann. Statist., Volume 37, Number 5B (2009), 2597-2625.

First available in Project Euclid: 17 July 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression
Secondary: 62G15: Tolerance and confidence regions 62G20: Asymptotic properties

Nonparametric regression confidence region confidence bands shape regularization smoothness regularization


Davies, P. L.; Kovac, A.; Meise, M. Nonparametric regression, confidence regions and regularization. Ann. Statist. 37 (2009), no. 5B, 2597--2625. doi:10.1214/07-AOS575.

Export citation


  • Baraud, Y. (2004). Confidence balls in Gaussian regression. Ann. Statist. 32 528–551.
  • Bernholt, T. and Hofmeister, T. (2006). An algorithm for a generalized maximum subsequence problem. In LATIN 2006: Theoretical informatics. Lecture Notes in Comput. Sci. 3887 178–189. Springer, Berlin.
  • Cai, T. T. and Low, M. G. (2006). Adaptive confidence balls. Ann. Statist. 34 202–228.
  • Davies, P. L. (1995). Data features. Statist. Neerlandica 49 185–245.
  • Davies, P. L. and Kovac, A. (2001). Local extremes, runs, strings and multiresolution (with discussion). Ann. Statist. 29 1–65.
  • Davies, P. L. and Meise, M. (2008). Approximating data with weighted smoothing splines. J. Nonparametr. Stat. 20 207–228.
  • Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 425–455.
  • Dümbgen, L. (1998). New goodness-of-fit tests and their application to nonparametric confidence sets. Ann. Statist. 26 288–314.
  • Dümbgen, L. (2003). Optimal confidence bands for shape-restricted curves. Bernoulli 9 423–449.
  • Dümbgen, L. (2007). Confidence bands for convex median curves using sign-tests. In Asymptotics: Particles, Processes and Inverse Problems (E. Cator, G. Jongbloed, C. Kraaikamp, R. Lopuhaä and J. Wellner, eds.). IMS Lecture Notes—Monograph Series 55 85–100. IMS, Hayward, USA.
  • Dümbgen, L. and Johns, R. (2004). Confidence bands for isotonic median curves using sign-tests. J. Comput. Graph. Statist. 13 519–533.
  • Dümbgen, L. and Spokoiny, V. G. (2001). Multiscale testing of qualitative hypotheses. Ann. Statist. 29 124–152.
  • Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Chapman and Hall, London.
  • Green, P. J. and Silverman, B. W. (1994). Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach. Chapman and Hall, London.
  • Groeneboom, P. (1996). Inverse problems in statistics. In Proceedings of the St. Flour Summer School in Probability. Lecture Notes in Math. 1648 67–164. Springer, Berlin.
  • Hoffmann, M. and Lepski, O. (2002). Random rates in anisotropic regression. Ann. Statist. 30 325–396.
  • Kabluchko, Z. and Munk, A. (2008). Exact convergence rate for the maximum of standardized Gaussian increments. Electron. Commun. Probab. 13 302–310.
  • Kovac, A. (2007). ftnonpar. The R Project for Statistical Computing, Contributed Packages.
  • Li, K.-C. (1989). Honset confidence regions for nonparametric regression. Ann. Statist. 17 1001–1008.
  • Mammen, E. (1991). Nonparametric regression under qualitative smoothness assumptions. Ann. Statist. 19 741–759.
  • Mildenberger, T. (2008). A geometric interpretation of the multiresolution criterion. J. Nonparametr. Stat. 20 599–609.
  • Robins, J. and van der Vaart, A. (2006). Adaptive nonparametric confidence sets. Ann. Statist. 34 229–253.
  • Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia, PA.
  • Wand, M. P. and Jones, M. C. (1995). Kernel Smoothing. Chapman and Hall, London.
  • Watson, G. S. (1964). Smooth regression analysis. Sankhyā 26 101–116.