The Annals of Statistics

Nonparametric estimation when data on derivatives are available

Peter Hall and Adonis Yatchew

Full-text: Open access


We consider settings where data are available on a nonparametric function and various partial derivatives. Such circumstances arise in practice, for example in the joint estimation of cost and input functions in economics. We show that when derivative data are available, local averages can be replaced in certain dimensions by nonlocal averages, thus reducing the nonparametric dimension of the problem. We derive optimal rates of convergence and conditions under which dimension reduction is achieved. Kernel estimators and their properties are analyzed, although other estimators, such as local polynomial, spline and nonparametric least squares, may also be used. Simulations and an application to the estimation of electricity distribution costs are included.

Article information

Ann. Statist., Volume 35, Number 1 (2007), 300-323.

First available in Project Euclid: 6 June 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 62G20: Asymptotic properties
Secondary: 62P20: Applications to economics [See also 91Bxx] 91B38: Production theory, theory of the firm

Dimension reduction kernel methods nonparametric regression partial derivative data rates of convergence statistical smoothing cost function estimation


Hall, Peter; Yatchew, Adonis. Nonparametric estimation when data on derivatives are available. Ann. Statist. 35 (2007), no. 1, 300--323. doi:10.1214/009053606000001127.

Export citation


  • Bickel, P. and Ritov, Y. (2003). Nonparametric estimators which can be “plugged-in”. Ann. Statist. 31 1033–1053.
  • Breiman, L., Friedman, J. H., Olshen, R. A. and Stone, C. J. (1984). Classification and Regression Trees. Wadsworth, Belmont, CA.
  • Buja, A., Hastie, T. and Tibshirani, R. (1989). Linear smoothers and additive models (with discussion). Ann. Statist. 17 453–555.
  • Donoho, D. L. and Johnstone, I. M. (1989). Projection-based approximation and a duality with kernel methods. Ann. Statist. 17 58–106.
  • Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and its Applications. Chapman and Hall, London.
  • Fan, J., Härdle, W. and Mammen, E. (1998). Direct estimation of low-dimensional components in additive models. Ann. Statist. 26 943–971.
  • Florens, J.-P., Ivaldi, M. and Larribeau, S. (1996). Sobolev estimation of approximate regressions. Econometric Theory 12 753–772.
  • Friedman, J. H. (1991). Multivariate adaptive regression splines (with discussion). Ann. Statist. 19 1–141.
  • Friedman, J. H. and Stuetzle, W. (1981). Projection pursuit regression. J. Amer. Statist. Assoc. 76 817–823.
  • Fuss, M. and McFadden, D., eds. (1978). Production Economics: A Dual Approach to Theory and Applications 1. The Theory of Production. North-Holland, Amsterdam.
  • Hastie, T. and Tibshirani, R. (1986). Generalized additive models (with discussion). Statist. Sci. 1 297–318.
  • Hastie, T. and Tibshirani, R. (1993). Varying-coefficient models (with discussion). J. Roy. Statist. Soc. Ser. B 55 757–796.
  • Huber, P. J. (1985). Projection pursuit (with discussion). Ann. Statist. 13 435–525.
  • Jorgenson, D. (1986). Econometric methods for modeling producer behavior. In Handbook of Econometrics 3 (Z. Griliches and M. D. Intriligator, eds.) 1841–1915. North-Holland, Amsterdam.
  • Linton, O. and Nielsen, J. (1995). A kernel method of estimating structured nonparametric regression based on marginal integration. Biometrika 82 93–100.
  • Murray-Smith, R. and Sbarbaro, D. (2002). Nonlinear adaptive control using non-parametric Gaussian process prior models. In 15th IFAC World Congress on Automatic Control 21–26. Barcelona.
  • Silberberg, E. and Suen, W. (2001). The Structure of Economics: A Mathematical Analysis. McGraw-Hill, New York.
  • Simonoff, J. S. (1996). Smoothing Methods in Statistics. Springer, New York.
  • Solak, E., Murray-Smith, R., Leithead, W. E., Leith, D. J. and Rasmussen, C. E. (2003). Derivative observations in Gaussian Process models of dynamic systems. In Advances in Neural Information Processing Systems 15 1033–1040. MIT Press, Cambridge, MA.
  • Stone, C. J. (1980). Optimal rates of convergence for nonparametric estimators. Ann. Statist. 8 1348–1360.
  • Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10 1040–1053.
  • Stone, C. J. (1985). Additive regression and other nonparametric models. Ann. Statist. 13 689–705.
  • Varian, H. (1992). Microeconomic Analysis. Norton, New York.
  • Yatchew, A. (2003). Semiparametric Regression for the Applied Econometrician. Cambridge Univ. Press.
  • Wand, M. P. and Jones, M. C. (1995). Kernel Smoothing. Chapman and Hall, London.
  • Zhang, H. (2004). Recursive partitioning and tree-based methods. In Handbook of Computational Statistics (J. E. Gentle, W. Härdle and Y. Mori, eds.) 813–840. Springer, Berlin.