Annals of Statistics

On nonparametric regression for IID observations in a general setting

Sam Efromovich

Full-text: Open access


We consider the problem of sharp-optimal estimation of a response function $f(x)$ in a random design nonparametric regression under a general model where a pair of observations $(Y, X)$ has a joint density $p(y, x) = p(y|f(x)) \pi(x)$. We wish to estimate the response function with optimal minimax mean integrated squared error convergence as the sample size tends to $\infty$. Traditional regularity assumptions on the conditional density $p(y| \theta)$ assumed for parameter $\theta$ estimation are sufficient for sharp-optimal nonparametric risk convergence as well as for the existence of the best constant and rate of risk convergence. This best constant is a nonparametric analog of Fisher information. Many examples are sketched including location and scale families, censored data, mixture models and some well-known applied examples. A sequential approach and some aspects of experimental design are considered as well.

Article information

Ann. Statist., Volume 24, Number 3 (1996), 1126-1144.

First available in Project Euclid: 20 September 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation
Secondary: 62G20: Asymptotic properties 62J02: General nonlinear regression 62E20: Asymptotic distribution theory 62F35: Robustness and adaptive procedures

Nonparametric regression curves estimation sharp-optimal risk convergence sequential estimation location and scale families censored data mixtures


Efromovich, Sam. On nonparametric regression for IID observations in a general setting. Ann. Statist. 24 (1996), no. 3, 1126--1144. doi:10.1214/aos/1032526960.

Export citation


  • ADAMS, R. A. 1975. Sobolev Spaces. Academic Press, New York. Z.
  • BARY, N. K. 1964. A Treatise on Trigonometric Series. Pergamon, Oxford. Z.
  • BICKEL, P. J., KLAASSEN, C. A. J., RITOV, Y. and WELLNER, J. A. 1993. Efficient and Adaptive Estimation for Semiparametric Models. Johns Hopkins Univ. Press. Z. Z.
  • BOROVKOV, A. A. 1984. Mathematical Statistics in Russian. Nauka, Moscow. Z.
  • DEMPSTER, A. P., LAIRD, N. M. and RUBIN, D. B. 1977. Maximum likelihood for incomplete data via the EM algorithm. J. Roy. Statist. Soc. Ser. B 39 1 22. Z.
  • DONOHO, D. L., LIU, R. C. and MACGIBBON, B. 1990. Minimax risk for hy perrectangles, and implications. Ann. Statist. 18 1416 1437. Z.
  • EFROMOVICH, S. 1985. Nonparametric estimation of a density with unknown smoothness. Theory Probab. Appl. 30 557 568. Z.
  • EFROMOVICH, S. 1986. The adaptive algorithm of nonparametric regression. In Abstracts of Second IFAC Sy mposium Stochastic Control 2 112 114. Vilnius Univ. Press. Z.
  • EFROMOVICH, S. 1989. On sequential nonparametric estimation of a density. Theory Probab. Appl. 34 228 239. Z.
  • EFROMOVICH, S. 1992. On nonparametric regression for iid observations in general setting. Technical report, Dept. Mathematics and Statistics, Univ. New Mexico.
  • EFROMOVICH, A. and PINSKER, M. S. 1996. Sharp-optimal and adaptive estimation for heteroscedastic nonparametric regression. Statist. Sinica 6. To appear. Z.
  • EFROMOVICH, S. and THOMAS, E. 1996. Applications of nonparametric binary regression to evaluate the sensitivity of explosives. Technometrics 38 50 58. Z.
  • EUBANK, R. L. 1988. Spline Smoothing and Nonparametric Regression. Dekker, New York. Z.
  • FAN, J. 1993. Local linear regression smoothers and their minimax efficiency. Ann. Statist. 21 196 216. Z.
  • GOLUBEV, G. K. 1991. LAN in problems of nonparametric estimation of functions and lower bounds for quadratic risks. Theory Probab. Appl. 36 152 157. Z.
  • GOLUBEV, G. K. and NUSSBAUM, M. 1990. A risk bound in Sobolev class regression. Ann. Statist. 18 758 778. Z.
  • GRENANDER, U. and SZEGO, G. 1958. Toeplitz Forms and Their Applications. Univ. California ¨ Press. Z.
  • HARDLE, W. 1990. Applied Nonparametric Regression. Cambridge Univ. Press. ¨ Z.
  • IBRAGIMOV, I. A. and KHASMINSKII, R. Z. 1981. Statistical Estimation: Asy mptotic Theory. Springer, New York. Z.
  • LEHMANN, E. L. 1983. Theory of Point Estimation. Wadsworth and Brooks Cole, Pacific Grove, CA. Z.
  • MULLER, H.-G. 1988. Nonparametric Regression Analy sis of Longitudinal Data. Springer, New ¨ York.Z.
  • NUSSBAUM, M. 1985. Spline smoothing in regression models and asy mptotic efficiency in L. 2 Ann. Statist. 13 984 997. Z.
  • PINSKER, M. S. 1980. Optimal filtration of functions from L in Gaussian noise. Problems 2 Inform. Transmission 16 52 68. Z.
  • SPECKMAN, P. 1985. Spline smoothing and optimal rates of convergence in nonparametric regression models. Ann. Statist. 13 970 983. Z.
  • WAHBA, G. 1990. Spline Models for Observational Data. SIAM, Philadelphia.