The Annals of Statistics

A universally acceptable smoothing factor for kernel density estimates

Luc Devroye and Gábor Lugosi

Full-text: Open access


We define a minimum distance estimate of the smoothing factor for kernel density estimates, based on a methodology first developed by Yatracos. It is shown that if $f_{nh}$ denotes the kernel density estimate on $\mathbb{R}^d$ for an i.i.d. sample of size n drawn from an unknown density f, where h is the smoothing factor, and if $f_n$ is the kernel estimate with the same kernel and with the proposed new data-based smoothing factor, then, under a regularity condition on the kernel K, $$\sup_f \limsup_{n \to \infty} \frac{E \int | f_n - f|dx}{\inf_{h>0} E \int |f_{nh} - f|dx} \leq 3.$$ This is the first published smoothing factor that can be proven to have this property.

Article information

Ann. Statist., Volume 24, Number 6 (1996), 2499-2512.

First available in Project Euclid: 16 September 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation

Density estimation kernel estimate convergence smoothing factor minimum distance estimate asymptotic optimality


Devroye, Luc; Lugosi, Gábor. A universally acceptable smoothing factor for kernel density estimates. Ann. Statist. 24 (1996), no. 6, 2499--2512. doi:10.1214/aos/1032181164.

Export citation


  • Akaike, H. (1954). An approximation to the density function. Ann. Inst. Statist. Math. 6 127-132.
  • Assouad, P. (1983). Deux remarques sur l'estimation. C. R. Acad. Sci. Paris S´er. I Math. 296 1021-1024.
  • Berlinet, A. and Devroy e, L. (1994). A comparison of kernel density estimates. Publ. Inst. Statist. Univ. Paris 38 3-59.
  • Birg´e, L. (1985). Non-asy mptotic minimax risk for Hellinger balls. Probab. Math. Statist. 5 21-29.
  • Birg´e, L. (1986). On estimating a density using Hellinger distance and some other strange facts. Probab. Theory Related Fields 71 271-291. Birg´e, L. (1987a). On the risk of histograms for estimating decreasing densities. Ann. Statist. 15 1013-1022. Birg´e, L. (1987b). Estimating a density under order restrictions: nonasy mptotic minimax risk. Ann. Statist. 15 995-1012.
  • Birg´e, L. (1989). The Grenander estimator: a nonasy mptotic approach. Ann. Statist. 17 1532- 1549.
  • Bowman, A. W. (1984). An alternative method of cross-validation for the smoothing of density estimates. Biometrika 71 353-360.
  • Bretagnolle, J. and Huber, C. (1979). Estimation des densit´es: risque mimimax. Z. Wahrsch. Verw. Gebiete 47 119-137.
  • Cao, R., Cuevas, A. and Gonz´alez-Manteiga, W. (1994). A comparative study of several smoothing methods in density estimation. Comput. Statist. Data Anal. 17 153-176. Deheuvels, P. (1977a). Estimation non param´etrique de la densit´e par histogrammes generalis´es. Rev. Statist. Appl. 25 5-42. Deheuvels, P. (1977b). Estimation nonparam´etrique de la densit´e par histogrammes generalis´es. Publ. Inst. Statist. Univ. Paris 22 1-23.
  • Deheuvels, P. and Hominal, P. (1980). Estimation automatique de la densit´e. Rev. Statist. Appl. 28 25-55.
  • Devroy e, L. (1983). The equivalence of weak, strong and complete convergence in L1 for kernel density estimates. Ann. Statist. 11 896-904.
  • Devroy e, L. (1987). A Course in Density Estimation. Birkh¨auser, Boston.
  • Devroy e, L. (1988). Asy mptotic performance bounds for the kernel estimate. Ann. Statist. 16 1162-1179. Devroy e, L. (1989a). A universal lower bound for the kernel estimate. Statist. Probab. Lett. 8 419-423. Devroy e, L. (1989b). The double kernel method in density estimation. Ann. Inst. H. Poincar´e 25 533-580.
  • Devroy e, L. (1991). Exponential inequalities in nonparametric estimation. In Nonparametric Functional Estimation and Related Topics (G. Roussas, ed.) 31-44. Kluwer, Dordrecht.
  • Devroy e, L. (1992). A note on the usefulness of superkernels in density estimation. Ann. Statist. 20 2037-2056.
  • Devroy e, L. (1994). On good deterministic smoothing sequences for kernel density estimates. Ann. Statist. 22 886-889.
  • Devroy e, L. and Gy ¨orfi, L. (1985). Nonparametric Density Estimation: The L1 View. Wiley, New York.
  • Devroy e, L. and Penrod, C. S. (1984). Distribution-free lower bounds in density estimation. Ann. Statist. 12 1250-1262.
  • Donoho, D. L., Kerky acharian, G. and Picard, D. (1995). Wavelet shrinkage: asy mptotia? (with discussion). J. Roy. Statist. Soc. Ser. B 57 301-369.
  • Donoho, D. L. and Liu, R. C. (1991). Geometrizing rates of convergence. III. Ann. Statist. 19 668-701.
  • Hall, P. (1989). On convergence rates in nonparametric problems. Internat. Statist. Rev. 57 45-58.
  • Hall, P. and Wand, M. P. (1988). Minimizing L1 distance in nonparametric density estimation. J. Multivariate Anal. 26 59-88.
  • Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13-30.
  • Ibragimov, I. A. and Khasminskii, R. Z. (1982). Estimation of distribution density belonging to a class of entire function. Theory Probab. Appl. 27 551-562.
  • Low, M. (1992). Non-existence of an adaptive estimator for the value of an unknown probability density. Ann. Statist. 20 598-602.
  • Marron, J. S. (1988). Automatic smoothing parameter selection: a survey. Empirical Economics 13 187-208.
  • Marron, J. S. (1989). Automatic smoothing parameter selection: a survey. In Semiparametric and Nonparametric Economics (A. Ullah, ed.) 65-86. Heidelberg.
  • McDiarmid, C. (1989). On the method of bounded differences. In Survey s in Combinatorics 1989. London Math. Soc. Lecture Notes Ser. 141 148-188. Cambridge Univ. Press.
  • Park, B.-U. and Turlach, B. A. (1992). Practical performance of several data driven bandwidth selectors (with discussion). Comput. Statist. 7 251-270.
  • Parzen, E. (1962). On the estimation of a probability density function and the mode. Ann. Math. Statist. 33 1065-1076.
  • Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density function. Ann. Math. Statist. 27 832-837.
  • Rudemo, M. (1982). Empirical choice of histograms and kernel density estimators. Scand. J. Statist. 9 65-78.
  • Stone, C. J. (1984). An asy mptotically optimal window selection rule for kernel density estimates. Ann. Statist. 12 1285-1297.
  • Turlach, B. A. (1993). Bandwidth selection in kernel density estimation: a review. Technical report, Univ. Catholique de Louvain.
  • Yatracos, Y. G. (1985). Rates of convergence of minimum distance estimators and Kolmogorov's entropy. Ann. Statist. 13 768-774.