Annales de l'Institut Henri Poincaré, Probabilités et Statistiques

Adaptive pointwise estimation of conditional density function

Karine Bertin, Claire Lacour, and Vincent Rivoirard

Full-text: Open access


In this paper we consider the problem of estimating $f$, the conditional density of $Y$ given $X$, by using an independent sample distributed as $(X,Y)$ in the multivariate setting. We consider the estimation of $f(x,\cdot)$ where $x$ is a fixed point. We define two different procedures of estimation, the first one using kernel rules, the second one inspired from projection methods. Both adaptive estimators are tuned by using the Goldenshluger and Lepski methodology. After deriving lower bounds, we show that these procedures satisfy oracle inequalities and are optimal from the minimax point of view on anisotropic Hölder balls. Furthermore, our results allow us to measure precisely the influence of $\mathrm{f}_{X}(x)$ on rates of convergence, where $\mathrm{f}_{X}$ is the density of $X$. Finally, some simulations illustrate the good behavior of our tuned estimates in practice.


Dans cet article, nous considérons le problème de l’estimation de $f$, la densité conditionnelle de $Y$ sachant $X$, en utilisant un échantillon de même loi que $(X,Y)$, dans le cadre multivarié. On considère l’estimation de $f(x,\cdot)$ où $x$ est un point fixé. Nous définissons deux procédures d’estimation différentes, la première utilisant des estimateurs à noyau, alors que la seconde s’inspire des méthodes de projection. Les deux procédures adaptatives sont calibrées en utilisant la méthodologie proposée par Goldenshulger et Lepski. Une fois obtenu le calcul des bornes inférieures du risque, nous montrons que ces procédures satisfont des inégalités oracles et sont optimales du point de vue minimax sur les boules de Hölder anisotropes. De plus, nos résultats nous permettent de mesurer précisément l’influence de $\mathrm{f}_{X}(x)$ sur les vitesses convergence, où $\mathrm{f}_{X}$ est la densité de $X$. Finalement, des simulations numériques illustrent le bon comportement de nos procédures calibrées en pratique.

Article information

Ann. Inst. H. Poincaré Probab. Statist., Volume 52, Number 2 (2016), 939-980.

Received: 23 December 2013
Revised: 8 October 2014
Accepted: 17 December 2014
First available in Project Euclid: 4 May 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation 62G20: Asymptotic properties

Conditional density Adaptive estimation Kernel rules Projection estimates Oracle inequality Minimax rates Anisotropic Hölder spaces


Bertin, Karine; Lacour, Claire; Rivoirard, Vincent. Adaptive pointwise estimation of conditional density function. Ann. Inst. H. Poincaré Probab. Statist. 52 (2016), no. 2, 939--980. doi:10.1214/14-AIHP665.

Export citation


  • [1] N. Akakpo and C. Lacour. Inhomogeneous and anisotropic conditional density estimation from dependent data. Electron. J. Stat. 5 (2011) 1618–1653.
  • [2] A. Barron, L. Birgé and P. Massart. Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 (3) (1999) 301–413.
  • [3] D. M. Bashtannyk and R. J. Hyndman. Bandwidth selection for kernel conditional density estimation. Comput. Statist. Data Anal. 36 (3) (2001) 279–298.
  • [4] M. Beaumont, W. Zhang and D. Balding. Approximate Bayesian computation in population genetics. Genetics 162 (4) (2002) 2025–2035.
  • [5] K. Bertin, C. Lacour and V. Rivoirard. Adaptive pointwise estimation of conditional density function. Preprint, 2013. Available at arXiv:1312.7402v1.
  • [6] G. Biau, F. Cérou and A. Guyader. New insights into approximate Bayesian computation. Ann. Inst. Henri Poincaré Probab. Stat. 51 (2015) 376–403.
  • [7] L. Birgé and P. Massart. Minimum contrast estimators on sieves: Exponential bounds and rates of convergence. Bernoulli 4 (3) (1998) 329–375.
  • [8] M. Blum. Approximate Bayesian computation: A nonparametric perspective. J. Amer. Statist. Assoc. 105 (491) (2010) 1178–1187.
  • [9] O. Bouaziz and O. Lopez. Conditional density estimation in a censored single-index regression model. Bernoulli 16 (2) (2010) 514–542.
  • [10] E. Brunel, F. Comte and C. Lacour. Adaptive estimation of the conditional density in the presence of censoring. Sankhyā 69 (4) (2007) 734–763.
  • [11] G. Chagny. Warped bases for conditional density estimation. Math. Methods Statist. 22 (2013) 253–282.
  • [12] X. Chen, O. Linton and P. Robinson. The estimation of conditional densities. In Asymptotics in Statistics and Probability: Papers in Honor of George Gregory Roussas 71–84. M. L. Puri (Ed.). VSP, Utrecht, 2000.
  • [13] S. Clémençon. Adaptive estimation of the transition density of a regular Markov chain. Math. Methods Statist. 9 (4) (2000) 323–357.
  • [14] S. X. Cohen and E. Le Pennec. Partition-based conditional density estimation. ESAIM Probab. Stat. 17 (2013) 672–697.
  • [15] J. G. De Gooijer and D. Zerom. On conditional density estimation. Statist. Neerlandica 57 (2) (2003) 159–176.
  • [16] S. Efromovich. Conditional density estimation in a regression setting. Ann. Statist. 35 (6) (2007) 2504–2535.
  • [17] S. Efromovich. Oracle inequality for conditional density estimation and an actuarial example. Ann. Inst. Statist. Math. 62 (2) (2010) 249–275.
  • [18] J. Fan, Q. Yao and H. Tong. Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika 83 (1) (1996) 189–206.
  • [19] J. Fan and T. H. Yim. A crossvalidation method for estimating conditional densities. Biometrika 91 (4) (2004) 819–834.
  • [20] O. P. Faugeras. A quantile-copula approach to conditional density estimation. J. Multivariate Anal. 100 (9) (2009) 2083–2099.
  • [21] A. Goldenshluger and O. Lepski. Universal pointwise selection rule in multivariate function estimation. Bernoulli 14 (4) (2008) 1150–1190.
  • [22] A. Goldenshluger and O. Lepski. Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality. Ann. Statist. 39 (3) (2011) 1608–1632.
  • [23] A. Goldenshluger and O. Lepski. On adaptive minimax density estimation on $\mathbb{R}^{d}$. Probab. Theory and Related Fields 159 (3–4) (2014) 479–543.
  • [24] A. Goldenshluger and O. Lepski. General selection rule from a family of linear estimators. Theory Probab. Appl. 57 (2013) 209–226.
  • [25] L. Györfi and M. Kohler. Nonparametric estimation of conditional distributions. IEEE Trans. Inform. Theory 53 (5) (2007) 1872–1879.
  • [26] P. Hall, J. Racine and Q. Li. Cross-validation and the estimation of conditional probability densities. J. Amer. Statist. Assoc. 99 (468) (2004) 1015–1026.
  • [27] R. J. Hyndman, D. M. Bashtannyk and G. K. Grunwald. Estimating and visualizing conditional densities. J. Comput. Graph. Statist. 5 (4) (1996) 315–336.
  • [28] R. J. Hyndman and Q. Yao. Nonparametric estimation and symmetry tests for conditional density functions. J. Nonparametr. Stat. 14 (3) (2002) 259–278.
  • [29] J. Jeon and J. W. Taylor. Using conditional kernel density estimation for wind power density forecasting. J. Amer. Statist. Assoc. 107 (497) (2012) 66–79.
  • [30] G. Kerkyacharian, O. Lepski and D. Picard. Nonlinear estimation in anisotropic multi-index denoising. Probab. Theory Related Fields 121 (2) (2001) 137–170.
  • [31] T. Klein and E. Rio. Concentration around the mean for maxima of empirical processes. Ann. Probab. 33 (3) (2005) 1060–1077.
  • [32] P. Reynaud-Bouret, V. Rivoirard and C. Tuleau-Malot. Adaptive density estimation: A curse of support? J. Statist. Plann. Inference 141 (1) (2011) 115–139.
  • [33] M. Rosenblatt. Conditional probability density and regression estimators. In Multivariate Analysis II (Proc. Second Internat. Sympos., Dayton, Ohio, 1968) 25–31. Academic Press, New York, 1969.
  • [34] M. Sart. Estimation of the transition density of a Markov chain. Ann. Inst. Henri Poincaré Probab. Stat. 50 (2014) 1028–1068.
  • [35] B. W. Silverman. Density Estimation for Statistics and Data Analysis. Monographs on Statistics and Applied Probability. Chapman & Hall, London, 1986.
  • [36] C. J. Stone. The use of polynomial splines and their tensor products in multivariate function estimation. Ann. Statist. 22 (1) (1994) 118–184.
  • [37] I. Takeuchi, K. Nomura and T. Kanamori. Nonparametric conditional density estimation using piecewise-linear solution path of kernel quantile regression. Neural Comput. 21 (2) (2009) 533–559.