The Annals of Statistics

Approximation and estimation of s-concave densities via Rényi divergences

Qiyang Han and Jon A. Wellner

Full-text: Open access

Enhanced PDF (296 KB)

Abstract

In this paper, we study the approximation and estimation of $s$-concave densities via Rényi divergence. We first show that the approximation of a probability measure $Q$ by an $s$-concave density exists and is unique via the procedure of minimizing a divergence functional proposed by [Ann. Statist. 38 (2010) 2998–3027] if and only if $Q$ admits full-dimensional support and a first moment. We also show continuity of the divergence functional in $Q$: if $Q_{n}\to Q$ in the Wasserstein metric, then the projected densities converge in weighted $L_{1}$ metrics and uniformly on closed subsets of the continuity set of the limit. Moreover, directional derivatives of the projected densities also enjoy local uniform convergence. This contains both on-the-model and off-the-model situations, and entails strong consistency of the divergence estimator of an $s$-concave density under mild conditions. One interesting and important feature for the Rényi divergence estimator of an $s$-concave density is that the estimator is intrinsically related with the estimation of log-concave densities via maximum likelihood methods. In fact, we show that for $d=1$ at least, the Rényi divergence estimators for $s$-concave densities converge to the maximum likelihood estimator of a log-concave density as $s\nearrow0$. The Rényi divergence estimator shares similar characterizations as the MLE for log-concave distributions, which allows us to develop pointwise asymptotic distribution theory assuming that the underlying density is $s$-concave.

Article information

Source
Ann. Statist., Volume 44, Number 3 (2016), 1332-1359.

Dates
Received: June 2015
Revised: October 2015
First available in Project Euclid: 11 April 2016

Permanent link to this document
https://projecteuclid.org/euclid.aos/1460381695

Digital Object Identifier
doi:10.1214/15-AOS1408

Mathematical Reviews number (MathSciNet)
MR3485962

Zentralblatt MATH identifier
1338.62105

Subjects
Primary: 62G07: Density estimation 62H12: Estimation
Secondary: 62G05: Estimation 62G20: Asymptotic properties

Keywords
$s$-concavity consistency projection asymptotic distribution mode estimation nonparametric estimation shape constraints

Citation

Han, Qiyang; Wellner, Jon A. Approximation and estimation of s -concave densities via Rényi divergences. Ann. Statist. 44 (2016), no. 3, 1332--1359. doi:10.1214/15-AOS1408. https://projecteuclid.org/euclid.aos/1460381695


Export citation

References

  • Avriel, M. (1972). $r$-convex functions. Math. Program. 2 309–323.
    Mathematical Reviews (MathSciNet): MR301151
    Digital Object Identifier: doi:10.1007/BF01584551
  • Balabdaoui, F., Rufibach, K. and Wellner, J. A. (2009). Limit distribution theory for maximum likelihood estimation of a log-concave density. Ann. Statist. 37 1299–1331.
    Mathematical Reviews (MathSciNet): MR2509075
    Digital Object Identifier: doi:10.1214/08-AOS609
    Project Euclid: euclid.aos/1239369023
  • Balabdaoui, F. and Wellner, J. A. (2007). Estimation of a $k$-monotone density: Limit distribution theory and the spline connection. Ann. Statist. 35 2536–2564.
    Mathematical Reviews (MathSciNet): MR2382657
    Digital Object Identifier: doi:10.1214/009053607000000262
    Project Euclid: euclid.aos/1201012971
  • Basu, A., Harris, I. R., Hjort, N. L. and Jones, M. C. (1998). Robust and efficient estimation by minimising a density power divergence. Biometrika 85 549–559.
    Mathematical Reviews (MathSciNet): MR1665873
    Digital Object Identifier: doi:10.1093/biomet/85.3.549
  • Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113–150.
    Mathematical Reviews (MathSciNet): MR1240719
    Digital Object Identifier: doi:10.1007/BF01199316
  • Borell, C. (1974). Convex measures on locally convex spaces. Ark. Mat. 12 239–252.
    Mathematical Reviews (MathSciNet): MR388475
  • Borell, C. (1975). Convex set functions in $d$-space. Period. Math. Hungar. 6 111–136.
    Mathematical Reviews (MathSciNet): MR404559
    Digital Object Identifier: doi:10.1007/BF02018814
  • Brascamp, H. J. and Lieb, E. H. (1976). On extensions of the Brunn–Minkowski and Prékopa–Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation. J. Funct. Anal. 22 366–389.
    Mathematical Reviews (MathSciNet): MR450480
    Digital Object Identifier: doi:10.1016/0022-1236(76)90004-5
  • Brunk, H. D. (1970). Estimation of isotonic regression. In Nonparametric Techniques in Statistical Inference (Proc. Sympos., Indiana Univ., Bloomington, Ind., 1969) 177–197. Cambridge Univ. Press, London.
    Mathematical Reviews (MathSciNet): MR277070
  • Cule, M. and Samworth, R. (2010). Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density. Electron. J. Stat. 4 254–270.
    Mathematical Reviews (MathSciNet): MR2645484
    Digital Object Identifier: doi:10.1214/09-EJS505
    Project Euclid: euclid.ejs/1266416850
  • Cule, M., Samworth, R. and Stewart, M. (2010). Maximum likelihood estimation of a multi-dimensional log-concave density. J. R. Stat. Soc. Ser. B Stat. Methodol. 72 545–607.
    Mathematical Reviews (MathSciNet): MR2758237
    Digital Object Identifier: doi:10.1111/j.1467-9868.2010.00753.x
  • Das Gupta, S. (1976). $S$-unimodal function: Related inequalities and statistical applications. Sankhyā Ser. B 38 301–314.
  • Dharmadhikari, S. and Joag-Dev, K. (1988). Unimodality, Convexity, and Applications. Academic Press, Boston, MA.
    Mathematical Reviews (MathSciNet): MR954608
  • Doss, C. and Wellner, J. A. (2016). Global rates of convergence of the MLEs of log-concave and s-concave densities. Ann. Statist. 44 954–981.
    Mathematical Reviews (MathSciNet): MR3485950
    Digital Object Identifier: doi:10.1214/15-AOS1394
    Project Euclid: euclid.aos/1460381683
  • Dümbgen, L. and Rufibach, K. (2009). Maximum likelihood estimation of a log-concave density and its distribution function: Basic properties and uniform consistency. Bernoulli 15 40–68.
    Mathematical Reviews (MathSciNet): MR2546798
    Digital Object Identifier: doi:10.3150/08-BEJ141
    Project Euclid: euclid.bj/1233669882
  • Dümbgen, L., Samworth, R. and Schuhmacher, D. (2011). Approximation by log-concave distributions, with applications to regression. Ann. Statist. 39 702–730.
    Mathematical Reviews (MathSciNet): MR2816336
    Digital Object Identifier: doi:10.1214/10-AOS853
    Project Euclid: euclid.aos/1299680952
  • Groeneboom, P. (1985). Estimating a monotone density. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, Vol. II (Berkeley, Calif., 1983). Wadsworth Statist./Probab. Ser. 539–555. Wadsworth, Belmont, CA.
    Mathematical Reviews (MathSciNet): MR822052
  • Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2001). Estimation of a convex function: Characterizations and asymptotic theory. Ann. Statist. 29 1653–1698.
    Mathematical Reviews (MathSciNet): MR1891742
    Digital Object Identifier: doi:10.1214/aos/1015345958
    Project Euclid: euclid.aos/1015345958
  • Guntuboyina, A. and Sen, B. (2015). Global risk bounds and adaptation in univariate convex regression. Probab. Theory Related Fields 163 379–411.
    Mathematical Reviews (MathSciNet): MR3405621
    Digital Object Identifier: doi:10.1007/s00440-014-0595-3
  • Han, Q. and Wellner, J. A. (2015). Supplement to “Approximation and estimation of $s$-concave densities via Rényi divergences.” DOI:10.1214/15-AOS1408SUPP.
    Mathematical Reviews (MathSciNet): MR3485962
    Digital Object Identifier: doi:10.1214/15-AOS1408
    Project Euclid: euclid.aos/1460381695
  • Jongbloed, G. (2000). Minimax lower bounds and moduli of continuity. Statist. Probab. Lett. 50 279–284.
    Mathematical Reviews (MathSciNet): MR1792307
  • Kim, A. K. and Samworth, R. J. (2015). Global rates of convergence in log-concave density estimation. Preprint. Available at arXiv:1404.2298v2.
    arXiv: 1404.2298v2
  • Koenker, R. and Mizera, I. (2010). Quasi-concave density estimation. Ann. Statist. 38 2998–3027.
    Mathematical Reviews (MathSciNet): MR2722462
    Digital Object Identifier: doi:10.1214/10-AOS814
    Project Euclid: euclid.aos/1282315406
  • Koenker, R. and Mizera, I. (2014). Convex optimization in R. J. Stat. Softw. 60(5) 1–23.
  • Lang, R. (1986). A note on the measurability of convex sets. Arch. Math. (Basel) 47 90–92.
    Mathematical Reviews (MathSciNet): MR855142
    Digital Object Identifier: doi:10.1007/BF01202504
  • MOSEK ApS Denmark (2011). The MOSEK Optimization Tools Manual, Version 6.0. Available at http://www.mosek.com.
  • Pal, J. K., Woodroofe, M. and Meyer, M. (2007). Estimating a Polya frequency function${}_{2}$. In Complex Datasets and Inverse Problems. Institute of Mathematical Statistics Lecture Notes—Monograph Series 54 239–249. IMS, Beachwood, OH.
    Mathematical Reviews (MathSciNet): MR2459192
    Digital Object Identifier: doi:10.1214/074921707000000184
  • Prakasa Rao, B. L. S. (1969). Estimation of a unimodal density. Sankhyā Ser. A 31 23–36.
    Mathematical Reviews (MathSciNet): MR267677
  • Rinott, Y. (1976). On convexity of measures. Ann. Probab. 4 1020–1026.
    Mathematical Reviews (MathSciNet): MR428540
    Digital Object Identifier: doi:10.1214/aop/1176995947
  • Rockafellar, R. T. (1971). Integrals which are convex functionals. II. Pacific J. Math. 39 439–469.
    Mathematical Reviews (MathSciNet): MR310612
    Digital Object Identifier: doi:10.2140/pjm.1971.39.439
    Project Euclid: euclid.pjm/1102969571
  • Rockafellar, R. T. (1997). Convex Analysis. Princeton Landmarks in Mathematics. Princeton Univ. Press, Princeton, NJ.
    Mathematical Reviews (MathSciNet): MR1451876
  • Seregin, A. and Wellner, J. A. (2010). Nonparametric estimation of multivariate convex-transformed densities. Ann. Statist. 38 3751–3781.
    Mathematical Reviews (MathSciNet): MR2766867
    Digital Object Identifier: doi:10.1214/10-AOS840
    Project Euclid: euclid.aos/1291126972
  • Uhrin, B. (1984). Some remarks about the convolution of unimodal functions. Ann. Probab. 12 640–645.
    Mathematical Reviews (MathSciNet): MR735860
    Digital Object Identifier: doi:10.1214/aop/1176993312
    Project Euclid: euclid.aop/1176993312
  • van de Geer, S. A. (2000). Applications of Empirical Process Theory. Cambridge Series in Statistical and Probabilistic Mathematics 6. Cambridge Univ. Press, Cambridge.
    Mathematical Reviews (MathSciNet): MR1739079
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1385671
  • Walther, G. (2002). Detecting the presence of mixing with multiscale maximum likelihood. J. Amer. Statist. Assoc. 97 508–513.
    Mathematical Reviews (MathSciNet): MR1941467
    Digital Object Identifier: doi:10.1198/016214502760047032
  • Wright, F. T. (1981). The asymptotic behavior of monotone regression estimates. Ann. Statist. 9 443–448.
    Mathematical Reviews (MathSciNet): MR606630
    Digital Object Identifier: doi:10.1214/aos/1176345411
    Project Euclid: euclid.aos/1176345411

Supplemental materials

  • Supplement to “Approximation and estimation of $s$-concave densities via Rényi divergences”. In the supplement Han and Wellner (2015), we provide details of the omitted proofs for Sections 2, 3, 4 and 6 and some auxiliary results from convex analysis used in the main paper.
    Digital Object Identifier: doi:10.1214/15-AOS1408SUPP
    Supplemental PDF (435 KB)

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more