The Annals of Statistics

The silhouette, concentration functions and ML-density estimation under order restrictions

Wolfgang Polonik

Full-text: Open access

Abstract

Based on empirical Lévy-type concentration functions, a new graphical representation of the ML-density estimator under order restrictions is given. This representation generalizes the well-known representation of the Grenander estimator of a monotone density as the slope of the least concave majorant of the empirical distribution function to higher dimensions and arbitrary order restrictions. From the given representation it follows that a density estimator called silhouette, which arises naturally out of the excess mass approach, is the ML-density estimator under order restrictions. This fact provides a new point of view to ML-density estimation from which one gains additional insight to this problem, as demonstrated in the present paper.

Article information

Source
Ann. Statist., Volume 26, Number 5 (1998), 1857-1877.

Dates
First available in Project Euclid: 21 June 2002

Permanent link to this document
https://projecteuclid.org/euclid.aos/1024691360

Digital Object Identifier
doi:10.1214/aos/1024691360

Mathematical Reviews number (MathSciNet)
MR1673281

Zentralblatt MATH identifier
1073.62523

Subjects
Primary: 62G07
Secondary: 62A10 62G30 62G20

Keywords
Empirical processes excess mass Grenander density estimator level set estimation least concave majorant minimum volume sets nonparametric maximum likelihood estimation

Citation

Polonik, Wolfgang. The silhouette, concentration functions and ML-density estimation under order restrictions. Ann. Statist. 26 (1998), no. 5, 1857--1877. doi:10.1214/aos/1024691360. https://projecteuclid.org/euclid.aos/1024691360


Export citation

References

  • (1972). Robust Estimation of Location: Survey and Advances. Princeton Univ. Press.
  • Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). Statistical Inference Under Order Restrictions. Wiley, London.
  • Chernoff, H. (1964). Estimation of the mode. Ann. Inst. Statist. Math. 16 31-41.
  • Devroy e, L. (1987). A course in density estimation. Birkh¨auser, Boston.
  • Einmahl, J. H. J. and Mason, D. M. (1992). Generalized quantile processes. Ann. Statist. 20 1062-1078.
  • Grenander, U. (1956). On the theory of mortality measuremant, II. Skand. Aktuarietidskr. J. 39 125-153.
  • Groeneboom, P. (1985). Estimating a monotone density. In Proceedings of the Berkeley Conference in Honor of Jerzy Ney mann and Jack Kiefer (L. Le Cam and R. Olshen, eds.) 2. Wadsworth, Monterey, CA.
  • Gr ¨ubel, R. (1988). The length of the shorth. Ann. Statist. 16 619-628.
  • Hartigan, J. A. (1975). Clustering Algorithms. Wiley, New York.
  • Hartigan, J. A. (1987). Estimation of a convex density contour in two dimensions. J. Amer. Statist. Assoc. 82 267-270.
  • Hengartner, W. and Theodorescu, R. (1973). Concentration Functions. Academic Press, New York.
  • Hickey, R. J. (1984). Continuous majorisation and randomness. J. Appl. Probab. 21 924-929.
  • Joe, H. (1993). Generalized majorization orderings. In Stochastic Inequalities (M. Shaked and Y. L. Tong, eds.) 22 145-158. IMS, Hay ward, CA.
  • Lientz, B. P. (1970). Results on nonparametric modal intervals. SIAM J. Appl. Math. 19 356-366.
  • Marshall, A. W. and Olkin, I. (1979). Inequalities: Theory of Majorization and Its Applications. Academic Press, New York.
  • M ¨uller, D. W. and Sawitzki, G. (1987). Using excess mass estimates to investigate the modality of a distribution. Preprint 398, SFB 123, Univ. Heidelberg.
  • M ¨uller, D. W. and Sawitzki, G. (1991). Excess mass estimates and tests of multimodality. J. Amer. Statist. Assoc. 86 738-746.
  • Nolan, D. (1991). The excess mass ellipsoid. J. Multivarite Anal. 39 348-371.
  • Polonik, W. (1992). The excess mass appraoch to cluster analysis and related estimation procedures. Ph.D. dissertation, Univ. Heidelberg. Polonik, W. (1995a). Measuring mass concentrations and estimating density contour clusters: an excess mass approach. Ann. Statist. 23 855-881. Polonik, W. (1995b). Density estimation under qualitative assumptions in higher dimensions. J. Multivariate Anal. 55 61-81.
  • Polonik, W. (1997). Minimum volume sets and generalized quantile processes. Stochastic Process. Appl. 69 1-24.
  • Robertson, T. (1967). On estimating a density measurable with respect to a -lattice. Ann. Math. Statist. 38 482-493.
  • Robertson, T., Wright, F. T. and Dy kstra, R. L. (1988). Order restricted statistical inference. Wiley, New York.
  • Sager, T. W. (1979). An iterative method for estimating a multivariate mode and isopleth. J. Amer. Statist. Assoc. 74 329-339.
  • Sager, T. W. (1982). Nonparametric maximum likelihood estimation of spatial patterns. Ann. Statist. 10 1125-1136.
  • Venter, J. H. (1967). On estimation of the mode. Ann. Math. Statist. 38 1446-1455.
  • Wegman, E. (1969). A note on estimating a unimodal density. Ann. Math. Statist. 40 1661-1667.
  • Wegman, E. (1970). Maximum likelihood estimation of a unimodal density function. Ann. Math. Statist. 41 457-471.