The Annals of Statistics

Convex Models, MLS and Misspecification

Valentin Patilea

Full-text: Open access

Abstract

We analyze the asymptotic behavior of maximum likelihood estimators (MLE) in convex dominated models when the true distribution generating the independent data does not necessarily belong to the model. Inspired by the Hellinger distance and its properties, we introduce a family of divergences (contrast functions) which allow a unified treatment of well- and misspecified convex models. Convergence and rates of convergence of the MLE with respect to our divergences are obtained from inequalities satisfied by these divergences and results from empirical process theory (uniform laws of large numbers and maximal inequalities). As a particular case we recover existing results for Hellinger convergence of MLE in well-specified convex models. Four examples are considered: mixtures of discrete distributions, monotone densities, decreasing failure rate distributions and a finite-dimensional parametric model.

Article information

Source
Ann. Statist., Volume 29, Number 1 (2001), 94-123.

Dates
First available in Project Euclid: 5 August 2001

Permanent link to this document
https://projecteuclid.org/euclid.aos/996986503

Digital Object Identifier
doi:10.1214/aos/996986503

Mathematical Reviews number (MathSciNet)
MR1833960

Zentralblatt MATH identifier
1029.62020

Subjects
Primary: 62A10
Secondary: 62G20: Asymptotic properties 62F12: Asymptotic properties of estimators

Keywords
convex models maximum likelihood misspecification empirical process

Citation

Patilea, Valentin. Convex Models, MLS and Misspecification. Ann. Statist. 29 (2001), no. 1, 94--123. doi:10.1214/aos/996986503. https://projecteuclid.org/euclid.aos/996986503


Export citation

References

  • [1] Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). Statistical Inference under Order Restrictions. Wiley, New York.
  • [2] Barlow, R. E, Marshall, A. and Prochan, F. (1963). Properties of probability distributions with monotone hazard rates. Ann. Math. Statist. 34 375-389.
  • [3] Billingsley, P. (1968). Convergence of Probability Measures. Wiley, New York.
  • [4] Birg´e, L. (1989). The Grenander estimator: a nonasymptotic approach. Ann. Statist. 17 1532-1549.
  • [5] Birg´e, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113-150.
  • CONVEXITY, MLE AND MISSPECIFICATION 123
  • [6] Dahlhaus, R. and Wefelmeyer, W. (1996). Asymptotically optimal estimation in misspecified time series models. Ann. Statist. 24 952-974.
  • [7] Grenander, U. (1956). On the theory of mortality measurement, part II. Scand. Actuar. J. 39 125-153.
  • [8] Huber, P. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proc. Fifth Berkeley Symp. Math. Statist. Probab. 1 221-233. Univ. California Press, Berkeley.
  • [9] Lambert, D. and Tierney, L. (1984). Asymptotic properties of the maximum likelihood estimates in mixed Poisson model. Ann. Statist. 12 1388-1399.
  • [10] Lindsay B. (1983). The geometry of mixture likelihood: a general theory. Ann. Statist. 11 86-94.
  • [11] Koltchinskii, V. I. (1997). M-estimation, convexity and quantiles. Ann. Statist. 25 435-477.
  • [12] L ¨uxmann-Ellinghaus, U. (1987). On the identifiability of mixture of infinitely divisible power series distributions. Statist. Probab. Lett. 5 375-379.
  • [13] Marshall, A. W. (1970). Discussion of Barlow and van Zwet's paper "Asymptotic properties of isotonic estimators for the generalized failure rate function: strong consistency." In Nonparametric Techniques in Statistical Inference (M. L. Puri, ed.) 174-176. Cambridge Univ. Press.
  • [14] Marshall, A. W. and Proschan, F. (1965). Maximum likelihood estimation for distribution with monotone failure rate. Ann. Math. Statist. 36 69-77.
  • [15] Noack, A. (1950). A class of random variables with discrete distribution Ann. Math. Statist. 21 127-132.
  • [16] Milhaud, X. and Mounime, S. (1996). A modified maximum likelihood estimator for infinite mixtures. Preprint, Univ. P. Sabatier, Toulouse.
  • [17] Patilea, V. (1997). Convex models, NPLME and misspecification, Ph.D dissertation, part I. Institute of Statistics, Univ. catholique de Louvain.
  • [18] Pfanzagl J. (1969). On the measurability and consistency of minimum contrast estimates. Metrika 249-272.
  • [19] Pfanzagl J. (1988). Consistency of maximum likelihood estimators for certain nonparametric families, in particular: Mixtures. J. Statist. Plann. Inference 19 137-158.
  • [20] Pfanzagl, J. (1990). Large deviations probabilities for certain nonparametric maximum likelihood estimators. Ann. Statist. 18 1868-1877.
  • [21] Pollard, D. (1984). Convergence of Stochastic Processes. Springer, New York.
  • [22] Rolin, J.-M. (1992). Personal notes.
  • [23] Roberts, W. and Varberg, D. (1973). Convex Functions. Academic Press, New York.
  • [24] Simar, L. (1976). Maximum likelihood estimation of a compound Poisson process. Ann. Statist. 4 1200-1209.
  • [25] van de Geer, S. (1993). Hellinger consistency of certain nonparametric maximum likelihood estimators. Ann. Statist. 21 14-44.
  • [26] van de Geer, S. (2000). Empirical Processes in M-estimation. Cambridge Univ. Press.
  • [27] van der Vaart, A. and Wellner, J. (1996). Weak Convergence and Empirical Process. Springer, New York.
  • [28] Wang, J.-L. (1985). Strong consistency of approximate maximum likelihood estimators with applications to nonparametrics. Ann. Statist. 13 932-946.
  • [29] Wong, W. H. and Shen, X. (1995). Probabilities inequalities for likelihood ratios and convergence rates of sieve MLE's. Ann. Statist. 23 339-362.