Open Access
October 2001 Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities
Subhashis Ghosal, Aad W. van der Vaart
Ann. Statist. 29(5): 1233-1263 (October 2001). DOI: 10.1214/aos/1013203452

Abstract

We study the rates of convergence of the maximum likelihood estimator (MLE) and posterior distribution in density estimation problems, where the densities are location or location-scale mixtures of normal distributions with the scale parameter lying between two positive numbers. The true density is also assumed to lie in this class with the true mixing distribution either compactly supported or having sub-Gaussian tails. We obtain bounds for Hellinger bracketing entropies for this class, and from these bounds, we deduce the convergence rates of (sieve) MLEs in Hellinger distance. The rate turns out to be $(log n)^\kappa /\sqrt{n}$, where $\kappa \ge 1$ is a constant that depends on the type of mixtures and the choice of the sieve. Next, we consider a Dirichlet mixture of normals as a prior on the unknown density. We estimate the prior probability of a certain Kullback-Leibler type neighborhood and then invoke a general theorem that computes the posterior convergence rate in terms the growth rate of the Hellinger entropy and the concentration rate of the prior. The posterior distribution is also seen to converge at the rate $(log n)^\kappa /\sqrt{n}$, where $\kappa$ now depends on the tail behavior of the base measure of the Dirichlet process.

Citation

Download Citation

Subhashis Ghosal. Aad W. van der Vaart. "Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities." Ann. Statist. 29 (5) 1233 - 1263, October 2001. https://doi.org/10.1214/aos/1013203452

Information

Published: October 2001
First available in Project Euclid: 8 February 2002

zbMATH: 1043.62025
MathSciNet: MR1873329
Digital Object Identifier: 10.1214/aos/1013203452

Subjects:
Primary: 62G07 , 62G20

Keywords: bracketing , Dirichlet mixture , Entropy , maximum likelihood , mixture of normals , posterior distribution , rate of convergence , sieve

Rights: Copyright © 2001 Institute of Mathematical Statistics

Vol.29 • No. 5 • October 2001
Back to Top