## The Annals of Statistics

### Convergence rates of parameter estimation for some weakly identifiable finite mixtures

#### Abstract

We establish minimax lower bounds and maximum likelihood convergence rates of parameter estimation for mean-covariance multivariate Gaussian mixtures, shape-rate Gamma mixtures and some variants of finite mixture models, including the setting where the number of mixing components is bounded but unknown. These models belong to what we call “weakly identifiable” classes, which exhibit specific interactions among mixing parameters driven by the algebraic structures of the class of kernel densities and their partial derivatives. Accordingly, both the minimax bounds and the maximum likelihood parameter estimation rates in these models, obtained under some compactness conditions on the parameter space, are shown to be typically much slower than the usual $n^{-1/2}$ or $n^{-1/4}$ rates of convergence.

#### Article information

Source
Ann. Statist., Volume 44, Number 6 (2016), 2726-2755.

Dates
Revised: January 2016
First available in Project Euclid: 23 November 2016

https://projecteuclid.org/euclid.aos/1479891633

Digital Object Identifier
doi:10.1214/16-AOS1444

Mathematical Reviews number (MathSciNet)
MR3576559

Zentralblatt MATH identifier
1359.62076

Subjects
Primary: 62F15: Bayesian inference 62G05: Estimation
Secondary: 62G20: Asymptotic properties

#### Citation

Ho, Nhat; Nguyen, XuanLong. Convergence rates of parameter estimation for some weakly identifiable finite mixtures. Ann. Statist. 44 (2016), no. 6, 2726--2755. doi:10.1214/16-AOS1444. https://projecteuclid.org/euclid.aos/1479891633

#### References

• [1] Allman, E. S., Matias, C. and Rhodes, J. A. (2009). Identifiability of parameters in latent structure models with many observed variables. Ann. Statist. 37 3099–3132.
• [2] Buchberger, B. (1965). An algorithm for finding the basis elements of the residue class ring of a zero dimensional polynomial ideal. Ph.D. thesis, Johannes Kepler Univ. Linz.
• [3] Caillerie, C., Chazal, F., Dedecker, J. and Michel, B. (2011). Deconvolution for the Wasserstein metric and geometric inference. Electron. J. Stat. 5 1394–1423.
• [4] Chen, H. and Chen, J. (2003). Tests for homogeneity in normal mixtures in the presence of a structural parameter. Statist. Sinica 13 351–365.
• [5] Chen, J. and Li, P. (2009). Hypothesis test for normal mixture models: The EM approach. Ann. Statist. 37 2523–2542.
• [6] Chen, J., Li, P. and Fu, Y. (2012). Inference on the order of a normal mixture. J. Amer. Statist. Assoc. 107 1096–1105.
• [7] Chen, J. and Tan, X. (2009). Inference for multivariate normal mixtures. J. Multivariate Anal. 100 1367–1383.
• [8] Chen, J., Tan, X. and Zhang, R. (2008). Inference for normal mixtures in mean and variance. Statist. Sinica 18 443–465.
• [9] Chen, J. H. (1995). Optimal rate of convergence for finite mixture models. Ann. Statist. 23 221–233.
• [10] DasGupta, A. (2008). Asymptotic Theory of Statistics and Probability. Springer, New York.
• [11] Day, N. E. (1969). Estimating the components of a mixture of normal distributions. Biometrika 56 463–474.
• [12] Fan, J. (1991). On the optimal rates of convergence for nonparametric deconvolution problems. Ann. Statist. 19 1257–1272.
• [13] Genovese, C. R. and Wasserman, L. (2000). Rates of convergence for the Gaussian mixture sieve. Ann. Statist. 28 1105–1127.
• [14] Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. 29 1233–1263.
• [15] Hathaway, R. J. (1985). A constrained formulation of maximum-likelihood estimation for normal mixture distributions. Ann. Statist. 13 795–800.
• [16] Ho, N. and Nguyen, X. (2016). Supplement to “Convergence rates of parameter estimation for some weakly identifiable finite mixtures.” DOI:10.1214/16-AOS1444SUPP.
• [17] Ho, N. and Nguyen, X. (2016). On strong identifiability and optimal rates of parameter estimation in finite mixtures. Electron. J. Stat. 10 271–307.
• [18] Kasahara, H. and Shimotsu, K. (2015). Testing the number of components in normal mixture regression models. J. Amer. Statist. Assoc. 110 1632–1645.
• [19] Kruijer, W., Rousseau, J. and van der Vaart, A. (2010). Adaptive Bayesian density estimation with location-scale mixtures. Electron. J. Stat. 4 1225–1257.
• [20] Lindsay, B. (1995). Mixture Models: Theory, Geometry and Applications. IMS, Hayward, CA.
• [21] Liu, X. and Shao, Y. (2003). Asymptotics for likelihood ratio tests under loss of identifiability. Ann. Statist. 31 807–832.
• [22] McLachlan, G. J. and Basford, K. E. (1988). Mixture Models: Inference and Applications to Clustering. Statistics: Textbooks and Monographs 84. Dekker, New York.
• [23] Nguyen, X. (2013). Convergence of latent mixing measures in finite and infinite mixture models. Ann. Statist. 41 370–400.
• [24] Nguyen, X. (2016). Borrowing strength in hierarchical Bayes: Convergence of the Dirichlet base measure. Bernoulli 22 1535–1571.
• [25] Rousseau, J. (2010). Rates of convergence for the posterior distributions of mixtures of betas and adaptive nonparametric estimation of the density. Ann. Statist. 38 146–180.
• [26] Rousseau, J. and Mengersen, K. (2011). Asymptotic behaviour of the posterior distribution in overfitted mixture models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 73 689–710.
• [27] Sturmfels, B. (2002). Solving Systems of Polynomial Equations. CBMS Regional Conference Series in Mathematics 97. Amer. Math. Soc., Providence, RI.
• [28] van de Geer, S. (2000). Empirical Processes in M-Estimation. Cambridge Univ. Press, Cambridge.
• [29] Villani, C. (2009). Optimal Transport: Old and New. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 338. Springer, Berlin.
• [30] Yu, B. (1997). Assouad, Fano, and Le Cam. In Festschrift for Lucien Le Cam 423–435. Springer, New York.
• [31] Zhang, C.-H. (1990). Fourier methods for estimating mixing densities and distributions. Ann. Statist. 18 806–831.

#### Supplemental materials

• Supplement to “Convergence rates of parameter estimation for some weakly identifiable finite mixtures”. In this supplemental material, we present the proofs of several remaining technical lemmas.