• Bernoulli
  • Volume 22, Number 1 (2016), 193-212.

Nonparametric finite translation hidden Markov models and extensions

Elisabeth Gassiat and Judith Rousseau

Full-text: Open access


In this paper, we consider nonparametric finite translation hidden Markov models, or more generally finite translation mixtures with dependent latent variables. We prove that all the parameters of the model are identifiable as soon as the matrix that defines the joint distribution of two consecutive latent variables is non-singular and the translation parameters are distinct. Under this assumption, we provide a consistent estimator of the number of populations, of the translation parameters and of the distribution of two consecutive latent variables, which we prove to be asymptotically normally distributed under mild dependency assumptions. We propose a nonparametric estimator of the unknown translated density. In case the latent variables form a Markov chain, we prove that this estimator is minimax adaptive over regularity classes of densities.

Article information

Bernoulli, Volume 22, Number 1 (2016), 193-212.

Received: June 2013
Revised: December 2013
First available in Project Euclid: 30 September 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

dependent latent variable models hidden Markov models nonparametric estimation translation mixtures


Gassiat, Elisabeth; Rousseau, Judith. Nonparametric finite translation hidden Markov models and extensions. Bernoulli 22 (2016), no. 1, 193--212. doi:10.3150/14-BEJ631.

Export citation


  • [1] Allman, E.S., Matias, C. and Rhodes, J.A. (2009). Identifiability of parameters in latent structure models with many observed variables. Ann. Statist. 37 3099–3132.
  • [2] Arlot, S. and Massart, P. (2009). Data-driven calibration of penalties for least-squares regression. J. Mach. Learn. Res. 10 245–279.
  • [3] Azzaline, A. and Bowman, A.W. (1990). A look at some data in the Old Faithful geyser. Appl. Statist. 39 357–365.
  • [4] Birgé, L. and Massart, P. (2007). Minimal penalties for Gaussian model selection. Probab. Theory Related Fields 138 33–73.
  • [5] Bonhomme, S., Jochman, K. and Robin, J. (2011). Nonparametric estimation of finite mixtures. Technical report.
  • [6] Bontemps, D. and Toussile, W. (2013). Clustering and variable selection for categorical multivariate data. Electron. J. Stat. 7 2344–2371.
  • [7] Bordes, L., Mottelet, S. and Vandekerkhove, P. (2006). Semiparametric estimation of a two-component mixture model. Ann. Statist. 34 1204–1232.
  • [8] Butucea, C. and Vandekerkhove, P. (2014). Semiparametric mixtures of symmetric distributions. Scand. J. Stat. 41 227–239.
  • [9] Cappé, O., Moulines, E. and Rydén, T. (2005). Inference in Hidden Markov Models. New York: Springer.
  • [10] Clemencon, S., Garivier, A. and Tressou, J. (2009). Pseudo-regenerative block-bootstrap for hidden Markov chains. In Statistical Signal Processing, 2009. IEEE.
  • [11] Doukhan, P., Massart, P. and Rio, E. (1994). The functional central limit theorem for strongly mixing processes. Ann. Inst. Henri Poincaré Probab. Stat. 30 63–82.
  • [12] Doukhan, P., Massart, P. and Rio, E. (1995). Invariance principles for absolutely regular empirical processes. Ann. Inst. Henri Poincaré Probab. Stat. 31 393–427.
  • [13] Gassiat, E., Cleynen, A. and Robin, S. (2013). Finite state space nonparametric hidden Markov models are in general identifiable. Available at arXiv:1306.4657.
  • [14] Gassiat, E. and Rousseau, J. (2015). Supplement to “Nonparametric finite translation hidden Markov models and extensions”. DOI:10.3150/14-BEJ631SUPP.
  • [15] Hall, P. and Zhou, X.-H. (2003). Nonparametric estimation of component distributions in a multivariate mixture. Ann. Statist. 31 201–224.
  • [16] Henry, M., Kitamura, Y. and Salanié, B. (2014). Partial identification of finite mixtures in econometric models. Quant. Econ. 5 123–144.
  • [17] Hunter, D.R., Wang, S. and Hettmansperger, T.P. (2007). Inference for mixtures of symmetric distributions. Ann. Statist. 35 224–251.
  • [18] Kasahara, H. and Shimotsu, K. (2009). Nonparametric identification of finite mixture models of dynamic discrete choices. Econometrica 77 135–175.
  • [19] Lambert, M.F., Whiting, J.P. and Metcalfe, A.V. (2003). A non-parametric hidden Markov model for climate state identification. Hydrol. Earth Syst. Sci. 7 652–667.
  • [20] Marin, J.-M., Mengersen, K. and Robert, C.P. (2005). Bayesian modelling and inference on mixtures of distributions. In Bayesian Thinking: Modeling and Computation (C. Rao and D. Dey, eds.). Handbook of Statist. 25 459–507. Amsterdam: Elsevier/North-Holland.
  • [21] Massart, P. (2007). Concentration Inequalities and Model Selection: Ecole d’Eté de Probabilités de Saint-Flour XXXIII – 2003. Berlin: Springer.
  • [22] Maugis, C. and Michel, B. (2011). Data-driven penalty calibration: A case study for Gaussian mixture model selection. ESAIM Probab. Stat. 15 320–339.
  • [23] Maugis-Rabusseau, C. and Michel, B. (2013). Adaptive density estimation for clustering with Gaussian mixtures. ESAIM Probab. Stat. 17 698–724.
  • [24] McLachlan, G. and Peel, D. (2000). Finite Mixture Models. New York: Wiley.
  • [25] Moreno, C.J. (1973). The zeros of exponential polynomials. I. Compos. Math. 26 69–78.
  • [26] Rio, E. (2000). Inégalités de Hoeffding pour les fonctions lipschitziennes de suites dépendantes. C. R. Acad. Sci. Paris Sér. I Math. 330 905–908.
  • [27] Stein, E.M. and Shakarchi, R. (2003). Complex Analysis. Princeton, NJ: Princeton Univ. Press.
  • [28] Verzelen, N. (2009). Adaptative estimation to regular Gaussian Markov random fields. Ph.D. thesis, Univ. Paris-Sud.
  • [29] Villers, F. (2007). Tests et sélection de modèles pour l’analyse de donnŕes protéomiques et transcriptomiques. Ph.D. thesis, Univ. Paris-Sud.
  • [30] Yau, C., Papaspiliopoulos, O., Roberts, G.O. and Holmes, C. (2011). Bayesian non-parametric hidden Markov models with applications in genomics. J. R. Stat. Soc. Ser. B Stat. Methodol. 73 37–57.

Supplemental materials

  • Supplement to “Nonparametric finite translation hidden Markov models and extensions”. In the supplementary material, we provide an oracle inequality which is used to prove Theorem 4, together with the proofs of the oracle inequality and of Theorem 4. We also give a concentration inequality which is used in various parts of these proofs.