• Bernoulli
  • Volume 6, Number 3 (2000), 457-489.

The stochastic EM algorithm: estimation and asymptotic results

Søren Feodor Nielsen

Full-text: Open access


The EM algorithm is a much used tool for maximum likelihood estimation in missing or incomplete data problems. However, calculating the conditional expectation required in the E-step of the algorithm may be infeasible, especially when this expectation is a large sum or a high-dimensional integral. Instead the expectation can be estimated by simulation. This is the common idea in the stochastic EM algorithm and the Monte Carlo EM algorithm.

Article information

Bernoulli, Volume 6, Number 3 (2000), 457-489.

First available in Project Euclid: 10 April 2004

Permanent link to this document

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

EM algorithm incomplete observations simulation


Feodor Nielsen, Søren. The stochastic EM algorithm: estimation and asymptotic results. Bernoulli 6 (2000), no. 3, 457--489.

Export citation


  • [1] Bickel, P.J., Klaassen, C.A.J., Ritov, Y. and Wellner, J.A. (1993) Efficient and Adaptive Estimation for Semiparametric Models. Baltimore, MD: Johns Hopkins University Press.
  • [2] Billingsley, P. (1968) Convergence of Probability Measures. New York: Wiley.
  • [3] Brooks, S.P. and Roberts, G.O. (1998) Convergence assessment techniques for Markov Chain Monte Carlo. Statist. Comput., 8, 319-335.
  • [4] Celeux, G. and Diebolt, J. (1986) The SEM algorithm: A probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Comput. Statist. Quart., 2, 73-82.
  • [5] Celeux, G. and Diebolt, J. (1993) Asymptotic properties of a stochastic EM algorithm for estimating mixing proportions. Comm. Statist. Stochastic Models, 9, 599-613.
  • [6] Celeux, G., Chauveau, D. and Diebolt, J. (1996) Stochastic versions of the EM algorithm: An experimental study in the mixture case. J. Statist. Comput. Simulation, 55, 287-314.
  • [7] Chan, K.S. and Ledolter, J. (1995) Monte Carlo EM estimation for time series models involving counts. J. Amer. Statist. Assoc., 90, 242-252.
  • [8] Dempster A.P., Laird N.M. and Rubin D.B. (1977) Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). J. Roy. Statist. Soc. Ser. B, 39, 1-38.
  • [9] Diebolt, J. and Ip, E.H.S. (1996) Stochastic EM: method and application. In W.R. Gilks, S. Richardson, D.J. Speigelhalter (eds), Markov Chain Monte Carlo in Practice. London: Chapman & Hall.
  • [10] Ethier, S.N. and Kurtz T.G. (1986) Markov Processes. Characterization and Convergence. New York: Wiley.
  • [11] Gelman, A. and Rubin, D.B. (1992) Inference from iterative simulation using multiple sequences (with discussion). Statist. Sci. 7, 457-511.
  • [12] Hajivassiliou, V.A. (1997) Some practical issues in maximum simulated likelihood.
  • [13] Ibragimov, I.A. and Has'minskii, R.Z. (1981) Statistical Estimation. Asymptotic Theory. New York: Springer-Verlag.
  • [14] Johansen, S. (1995) Likelihood-Based Inference in Cointegrated Vector Autoregression Models. Oxford: Oxford University Press.
  • [15] Lehmann, E.L. (1983) Theory of Point Estimation. New York: Wiley.
  • [16] Louis, T.A. (1982) Finding the observed information matrix when using the EM algorithm. J. R. Statist. Soc. Ser. B, 44, 226-233.
  • [17] Meng, X.-L. and Rubin, D.B. (1991) Using EM to obtain asymptotic variance-covariance matrices: The SEM algorithm. J. Amer. Statist. Assoc., 86, 899-909.
  • [18] Meyn, S.P. and Tweedie, R.L. (1993) Markov Chains and Stochastic Stability. New York: Springer- Verlag.
  • [19] Raftery, A.E. and Lewis, S.M. (1992) How many iterations in the Gibbs sampler?. In J.M. Bernardo, J.O. Berger, A.P. Dawid and A.F.M. Smith (eds), Bayesian Statistics, Vol. 4. Oxford: Oxford University Press.
  • [20] Roussas, G.G. (1972) Contiguity of Probability Measures. London: Cambridge University Press.
  • [21] Ruud, P.A. (1991) Extensions of estimation methods using the EM algorithm. J Econometrics, 49, 305-341.
  • [22] Schenker, N. and Welsh, A.H. (1987) Asymptotic results for multiple imputation. Ann. Statist., 16, 1550-1566.
  • [23] Schick, A. (1987) A note on the construction of asymptotically linear estimators. J. Statist. Plann. Inference, 16, 89-105.
  • [24] Tanner, M.A. and Wong, W.H. (1987) An application of imputation to an estimation problem in grouped lifetime analysis. Technometrics, 29, 23-32.
  • [25] Wei, G.C.G. and Tanner, M.A. (1990) A Monte Carlo implementation of the EM algorithm and the poor man's data augmentation algorithm. J. Amer. Statist. Assoc., 85, 699-704.