The Annals of Statistics

Convergence analysis of the Gibbs sampler for Bayesian general linear mixed models with improper priors

Jorge Carlos Román and James P. Hobert

Full-text: Open access


Bayesian analysis of data from the general linear mixed model is challenging because any nontrivial prior leads to an intractable posterior density. However, if a conditionally conjugate prior density is adopted, then there is a simple Gibbs sampler that can be employed to explore the posterior density. A popular default among the conditionally conjugate priors is an improper prior that takes a product form with a flat prior on the regression parameter, and so-called power priors on each of the variance components. In this paper, a convergence rate analysis of the corresponding Gibbs sampler is undertaken. The main result is a simple, easily-checked sufficient condition for geometric ergodicity of the Gibbs–Markov chain. This result is close to the best possible result in the sense that the sufficient condition is only slightly stronger than what is required to ensure posterior propriety. The theory developed in this paper is extremely important from a practical standpoint because it guarantees the existence of central limit theorems that allow for the computation of valid asymptotic standard errors for the estimates computed using the Gibbs sampler.

Article information

Ann. Statist., Volume 40, Number 6 (2012), 2823-2849.

First available in Project Euclid: 8 February 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60J27: Continuous-time Markov processes on discrete state spaces
Secondary: 62F15: Bayesian inference

Convergence rate geometric drift condition geometric ergodicity Markov chain Monte Carlo posterior propriety


Román, Jorge Carlos; Hobert, James P. Convergence analysis of the Gibbs sampler for Bayesian general linear mixed models with improper priors. Ann. Statist. 40 (2012), no. 6, 2823--2849. doi:10.1214/12-AOS1052.

Export citation


  • Bednorz, W. and Łatuszyński, K. (2007). A few remarks on “Fixed-width output analysis for Markov chain Monte Carlo” by Jones et al. [MR2279478]. J. Amer. Statist. Assoc. 102 1485–1486.
  • Daniels, M. J. (1999). A prior for the variance in hierarchical models. Canad. J. Statist. 27 567–578.
  • Diaconis, P., Khare, K. and Saloff-Coste, L. (2008). Gibbs sampling, exponential families and orthogonal polynomials (with discussion). Statist. Sci. 23 151–178.
  • Flegal, J. M., Haran, M. and Jones, G. L. (2008). Markov chain Monte Carlo: Can we trust the third significant figure? Statist. Sci. 23 250–260.
  • Flegal, J. M. and Jones, G. L. (2010). Batch means and spectral variance estimators in Markov chain Monte Carlo. Ann. Statist. 38 1034–1070.
  • Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Anal. 1 515–534.
  • Johnson, A. A. and Jones, G. L. (2010). Gibbs sampling for a Bayesian hierarchical general linear model. Electron. J. Stat. 4 313–333.
  • Jones, G. L., Haran, M., Caffo, B. S. and Neath, R. (2006). Fixed-width output analysis for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 101 1537–1547.
  • Khare, K. and Hobert, J. P. (2011). A spectral analytic comparison of trace-class data augmentation algorithms and their sandwich variants. Ann. Statist. 39 2585–2606.
  • Liu, J. S., Wong, W. H. and Kong, A. (1994). Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika 81 27–40.
  • Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.
  • Papaspiliopoulos, O. and Roberts, G. (2008). Stability of the Gibbs sampler for Bayesian hierarchical models. Ann. Statist. 36 95–117.
  • Roberts, G. O. and Rosenthal, J. S. (1998). Markov-chain Monte Carlo: Some practical implications of theoretical results (with discussion). Canad. J. Statist. 26 5–31.
  • Roberts, G. O. and Rosenthal, J. S. (2001). Markov chains and de-initializing processes. Scand. J. Stat. 28 489–504.
  • Roberts, G. O. and Rosenthal, J. S. (2004). General state space Markov chains and MCMC algorithms. Probab. Surv. 1 20–71.
  • Román, J. C. (2012). Convergence analysis of block Gibbs samplers for Bayesian general linear mixed models. Ph.D. thesis, Dept. Statistics, Univ. Florida, Gainesville, FL.
  • Searle, S. R., Casella, G. and McCulloch, C. E. (1992). Variance Components. Wiley, New York.
  • Sun, D., Tsutakawa, R. K. and He, Z. (2001). Propriety of posteriors with improper priors in hierarchical linear mixed models. Statist. Sinica 11 77–95.
  • Tan, A. and Hobert, J. P. (2009). Block Gibbs sampling for Bayesian random effects models with improper priors: Convergence and regeneration. J. Comput. Graph. Statist. 18 861–878.