The Annals of Statistics

Convergence analysis of the Gibbs sampler for Bayesian general linear mixed models with improper priors

Jorge Carlos Román and James P. Hobert

Full-text: Open access

Abstract

Bayesian analysis of data from the general linear mixed model is challenging because any nontrivial prior leads to an intractable posterior density. However, if a conditionally conjugate prior density is adopted, then there is a simple Gibbs sampler that can be employed to explore the posterior density. A popular default among the conditionally conjugate priors is an improper prior that takes a product form with a flat prior on the regression parameter, and so-called power priors on each of the variance components. In this paper, a convergence rate analysis of the corresponding Gibbs sampler is undertaken. The main result is a simple, easily-checked sufficient condition for geometric ergodicity of the Gibbs–Markov chain. This result is close to the best possible result in the sense that the sufficient condition is only slightly stronger than what is required to ensure posterior propriety. The theory developed in this paper is extremely important from a practical standpoint because it guarantees the existence of central limit theorems that allow for the computation of valid asymptotic standard errors for the estimates computed using the Gibbs sampler.

Article information

Source
Ann. Statist., Volume 40, Number 6 (2012), 2823-2849.

Dates
First available in Project Euclid: 8 February 2013

Permanent link to this document
https://projecteuclid.org/euclid.aos/1360332185

Digital Object Identifier
doi:10.1214/12-AOS1052

Mathematical Reviews number (MathSciNet)
MR3097961

Zentralblatt MATH identifier
1296.60204

Subjects
Primary: 60J27: Continuous-time Markov processes on discrete state spaces
Secondary: 62F15: Bayesian inference

Keywords
Convergence rate geometric drift condition geometric ergodicity Markov chain Monte Carlo posterior propriety

Citation

Román, Jorge Carlos; Hobert, James P. Convergence analysis of the Gibbs sampler for Bayesian general linear mixed models with improper priors. Ann. Statist. 40 (2012), no. 6, 2823--2849. doi:10.1214/12-AOS1052. https://projecteuclid.org/euclid.aos/1360332185


Export citation

References

  • Bednorz, W. and Łatuszyński, K. (2007). A few remarks on “Fixed-width output analysis for Markov chain Monte Carlo” by Jones et al. [MR2279478]. J. Amer. Statist. Assoc. 102 1485–1486.
  • Daniels, M. J. (1999). A prior for the variance in hierarchical models. Canad. J. Statist. 27 567–578.
  • Diaconis, P., Khare, K. and Saloff-Coste, L. (2008). Gibbs sampling, exponential families and orthogonal polynomials (with discussion). Statist. Sci. 23 151–178.
  • Flegal, J. M., Haran, M. and Jones, G. L. (2008). Markov chain Monte Carlo: Can we trust the third significant figure? Statist. Sci. 23 250–260.
  • Flegal, J. M. and Jones, G. L. (2010). Batch means and spectral variance estimators in Markov chain Monte Carlo. Ann. Statist. 38 1034–1070.
  • Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Anal. 1 515–534.
  • Johnson, A. A. and Jones, G. L. (2010). Gibbs sampling for a Bayesian hierarchical general linear model. Electron. J. Stat. 4 313–333.
  • Jones, G. L., Haran, M., Caffo, B. S. and Neath, R. (2006). Fixed-width output analysis for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 101 1537–1547.
  • Khare, K. and Hobert, J. P. (2011). A spectral analytic comparison of trace-class data augmentation algorithms and their sandwich variants. Ann. Statist. 39 2585–2606.
  • Liu, J. S., Wong, W. H. and Kong, A. (1994). Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika 81 27–40.
  • Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.
  • Papaspiliopoulos, O. and Roberts, G. (2008). Stability of the Gibbs sampler for Bayesian hierarchical models. Ann. Statist. 36 95–117.
  • Roberts, G. O. and Rosenthal, J. S. (1998). Markov-chain Monte Carlo: Some practical implications of theoretical results (with discussion). Canad. J. Statist. 26 5–31.
  • Roberts, G. O. and Rosenthal, J. S. (2001). Markov chains and de-initializing processes. Scand. J. Stat. 28 489–504.
  • Roberts, G. O. and Rosenthal, J. S. (2004). General state space Markov chains and MCMC algorithms. Probab. Surv. 1 20–71.
  • Román, J. C. (2012). Convergence analysis of block Gibbs samplers for Bayesian general linear mixed models. Ph.D. thesis, Dept. Statistics, Univ. Florida, Gainesville, FL.
  • Searle, S. R., Casella, G. and McCulloch, C. E. (1992). Variance Components. Wiley, New York.
  • Sun, D., Tsutakawa, R. K. and He, Z. (2001). Propriety of posteriors with improper priors in hierarchical linear mixed models. Statist. Sinica 11 77–95.
  • Tan, A. and Hobert, J. P. (2009). Block Gibbs sampling for Bayesian random effects models with improper priors: Convergence and regeneration. J. Comput. Graph. Statist. 18 861–878.