Electronic Journal of Statistics

The Polya-Gamma Gibbs sampler for Bayesian logistic regression is uniformly ergodic

Hee Min Choi and James P. Hobert

Full-text: Open access

Abstract

One of the most widely used data augmentation algorithms is Albert and Chib’s (1993) algorithm for Bayesian probit regression. Polson, Scott, and Windle (2013) recently introduced an analogous algorithm for Bayesian logistic regression. The main difference between the two is that Albert and Chib’s (1993) truncated normals are replaced by so-called Polya-Gamma random variables. In this note, we establish that the Markov chain underlying Polson, Scott, and Windle’s (2013) algorithm is uniformly ergodic. This theoretical result has important practical benefits. In particular, it guarantees the existence of central limit theorems that can be used to make an informed decision about how long the simulation should be run.

Article information

Source
Electron. J. Statist., Volume 7 (2013), 2054-2064.

Dates
First available in Project Euclid: 20 August 2013

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1377005819

Digital Object Identifier
doi:10.1214/13-EJS837

Mathematical Reviews number (MathSciNet)
MR3091616

Zentralblatt MATH identifier
1349.60123

Subjects
Primary: 60J27: Continuous-time Markov processes on discrete state spaces
Secondary: 62F15: Bayesian inference

Keywords
Polya-Gamma distribution data augmentation algorithm minorization condition Markov chain Monte Carlo

Citation

Choi, Hee Min; Hobert, James P. The Polya-Gamma Gibbs sampler for Bayesian logistic regression is uniformly ergodic. Electron. J. Statist. 7 (2013), 2054--2064. doi:10.1214/13-EJS837. https://projecteuclid.org/euclid.ejs/1377005819


Export citation

References

  • Albert, J. H. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data., Journal of the American Statistical Association 88 669–679.
  • Polson, N.G., Scott, J.G. and Windle, J. (2013). Bayesian inference for logistic models using Polya-Gamma latent variables., Journal of the American Statistical Association (to appear). arXiv:1205.0310v3.
  • Roy, V. and Hobert, J.P. (2007). Convergence rates and asymptotic standard errors for Markov chain Monte Carlo algorithms for Bayesian probit regression., Journal of the Royal Statistical Society, Series B 69 607–623.
  • Holmes, C. and Held, L. (2006). Bayesian auxiliary variable models for binary and multinomial regression., Bayesian Analysis 1 145–168.
  • Frühwirth-Schnatter, S. and Frühwirth, R. (2010). Data augmentation and MCMC for binary and multinomial logit models. In, Statistical Modelling and Regression Structures, 111–132. Springer-Verlag.
  • Marchev, D. (2011). Markov chain Monte Carlo algorithms for the Bayesian logistic regression model. In, Proceedings of the Annual International Conference on Operations Research and Statistics, 154–159.
  • Tierney, L. (1994). Markov chains for exploring posterior distributions (with discussion)., The Annals of Statistics 22 1701–1762.
  • Roberts, G.O. and Rosenthal, J.S. (2004). General state space Markov chains and MCMC algorithms., Probability Surveys 1 20–71.
  • Flegal, J.M., Haran, M. and Jones, G.L. (2008). Markov chain Monte Carlo: Can we trust the third significant figure?, Statistical Science 23 250–260.
  • Biane, P., Pitman, J. and Yor, M. (2001). Probability laws related to the Jacobi theta and Riemann zeta functions, and Brownian excursions., Bulletin of the American Mathematical Society 38 435–465.
  • Hobert, J.P. (2011). The data augmentation algorithm: Theory and methodology. In, Handbook of Markov Chain Monte Carlo ( S. Brooks, A. Gelman, G. Jones and X.L. Meng, eds.). Chapman & Hall/CRC Press.
  • Jones, G.L. and Hobert, J.P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo., Statistical Science 16 312–34.
  • Chen, M.-H. and Shao, Q.-M. (2000). Propriety of posterior distribution for dichotomous quantal response models., Proceedings of the American Mathematical Society 129 293–302.