Bayesian Analysis

Robust Bayesian Graphical Modeling Using Dirichlet t-Distributions

Michael Finegold and Mathias Drton

Full-text: Open access

Abstract

Bayesian graphical modeling provides an appealing way to obtain uncertainty estimates when inferring network structures, and much recent progress has been made for Gaussian models. For more robust inferences, it is natural to consider extensions to t-distribution models. We argue that the classical multivariate t-distribution, defined using a single latent Gamma random variable to rescale a Gaussian random vector, is of little use in more highly multivariate settings, and propose other, more flexible t-distributions. Using an independent Gamma-divisor for each component of the random vector defines what we term the alternative t-distribution. The associated model allows one to extract information from highly multivariate data even when most experiments contain outliers for some of their measurements. However, the use of this alternative model comes at increased computational cost and imposes constraints on the achievable correlation structures, raising the need for a compromise between the classical and alternative models. To this end we propose the use of Dirichlet processes for adaptive clustering of the latent Gamma-scalars, each of which may then divide a group of latent Gaussian variables. The resulting Dirichlet t-distribution interpolates naturally between the two extreme cases of the classical and alternative t-distributions and combines more appealing modeling of the multivariate dependence structure with favorable computational properties.

This paper was invited by the Editor-in-Chief of Bayesian Analysis to be presented as the 2014 Best Bayesian Analysis Paper at the Twelfth World Meeting of the International Society for Bayesian Analysis (ISBA2014), held in Cancun, Mexico, on July 14–18, 2014, with invited discussions by Babak Shahbaba and François Caron.

Article information

Source
Bayesian Anal., Volume 9, Number 3 (2014), 521-550.

Dates
First available in Project Euclid: 5 September 2014

Permanent link to this document
https://projecteuclid.org/euclid.ba/1409921102

Digital Object Identifier
doi:10.1214/13-BA856

Mathematical Reviews number (MathSciNet)
MR3256052

Zentralblatt MATH identifier
1327.62144

Keywords
Bayesian inference Dirichlet process graphical model Markov chain Monte Carlo t-distribution

Citation

Finegold, Michael; Drton, Mathias. Robust Bayesian Graphical Modeling Using Dirichlet $t$ -Distributions. Bayesian Anal. 9 (2014), no. 3, 521--550. doi:10.1214/13-BA856. https://projecteuclid.org/euclid.ba/1409921102


Export citation

References

  • Antoniak, C. E. (1974). “Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems.” Annals of Statistics , 2: 1152–1174.
  • Armstrong, H., Carter, C. K., Wong, K. F., and Kohn, R. (2009). “Bayesian covariance matrix estimation using a mixture of decomposable graphical models.” Statistics and Computing , 19(3): 303–316.
  • Atay-Kayis, A. and Massam, H. (2005). “A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models.” Biometrika , 92(2): 317–335.
  • Carvalho, C. and Scott, J. (2009). “Objective Bayesian model selection in Gaussian graphical models.” Biometrika , 96(3): 1–16.
  • Carvalho, C. M., Massam, H., and West, M. (2007). “Simulation of hyper-inverse Wishart distributions in graphical models.” Biometrika , 94(3): 647–659.
  • Cowell, R. G., Dawid, A. P., Lauritzen, S. L., and Spiegelhalter, D. J. (1999). Probabilistic networks and expert systems . Statistics for Engineering and Information Science. New York: Springer-Verlag.
  • Dawid, A. P. and Lauritzen, S. L. (1993). “Hyper-Markov laws in the statistical analysis of decomposable graphical models.” Annals of Statistics , 21(3): 1272–1317.
  • De Finetti, B. (1961). “The Bayesian approach to the rejection of outliers.” In Proceedings of the Fourth Berkeley Symposium on Probability and Statistics , volume 1, 199–210.
  • Dellaportas, P., Giudici, P., and Roberts, G. (2003). “Bayesian inference for nondecomposable graphical Gaussian models.” Sankhyā , 65(1): 43–55.
  • Dobra, A., Hans, C., Jones, B., Nevins, J. R., Yao, G., and West, M. (2004). “Sparse graphical models for exploring gene expression data.” Journal of Multivariate Analysis , 90(1): 196–212.
  • Donnet, S. and Marin, J.-M. (2012). “An empirical Bayes procedure for the selection of Gaussian graphical models.” Statistics and Computing , 22(5): 1113–1123.
  • Escobar, M. D. and West, M. (1995). “Bayesian density estimation and inference using mixtures.” Journal of the American Statistical Association , 90(430): 577–588.
  • Ferguson, T. S. (1973). “A Bayesian analysis of some nonparametric problems.” Annals of Statistics , 1: 209–230.
  • Finegold, M. and Drton, M. (2011). “Robust graphical modeling with classical and alternative $t$-distributions.” Annals of Applied Statistics , 5(2A): 1057–1080.
  • Gasch, A. P., Spellman, P. T., Kao, C. M., Carmel-Harel, O., Eisen, M. B., Storz, G., Botstein, D., and Brown, P. O. (2000). “Genomic expression programs in the response of yeast cells to environmental changes.” Molecular Biology of the Cell , 11: 4241–4257.
  • Giudici, P. and Green, P. J. (1999). “Decomposable graphical Gaussian model determination.” Biometrika , 86(4): 785–801.
  • Jones, B., Carvalho, C., Dobra, A., Hans, C., Carter, C., and West, M. (2005). “Experiments in stochastic computation for high-dimensional graphical models.” Statistical Science , 20(4): 388–400.
  • Kotz, S. and Nadarajah, S. (2004). Multivariate $t$ distributions and their applications . Cambridge: Cambridge University Press.
  • Lauritzen, S. L. (1996). Graphical models , volume 17 of Oxford Statistical Science Series . New York: The Clarendon Press Oxford University Press. Oxford Science Publications.
  • Lenkoski, A. and Dobra, A. (2011). “Computational aspects related to inference in Gaussian graphical models with the G-Wishart prior.” Journal of Computational and Graphical Statistics , 20(1): 140–157.
  • Liu, C. and Rubin, D. B. (1995). “ML estimation of the $t$ distribution using EM and its extensions, ECM and ECME.” Statistica Sinica , 5(1): 19–39.
  • Liu, J. S. (2001). Monte Carlo strategies in scientific computing . Springer Series in Statistics. New York: Springer-Verlag.
  • Liu, Y., Wichura, M. J., and Drton, M. (2013). “Rejection sampling for an extended Gamma distribution.” Unpublished manuscript.
  • R Development Core Team (2010). R: A Language and Environment for Statistical Computing . R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0.
  • Rajaratnam, B., Massam, H., and Carvalho, C. M. (2008). “Flexible covariance estimation in graphical Gaussian models.” Annals of Statistics , 36(6): 2818–2849.
  • Roverato, A. (2002). “Hyper inverse Wishart distribution for non-decomposable graphs and its application to Bayesian inference for Gaussian graphical models.” Scandinavian Journal of Statistics , 29(3): 391–411.
  • Sethuraman, J. (1994). “A constructive definition of Dirichlet priors.” Statistica Sinica , 4(2): 639–650.
  • Teh, Y. W., Jordan, M. I., Beal, M. J., and Blei, D. M. (2006). “Hierarchical Dirichlet processes.” Journal of the American Statistical Association , 101(476): 1566–1581.
  • West, M. (1984). “Outlier models and prior distributions in Bayesian linear regression.” Journal of the Royal Statistical Society, Series B , 46(3): 431–439.
  • Yuan, M. and Huang, J. (2009). “Regularized parameter estimation of high dimensional $t$ distribution.” Journal of Statistical Planning and Inference , 139: 2284–2292.

See also

  • Related item: François Caron, Luke Bornn. Comment on Article by Finegold and Drton. Bayesian Anal., Vol. 9, Iss. 3 (2014) 551–556.
  • Related item: Babak Shahbaba. Comment on Article by Finegold and Drton. Bayesian Anal., Vol. 9, Iss. 3 (2014) 557–560.
  • Related item: Contributed Discussion on Article by Finegold and Drton. Bayesian Anal., Vol. 9, Iss. 3 (2014) 561–590.
  • Related item: Michael Finegold, Mathias Drton. Rejoinder. Bayesian Anal., Vol. 9, Iss. 3 (2014) 591–596.