Bayesian Analysis

Post-Processing Posteriors Over Precision Matrices to Produce Sparse Graph Estimates

Amir Bashir, Carlos M. Carvalho, P. Richard Hahn, and M. Beatrix Jones

Advance publication

This article is in its final form and can be cited using the date of online publication and the DOI.

Full-text: Open access

Abstract

A variety of computationally efficient Bayesian models for the covariance matrix of a multivariate Gaussian distribution are available. However, all produce a relatively dense estimate of the precision matrix, and are therefore unsatisfactory when one wishes to use the precision matrix to consider the conditional independence structure of the data. This paper considers the posterior predictive distribution of model fit for these covariance models. We then undertake post-processing of the Bayes point estimate for the precision matrix to produce a sparse model whose expected fit lies within the upper 95% of the posterior predictive distribution of fit. The impact of the method for selecting the zero elements of the precision matrix is evaluated. Good results were obtained using models that encouraged a sparse posterior (G-Wishart, Bayesian adaptive graphical lasso) and selection using credible intervals. We also find that this approach is easily extended to the problem of finding a sparse set of elements that differ across a set of precision matrices, a natural summary when a common set of variables is observed under multiple conditions. We illustrate our findings with moderate dimensional data examples from finance and metabolomics.

Article information

Source
Bayesian Anal., Advance publication (2018), 16 pages.

Dates
First available in Project Euclid: 20 December 2018

Permanent link to this document
https://projecteuclid.org/euclid.ba/1545296446

Digital Object Identifier
doi:10.1214/18-BA1139

Keywords
covariance selection decoupling shrinkage and selection Gaussian graphical models posterior summary shrinkage prior

Rights
Creative Commons Attribution 4.0 International License.

Citation

Bashir, Amir; Carvalho, Carlos M.; Hahn, P. Richard; Jones, M. Beatrix. Post-Processing Posteriors Over Precision Matrices to Produce Sparse Graph Estimates. Bayesian Anal., advance publication, 20 December 2018. doi:10.1214/18-BA1139. https://projecteuclid.org/euclid.ba/1545296446


Export citation

References

  • Cai, T. T., Li, H., Liu, W., and Xie, J. (2016). “Joint estimation of multiple high dimensional precision matrices.” Statistica Sinica, 26(2): 445–464.
  • Danaher, P., Wang, P., and Witten, D. M. (2013). “The joint graphical lasso for inverse covariance estimation across multiple classes.” Journal of the Royal Statistical Society Series B–Statistical Methodology, 76(2): 373–397.
  • Dempster, A. P. (1973). “Covariance selection.” Biometrics, 21(1): 157–175.
  • Fan, J., Feng, Y., and Wu, Y. (2009). “Network exploration via the adaptive LASSO and SCAD penalties.” Annals of Applied Statistics, 3(2): 521–541.
  • Fitch, A. M., Jones, M. B., and Massam, H. (2014). “The performance of covariance selection methods that consider decomposable models only.” Bayesian Analysis, 9(3): 659–684.
  • Friedman, J., Hastie, T., and Tibshirani, R. (2008). “Sparse inverse covariance estimation with the graphical lasso.” Biostatistics, 9: 432–441.
  • Green, P. and Thomas, A. (2013). “Sampling decomposable graphs using a Markov chain on junction trees.” Biometrika, 100(1): 91–110.
  • Guo, J., Levina, E., Michailidis, G., and Zhu, J. (2011). “Joint estimation of multiple graphical models.” Biometrika, 98(1): 1–15.
  • Hahn, P. R. and Carvalho, C. M. (2015). “Decoupling shrinkage and selection in Bayesian linear models: a posterior summary perspective.” Journal of the American Statistical Association, 110: 435–448.
  • Huang, A. and Wand, M. P. (2013). “Simple marginally noninformative prior distributions for covariance matrices.” Bayesian Analysis, 8(2): 439–452.
  • Jayan, S. (2016). “Analysis of the metabolic profiles from faeces and plasma of children born very preterm.” Master’s thesis, University of Auckland.
  • Kundu, S., Mallick, B. K., and Baladandayuthapani, V. (2018). “Efficient Bayesian regularization for graphical model selection.” Bayesian Analysis. Advance publication.
  • Ledoit, O. and Wolf, M. (2004). “A well-conditioned estimator for large-dimensional covariance matrices.” Journal of Multivariate Analysis, 88(2): 365–411.
  • Letac, G. and Massam, H. (2007). “Wishart distributions for decomposable graphs.” Annals of Statistics, 35(3): 1278–1323.
  • Liu, H., Lafferty, J., and Wasserman, L. (2009). “The nonparanormal: semiparametric estimation of high dimensional undirected graphs.” Journal of Machine Learning Research, 10: 2295–2328.
  • Mitra, R., Müller, P., and Ji, Y. (2016). “Bayesian graphical models for differential pathways.” Bayesian Analysis, 11(1): 99–124.
  • Mohammadi, A. and Wit, E. (2015). “Bayesian structure learning in sparse Gaussian graphical models.” Bayesian Analysis, 10(1): 109–138.
  • Murray, J. (2016). “bfa: Bayesian factor analysis.” R package version 0.4.
  • Peterson, C. B., Stingo, F. C., and Vannucci, M. (2015). “Bayesian inference of multiple Gaussian graphical models.” Journal of the American Statistical Association, 110(509): 159–174.
  • Peterson, C. B., Vannucci, M., Karakas, C., Choi, W., Ma, L., and Maletic-Savatic, M. (2013). “Inferring metabolic networks using the Bayesian adaptive graphical lasso with informative priors.” Statistics and its Interface, 6(4): 437–558.
  • Roverato, A. (2002). “Hyper inverse Wishart distribution for non-decomposable graphs and its application to Bayesian inference for Gaussian graphical models.” Scandinavian Journal of Statistics, 29(3): 391–411.
  • Scott, J. and Carvalho, C. (2008). “Feature inclusion stochastic search for Gaussian graphical models.” Journal of Computational and Graphical Statistics, 17(4): 790–808.
  • Tian, D., Gu, Q., and Ma, J. (2016). “Identifying gene regulatory network rewiring using latent differential graphical models.” Nucleic Acids Research, 44(17): e140.
  • Wang, H. (2012). “Bayesian graphical lasso models and efficient posterior computation.” Bayesian Analysis, 7: 867–886.
  • West, M. (2003). “Bayesian factor regression models in the large p, small n paradigm.” In Bernardo, J., Bayarri, M., Berger, J., Dawid, A., Heckerman, D., Smith, A. F. M., and West, M. (eds.), Bayesian Statistics., volume 7. Oxford: Oxford University Press.
  • Zhao, S. D., Cai, T. T., and Li, H. (2014). “Direct estimation of differential networks.” Biometrika, 101(2): 253–268.