Bayesian Analysis

The Performance of Covariance Selection Methods That Consider Decomposable Models Only

A. Marie Fitch, M. Beatrix Jones, and Hélène Massam

Full-text: Open access

Abstract

We consider the behavior of Bayesian procedures that perform model selection for decomposable Gaussian graphical models when the true model is in fact non-decomposable. We examine the asymptotic behavior of the posterior when models are misspecified in this way, and find that the posterior will converge to graphical structures that are minimal triangulations of the true structure. The marginal log likelihood ratio comparing different minimal triangulations is stochastically bounded, and appears to remain data dependent regardless of the sample size. The covariance matrices corresponding to the different minimal triangulations are essentially equivalent, so model averaging is of minimal benefit. Using simulated data sets and a particular high performing Bayesian method for fitting decomposable models, feature inclusion stochastic search, we illustrate that these predictions are borne out in practice. Finally, a comparison is made to penalized likelihood methods for graphical models, which make no decomposability restriction. Despite its inability to fit the true model, feature inclusion stochastic search produces models that are competitive or superior to the penalized likelihood methods, especially at higher dimensions.

Article information

Source
Bayesian Anal., Volume 9, Number 3 (2014), 659-684.

Dates
First available in Project Euclid: 5 September 2014

Permanent link to this document
https://projecteuclid.org/euclid.ba/1409921109

Digital Object Identifier
doi:10.1214/14-BA874

Mathematical Reviews number (MathSciNet)
MR3256059

Zentralblatt MATH identifier
1327.62389

Keywords
undirected Gaussian graphical models covariance selection feature inclusion stochastic search decomposable non-decomposable graphical lasso asymptotic behavior

Citation

Fitch, A. Marie; Jones, M. Beatrix; Massam, Hélène. The Performance of Covariance Selection Methods That Consider Decomposable Models Only. Bayesian Anal. 9 (2014), no. 3, 659--684. doi:10.1214/14-BA874. https://projecteuclid.org/euclid.ba/1409921109


Export citation

References

  • Armstrong, H., Carter, C. K., Wang, K. F. K., and Kohn, R. (2009). “Bayesian covariance matrix estimation using a mixture of decomposable graphical models.” Statistical Computing, 19: 303–316.
  • Carvalho, C. M. and Scott, J. G. (2009). “Objective Bayesian model selection in Gaussian graphical models.” Biometrika. URL http://biomet.oxfordjournals.org/content/early/2009/05/04/biomet.asp017.abstract
  • Dellaportas, P., Giudici, P., and Roberts, G. (2003). “Bayesian inference for nondecomposable graphical Gaussian models.” Sankhyā, 65: 43–55.
  • Dempster, A. (1972). “Covariance Selection.” Biometrics, 28: 157–175.
  • Dobra, A., Lenkoski, A., and Rodriguez, A. (2011). “Bayesian inference for general Gaussian graphical models with application to multivariate lattice data.” Journal of the American Statistical Association, 106: 1418–1433.
  • Fan, J., Feng, Y., and Wu, Y. (2009). “Network exploration via the adaptive LASSO and SCAD penalties.” The Annals of Applied Statistics, 3(2): 521–541.
  • Friedman, J., Hastie, T., and Tibshirani, R. (2008a). glasso: Graphical lasso- estimation of Gaussian graphical models. R package version 1.2. URL http://www-stat.stanford.edu/~tibs/glasso
  • (2008b). “Sparse inverse covariance estimation with the graphical lasso.” Biostatistics, 9(3): 432–441. URL http://biostatistics.oxfordjournals.org/cgi/content/abstract/kxm045v1
  • Jones, B., Carvalho, C., Dobra, A., Hans, C., Carter, C., and West, M. (2005). “Experiments in Stochastic Computation for High-Dimensional Graphical Models.” Statistical Science, 20(4): 388–400.
  • Lauritzen, S. L. (1996). Graphical Models. Oxford: Clarendon Press.
  • Meinhausen, N. and Bühlmann, P. (2006). “High-dimensional Graphs and Variable Selection with the Lasso.” The Annals of Statistics, 34(3): 1436–1462.
  • Moghaddam, B., Marlin, B. M., Khan, M. E., and Murphy, K. P. (2009). “Accelerating Bayesian Structural Inference for Non-Decomposable Gaussian Graphical Models.” Proceedings of the 23rd Neural Information Processing Systems Conference, 1285–1293.
  • R Development Core Team (2009). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. URL http://www.R-project.org
  • Roverato, A. (2002). “Hyper inverse Wishart distribution for non-decomposable graphs and its application to Bayesian inference for Gaussian Graphical Models.” Scandinavian Journal of Statistics, 29: 391–411.
  • Scott, J. G. and Carvalho, C. M. (2008). “Feature-Inclusion Stochastic Search for Gaussian Graphical Models.” Journal of Computational and Graphical Statistics, 17(4): 790–808.
  • Whittaker, J. (2008). Graphical Models in Applied Multivariate Statistics. UK: John Wiley and Sons.
  • Wong, F., Carter, C. K., and Kohn, R. (2003). “Efficient estimation of covariance selection models.” Biometrika, 90(4): 809–830.