Bayesian Analysis

High-Dimensional Posterior Consistency for Hierarchical Non-Local Priors in Regression

Xuan Cao, Kshitij Khare, and Malay Ghosh

Advance publication

This article is in its final form and can be cited using the date of online publication and the DOI.

Full-text: Open access

Abstract

The choice of tuning parameters in Bayesian variable selection is a critical problem in modern statistics. In particular, for Bayesian linear regression with non-local priors, the scale parameter in the non-local prior density is an important tuning parameter which reflects the dispersion of the non-local prior density around zero, and implicitly determines the size of the regression coefficients that will be shrunk to zero. Current approaches treat the scale parameter as given, and suggest choices based on prior coverage/asymptotic considerations. In this paper, we consider the fully Bayesian approach introduced in (Wu, 2016) with the pMOM non-local prior and an appropriate Inverse-Gamma prior on the tuning parameter to analyze the underlying theoretical property. Under standard regularity assumptions, we establish strong model selection consistency in a high-dimensional setting, where p is allowed to increase at a polynomial rate with n or even at a sub-exponential rate with n. Through simulation studies, we demonstrate that our model selection procedure can outperform other Bayesian methods which treat the scale parameter as given, and commonly used penalized likelihood methods, in a range of simulation settings.

Article information

Source
Bayesian Anal., Advance publication (2018), 22 pages.

Dates
First available in Project Euclid: 19 April 2019

Permanent link to this document
https://projecteuclid.org/euclid.ba/1555660820

Digital Object Identifier
doi:10.1214/19-BA1154

Keywords
posterior consistency high-dimensional data non-local prior model selection multivariate regression

Rights
Creative Commons Attribution 4.0 International License.

Citation

Cao, Xuan; Khare, Kshitij; Ghosh, Malay. High-Dimensional Posterior Consistency for Hierarchical Non-Local Priors in Regression. Bayesian Anal., advance publication, 19 April 2019. doi:10.1214/19-BA1154. https://projecteuclid.org/euclid.ba/1555660820


Export citation

References

  • Bian, Y. and Wu, H.-H. (2017). “A Note on Nonlocal Prior Method.” arXiv:1702.07778.
  • Cao, X., Khare, K., and Ghosh, M. (2019). “Posterior graph selection and estimation consistency for high-dimensional Bayesian DAG models.” Annals of Statistics, 47(1): 319–348.
  • Cao, X., Khare, K., and Ghosh, M. (2019). “Supplementary Material for “High-Dimensional Posterior Consistency for Hierarchical Non-Local Priors in Regression”.” Bayesian Analysis.
  • Castillo, I., Schmidt-Hieber, J., and van der Vaart, A. (2015). “Bayesian linear regression with sparse priors.” Annals of Statistics, 43: 1986–2018.
  • Fan, J. and Li, R. (2001). “Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties.” Journal of the American Statistical Association, 96: 1348–1360.
  • Friedman, J., Hastie, T., and Tibshirani, R. (2010). “Regularization Paths for Generalized Linear Models via Coordinate Descent.” Journal of Statistical Software, Articles, 33(1): 1–22.
  • George, E. I. and McCulloch, R. E. (1993). “Variable Selection via Gibbs Sampling.” Journal of the American Statistical Association, 88: 881–889.
  • Ishwaran, H., Kogalur, U. B., and Rao, J. S. (2005). “Spike and slab variable selection: Frequentist and Bayesian strategies.” Annals of Statistics, 33: 730–773.
  • Johnson, V. and Rossell, D. (2010). “On the Use of Non-Local Prior Densities in Bayesian Hypothesis Tests Hypothesis.” Journal of the Royal Statistical Society. Series B, 72: 143–170.
  • Johnson, V. and Rossell, D. (2012). “Bayesian Model Selection in High-Dimensional Settings.” Journal of the American Statistical Association, 107: 649–660.
  • Liang, F., Paulo, R., Molina, G., Clyde, A. M., and Berger, O. J. (2008). “Mixtures of $g$ Priors for Bayesian Variable Selection.” Journal of the American Statistical Association, 103: 410–423.
  • Narisetty, N. and He, X. (2014). “Bayesian variable selection with shrinking and diffusing priors.” Annals of Statistics, 42: 789–817.
  • Rossell, D. and Telesca, D. (2017). “Nonlocal Priors for High-Dimensional Estimation.” Journal of the American Statistical Association, 112(517): 254–265.
  • Rossell, D., Telesca, D., and Johnson, V. E. (2013). “High-Dimensional Bayesian Classifiers Using Non-Local Priors.” In Statistical Models for Data Analysis. Heidelberg: Springer International Publishing.
  • Shin, M., Bhattacharya, A., and Johnson, V. (2018). “Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-Dimensional Settings.” Statist. Sinica, 28: 1053–1078.
  • Song, Q. and Liang, F. (2015). “High-Dimensional Variable Selection With Reciprocal $L_{1}$-Regularization.” Journal of the American Statistical Association, 110: 1607–1620.
  • Tibshirani, R. (1996). “Regression Shrinkage and Selection Via the Lasso.” Journal of the Royal Statistical Society. Series B, 58: 267–288.
  • Wu, H.-H. (2016). “Nonlocal Priors for Bayesian Variable Selection in Generalized Linear Models and Generalized Linear Mixed Models and Their Applications in Biology Data.” Ph.D. thesis, University of Missouri.
  • Yuan, M. and Lin, Y. (2005). “Efficient Empirical Bayes Variable Selection and Estimation in Linear Models.” Journal of the American Statistical Association, 100(472): 1215–1225.
  • Zellner, A. (1986). “On assessing prior distributions and Bayesian regression analysis with g-prior distributions.” Bayesian Inference and Decision Techniques, Stud. Bayesian Econometrics Statist., 6: 233–243.

Supplemental materials