Open Access
October 2020 A general framework for Bayes structured linear models
Chao Gao, Aad W. van der Vaart, Harrison H. Zhou
Ann. Statist. 48(5): 2848-2878 (October 2020). DOI: 10.1214/19-AOS1909

Abstract

High dimensional statistics deals with the challenge of extracting structured information from complex model settings. Compared with a large number of frequentist methodologies, there are rather few theoretically optimal Bayes methods for high dimensional models. This paper provides a unified approach to both Bayes high dimensional statistics and Bayes nonparametrics in a general framework of structured linear models. With a proposed two-step prior, we prove a general oracle inequality for posterior contraction under an abstract setting that allows model misspecification. The general result can be used to derive new results on optimal posterior contraction under many complex model settings including recent works for stochastic block model, graphon estimation and dictionary learning. It can also be used to improve upon posterior contraction results in literature including sparse linear regression and nonparametric aggregation. The key of the success lies in the novel two-step prior distribution: one for model structure, that is, model selection, and the other one for model parameters. The prior on the parameters of a model is an elliptical Laplace distribution that is capable of modeling signals with large magnitude, and the prior on the model structure involves a factor that compensates the effect of the normalizing constant of the elliptical Laplace distribution, which is important to attain rate-optimal posterior contraction.

Citation

Download Citation

Chao Gao. Aad W. van der Vaart. Harrison H. Zhou. "A general framework for Bayes structured linear models." Ann. Statist. 48 (5) 2848 - 2878, October 2020. https://doi.org/10.1214/19-AOS1909

Information

Received: 1 July 2016; Revised: 1 July 2019; Published: October 2020
First available in Project Euclid: 19 September 2020

MathSciNet: MR4152123
Digital Object Identifier: 10.1214/19-AOS1909

Subjects:
Primary: 62C10
Secondary: 62F15

Keywords: Aggregation , dictionary learning , graphon , Oracle inequality , posterior contraction , sparse linear regression , Stochastic block model

Rights: Copyright © 2020 Institute of Mathematical Statistics

Vol.48 • No. 5 • October 2020
Back to Top