Open Access
2011 PAC-Bayesian bounds for sparse regression estimation with exponential weights
Pierre Alquier, Karim Lounici
Electron. J. Statist. 5: 127-145 (2011). DOI: 10.1214/11-EJS601


We consider the sparse regression model where the number of parameters p is larger than the sample size n. The difficulty when considering high-dimensional problems is to propose estimators achieving a good compromise between statistical and computational performances. The Lasso is solution of a convex minimization problem, hence computable for large value of p. However stringent conditions on the design are required to establish fast rates of convergence for this estimator. Dalalyan and Tsybakov [17–19] proposed an exponential weights procedure achieving a good compromise between the statistical and computational aspects. This estimator can be computed for reasonably large p and satisfies a sparsity oracle inequality in expectation for the empirical excess risk only under mild assumptions on the design. In this paper, we propose an exponential weights estimator similar to that of [17] but with improved statistical performances. Our main result is a sparsity oracle inequality in probability for the true excess risk.


Download Citation

Pierre Alquier. Karim Lounici. "PAC-Bayesian bounds for sparse regression estimation with exponential weights." Electron. J. Statist. 5 127 - 145, 2011.


Published: 2011
First available in Project Euclid: 14 March 2011

zbMATH: 1274.62463
MathSciNet: MR2786484
Digital Object Identifier: 10.1214/11-EJS601

Primary: 62J07
Secondary: 62B10 , 62F15 , 62G08 , 62J05 , 68T05

Keywords: Exponential weights , high-dimensional regression , PAC-Bayesian inequalities , Sparsity oracle inequality

Rights: Copyright © 2011 The Institute of Mathematical Statistics and the Bernoulli Society

Back to Top