Open Access
December 2009 A unified approach to model selection and sparse recovery using regularized least squares
Jinchi Lv, Yingying Fan
Ann. Statist. 37(6A): 3498-3528 (December 2009). DOI: 10.1214/09-AOS683


Model selection and sparse recovery are two important problems for which many regularization methods have been proposed. We study the properties of regularization methods in both problems under the unified framework of regularized least squares with concave penalties. For model selection, we establish conditions under which a regularized least squares estimator enjoys a nonasymptotic property, called the weak oracle property, where the dimensionality can grow exponentially with sample size. For sparse recovery, we present a sufficient condition that ensures the recoverability of the sparsest solution. In particular, we approach both problems by considering a family of penalties that give a smooth homotopy between L0 and L1 penalties. We also propose the sequentially and iteratively reweighted squares (SIRS) algorithm for sparse recovery. Numerical studies support our theoretical results and demonstrate the advantage of our new methods for model selection and sparse recovery.


Download Citation

Jinchi Lv. Yingying Fan. "A unified approach to model selection and sparse recovery using regularized least squares." Ann. Statist. 37 (6A) 3498 - 3528, December 2009.


Published: December 2009
First available in Project Euclid: 17 August 2009

zbMATH: 1369.62156
MathSciNet: MR2549567
Digital Object Identifier: 10.1214/09-AOS683

Primary: 62J99
Secondary: 62F99

Keywords: concave penalty , high dimensionality , Model selection , regularized least squares , sparse recovery , weak oracle property

Rights: Copyright © 2009 Institute of Mathematical Statistics

Vol.37 • No. 6A • December 2009
Back to Top