Open Access
December 2017 Support recovery without incoherence: A case for nonconvex regularization
Po-Ling Loh, Martin J. Wainwright
Ann. Statist. 45(6): 2455-2482 (December 2017). DOI: 10.1214/16-AOS1530


We develop a new primal-dual witness proof framework that may be used to establish variable selection consistency and $\ell_{\infty}$-bounds for sparse regression problems, even when the loss function and regularizer are nonconvex. We use this method to prove two theorems concerning support recovery and $\ell_{\infty}$-guarantees for a regression estimator in a general setting. Notably, our theory applies to all potential stationary points of the objective and certifies that the stationary point is unique under mild conditions. Our results provide a strong theoretical justification for the use of nonconvex regularization: For certain nonconvex regularizers with vanishing derivative away from the origin, any stationary point can be used to recover the support without requiring the typical incoherence conditions present in $\ell_{1}$-based methods. We also derive corollaries illustrating the implications of our theorems for composite objective functions involving losses such as least squares, nonconvex modified least squares for errors-in-variables linear regression, the negative log likelihood for generalized linear models and the graphical Lasso. We conclude with empirical studies that corroborate our theoretical predictions.


Download Citation

Po-Ling Loh. Martin J. Wainwright. "Support recovery without incoherence: A case for nonconvex regularization." Ann. Statist. 45 (6) 2455 - 2482, December 2017.


Received: 1 May 2015; Revised: 1 July 2016; Published: December 2017
First available in Project Euclid: 15 December 2017

zbMATH: 1385.62008
MathSciNet: MR3737898
Digital Object Identifier: 10.1214/16-AOS1530

Primary: 62F12

Keywords: $M$-estimator , High-dimensional statistics , Lasso , nonconvex regularizer , Sparsity , Variable selection

Rights: Copyright © 2017 Institute of Mathematical Statistics

Vol.45 • No. 6 • December 2017
Back to Top