Open Access
November 2012 A Unified Framework for High-Dimensional Analysis of $M$-Estimators with Decomposable Regularizers
Sahand N. Negahban, Pradeep Ravikumar, Martin J. Wainwright, Bin Yu
Statist. Sci. 27(4): 538-557 (November 2012). DOI: 10.1214/12-STS400


High-dimensional statistical inference deals with models in which the the number of parameters $p$ is comparable to or larger than the sample size $n$. Since it is usually impossible to obtain consistent procedures unless $p/n\rightarrow0$, a line of recent work has studied models with various types of low-dimensional structure, including sparse vectors, sparse and structured matrices, low-rank matrices and combinations thereof. In such settings, a general approach to estimation is to solve a regularized optimization problem, which combines a loss function measuring how well the model fits the data with some regularization function that encourages the assumed structure. This paper provides a unified framework for establishing consistency and convergence rates for such regularized $M$-estimators under high-dimensional scaling. We state one main theorem and show how it can be used to re-derive some existing results, and also to obtain a number of new results on consistency and convergence rates, in both $\ell_{2}$-error and related norms. Our analysis also identifies two key properties of loss and regularization functions, referred to as restricted strong convexity and decomposability, that ensure corresponding regularized $M$-estimators have fast convergence rates and which are optimal in many well-studied cases.


Download Citation

Sahand N. Negahban. Pradeep Ravikumar. Martin J. Wainwright. Bin Yu. "A Unified Framework for High-Dimensional Analysis of $M$-Estimators with Decomposable Regularizers." Statist. Sci. 27 (4) 538 - 557, November 2012.


Published: November 2012
First available in Project Euclid: 21 December 2012

zbMATH: 1331.62350
MathSciNet: MR3025133
Digital Object Identifier: 10.1214/12-STS400

Keywords: $\ell_{1}$-regularization , $M$-estimator , group lasso , High-dimensional statistics , Lasso , nuclear norm , Sparsity

Rights: Copyright © 2012 Institute of Mathematical Statistics

Vol.27 • No. 4 • November 2012
Back to Top