Open Access
December 2007 The Dantzig selector: Statistical estimation when p is much larger than n
Emmanuel Candes, Terence Tao
Ann. Statist. 35(6): 2313-2351 (December 2007). DOI: 10.1214/009053606000001523


In many important statistical applications, the number of variables or parameters p is much larger than the number of observations n. Suppose then that we have observations y=+z, where βRp is a parameter vector of interest, X is a data matrix with possibly far fewer rows than columns, np, and the zi’s are i.i.d. N(0, σ2). Is it possible to estimate β reliably based on the noisy data y?

To estimate β, we introduce a new estimator—we call it the Dantzig selector—which is a solution to the 1-regularization problem $$\min_{\tilde{\beta}\in\mathbf{R}^{p}}\|\tilde{\beta}\|_{\ell_{1}}\quad\mbox{subject to}\quad \|X^{*}r\|_{\ell_{\infty}}\leq(1+t^{-1})\sqrt{2\log p}\cdot\sigma,$$ where r is the residual vector yXβ̃ and t is a positive scalar. We show that if X obeys a uniform uncertainty principle (with unit-normed columns) and if the true parameter vector β is sufficiently sparse (which here roughly guarantees that the model is identifiable), then with very large probability,

β̂β22C2⋅2log p⋅(σ2+∑imin(βi2, σ2)).

Our results are nonasymptotic and we give values for the constant C. Even though n may be much smaller than p, our estimator achieves a loss within a logarithmic factor of the ideal mean squared error one would achieve with an oracle which would supply perfect information about which coordinates are nonzero, and which were above the noise level.

In multivariate regression and from a model selection viewpoint, our result says that it is possible nearly to select the best subset of variables by solving a very simple convex program, which, in fact, can easily be recast as a convenient linear program (LP).


Download Citation

Emmanuel Candes. Terence Tao. "The Dantzig selector: Statistical estimation when p is much larger than n." Ann. Statist. 35 (6) 2313 - 2351, December 2007.


Published: December 2007
First available in Project Euclid: 22 January 2008

zbMATH: 1139.62019
MathSciNet: MR2382644
Digital Object Identifier: 10.1214/009053606000001523

Primary: 62C05 , 62G05
Secondary: 94A08 , 94A12

Keywords: geometry in high dimensions , ideal estimation , ℓ_1-minimization , linear programming , Model selection , Oracle inequalities , random matrices , restricted orthonormality , sparse solutions to underdetermined systems , Statistical linear model

Rights: Copyright © 2007 Institute of Mathematical Statistics

Vol.35 • No. 6 • December 2007
Back to Top