The Annals of Statistics
- Ann. Statist.
- Volume 39, Number 1 (2011), 82-130.
ℓ1-penalized quantile regression in high-dimensional sparse models
Alexandre Belloni and Victor Chernozhukov
Abstract
We consider median regression and, more generally, a possibly infinite collection of quantile regressions in high-dimensional sparse models. In these models, the number of regressors p is very large, possibly larger than the sample size n, but only at most s regressors have a nonzero impact on each conditional quantile of the response variable, where s grows more slowly than n. Since ordinary quantile regression is not consistent in this case, we consider ℓ1-penalized quantile regression (ℓ1-QR), which penalizes the ℓ1-norm of regression coefficients, as well as the post-penalized QR estimator (post-ℓ1-QR), which applies ordinary QR to the model selected by ℓ1-QR. First, we show that under general conditions ℓ1-QR is consistent at the near-oracle rate $\sqrt{s/n}\sqrt{\log(p\vee n)}$, uniformly in the compact set $\mathcal{U}\subset(0,1)$ of quantile indices. In deriving this result, we propose a partly pivotal, data-driven choice of the penalty level and show that it satisfies the requirements for achieving this rate. Second, we show that under similar conditions post-ℓ1-QR is consistent at the near-oracle rate $\sqrt{s/n}\sqrt{\log(p\vee n)}$, uniformly over $\mathcal{U}$, even if the ℓ1-QR-selected models miss some components of the true models, and the rate could be even closer to the oracle rate otherwise. Third, we characterize conditions under which ℓ1-QR contains the true model as a submodel, and derive bounds on the dimension of the selected model, uniformly over $\mathcal{U}$; we also provide conditions under which hard-thresholding selects the minimal true model, uniformly over $\mathcal{U}$.
Article information
Source
Ann. Statist., Volume 39, Number 1 (2011), 82-130.
Dates
First available in Project Euclid: 3 December 2010
Permanent link to this document
https://projecteuclid.org/euclid.aos/1291388370
Digital Object Identifier
doi:10.1214/10-AOS827
Mathematical Reviews number (MathSciNet)
MR2797841
Zentralblatt MATH identifier
1209.62064
Subjects
Primary: 62H12: Estimation 62J99: None of the above, but in this section
Secondary: 62J07: Ridge regression; shrinkage estimators
Keywords
Median regression quantile regression sparse models
Citation
Belloni, Alexandre; Chernozhukov, Victor. ℓ 1 -penalized quantile regression in high-dimensional sparse models. Ann. Statist. 39 (2011), no. 1, 82--130. doi:10.1214/10-AOS827. https://projecteuclid.org/euclid.aos/1291388370
Supplemental materials
- Supplementary material: Supplement to “ℓ1-penalized quantile regression in high-dimensional sparse models”. We included technical proofs omitted from the main text: Examples of simple sufficient conditions, VC index bounds and Gaussian sparse eigenvalues.Digital Object Identifier: doi:10.1214/10-AOS827SUPP
