Electronic Journal of Statistics

An $\{\ell_{1},\ell_{2},\ell_{\infty}\}$-regularization approach to high-dimensional errors-in-variables models

Alexandre Belloni, Mathieu Rosenbaum, and Alexandre B. Tsybakov

Full-text: Open access

Abstract

Several new estimation methods have been recently proposed for the linear regression model with observation errors in the design. Different assumptions on the data generating process have motivated different estimators and analysis. In particular, the literature considered (1) observation errors in the design uniformly bounded by some $\bar{\delta}$, and (2) zero-mean independent observation errors. Under the first assumption, the rates of convergence of the proposed estimators depend explicitly on $\bar{\delta}$, while the second assumption has been essentially applied when an estimator for the second moment of the observational error is available. This work proposes and studies two new estimators which, compared to other procedures for regression models with errors in the design, exploit an additional $\ell_{\infty}$-norm regularization. The first estimator is applicable when both (1) and (2) hold but does not require an estimator for the second moment of the observational error. The second estimator is applicable under (2) and requires an estimator for the second moment of the observation error. Importantly, we impose no assumption on the accuracy of this pilot estimator, in contrast to the previously known procedures. As the recent proposals, we allow the number of covariates to be much larger than the sample size. We establish the rates of convergence of the estimators and compare them with the bounds obtained for related estimators in the literature. These comparisons show interesting insights on the interplay of the assumptions and the achievable rates of convergence.

Article information

Source
Electron. J. Statist., Volume 10, Number 2 (2016), 1729-1750.

Dates
Received: December 2014
First available in Project Euclid: 18 July 2016

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1468849962

Digital Object Identifier
doi:10.1214/15-EJS1095

Mathematical Reviews number (MathSciNet)
MR3522659

Zentralblatt MATH identifier
06624500

Citation

Belloni, Alexandre; Rosenbaum, Mathieu; Tsybakov, Alexandre B. An $\{\ell_{1},\ell_{2},\ell_{\infty}\}$-regularization approach to high-dimensional errors-in-variables models. Electron. J. Statist. 10 (2016), no. 2, 1729--1750. doi:10.1214/15-EJS1095. https://projecteuclid.org/euclid.ejs/1468849962


Export citation

References

  • [1] Belloni, A., Rosenbaum, M. and Tsybakov, A. B. (2014) Linear and conic programming approaches to high-dimensional errors-in-variables models., arxiv:1408.0241
  • [2] Chen, Y. and Caramanis, C. (2012) Orthogonal matching pursuit with noisy and missing data: Low and high-dimensional results., arxiv:1206.0823
  • [3] Chen, Y. and Caramanis, C. (2013) Noisy and missing data regression: Distribution-oblivious support recovery., Proc. of International Conference on Machine Learning (ICML).
  • [4] Freedman, L. S., Midthune, D., Carroll, R. J. and Kipnis, V. (2008) A comparison of regression calibration, moment reconstruction and imputation for adjusting for covariate measurement error in regression., Statistics in Medicine, 27, 5195–5216.
  • [5] Gautier, E. and Tsybakov, A. B. (2011) High-dimensional instrumental variables regression and confidence sets., arxiv:1105.2454
  • [6] Gautier, E. and Tsybakov, A. B. (2013) Pivotal estimation in high-dimensional regression via linear programming. In:, Empirical Inference – Festschrift in Honor of Vladimir N. Vapnik, Schölkopf, B., Luo, Z., Vovk, V. eds., 195–204. Springer, New York e.a.
  • [7] Loh, P.-L. and Wainwright, M. J. (2012) High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity., Annals of Statistics, 40, 1637–1664.
  • [8] Rosenbaum, M. and Tsybakov, A. B. (2010) Sparse Recovery under Matrix Uncertainty., Annals of Statistics, 38, 2620–2651.
  • [9] Rosenbaum, M. and Tsybakov, A. B. (2013) Improved matrix uncertainty selector. In:, From Probability to Statistics and Back: High-Dimensional Models and Processes – A Festschrift in Honor of Jon A. Wellner, Banerjee, M. et al. ed., IMS Collections, vol. 9, 276–290, Institute of Mathematical Statistics.
  • [10] Sørensen, Ø., Frigessi, A. and Thoresen, M. (2012) Measurement error in Lasso: Impact and likelihood bias correction., arxiv:1210.5378
  • [11] Sørensen, Ø., Frigessi, A. and Thoresen, M. (2014) Covariate selection in high-dimensional generalized linear models with measurement error., arxiv:1407.1070