## Electronic Journal of Statistics

### Error bounds for the convex loss Lasso in linear models

#### Abstract

In this paper we investigate error bounds for convex loss functions for the Lasso in linear models, by first establishing a gap in the theory with respect to the existing error bounds. Then, under the compatibility condition, we recover bounds for the absolute value estimation error and the squared prediction error under mild conditions, which appear to be far more appropriate than the existing bounds for the convex loss Lasso. Interestingly, asymptotically the only difference between the new bounds of the convex loss Lasso and the classical Lasso is a term solely depending on a well-known expression in the robust statistics literature appearing multiplicatively in the bounds. We show that this result holds whether or not the scale parameter needs to be estimated jointly with the regression coefficients. Finally, we use the ratio to optimize our bounds in terms of minimaxity.

#### Article information

Source
Electron. J. Statist., Volume 11, Number 2 (2017), 2832-2875.

Dates
First available in Project Euclid: 8 August 2017

https://projecteuclid.org/euclid.ejs/1502157624

Digital Object Identifier
doi:10.1214/17-EJS1304

Mathematical Reviews number (MathSciNet)
MR3694570

Zentralblatt MATH identifier
1373.62369

Subjects
Primary: 62F35: Robustness and adaptive procedures
Secondary: 62J07: Ridge regression; shrinkage estimators

#### Citation

Hannay, Mark; Deléamont, Pierre-Yves. Error bounds for the convex loss Lasso in linear models. Electron. J. Statist. 11 (2017), no. 2, 2832--2875. doi:10.1214/17-EJS1304. https://projecteuclid.org/euclid.ejs/1502157624

#### References

• [1] Bühlmann, P. and van de Geer, S. (2011)., Statistics for High-dimensional Data: Methods, Theory and Applications. Heidelberg: Springer.
• [2] Fan, J., Fan, Y. and Barut, E. (2014). Adaptive robust variable selection., Ann. Statist. 42 324–351.
• [3] Gao, X. and Huang, J. (2010). Asymptotic analysis of high-dimensional LAD regression with Lasso., Statist. Sinica 20 1485–1506.
• [4] Greenshtein, E. (2006). Best subset selection, persistence in high-dimensional statistical learning and optimization under $l_1$ constraint., Ann. Statist. 34 2367–2386.
• [5] Greenshtein, E. and Ritov, Y. (2004). Persistence in high-dimensional linear predictor selection and the virtue of overparametrization., Bernoulli 10 971–988.
• [6] Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables., J. Amer. Statist. Assoc. 58 13–30.
• [7] Huber, P. J. (1964). Robust estimation of a location parameter., Ann. Math. Statist. 35 73–101.
• [8] Huber, P. J. and Ronchetti, E. M. (2009)., Robust Statistics. New York: Wiley.
• [9] Koltchinskii, V. (2011)., Oracle inequalities in empirical risk minimization and sparse recovery problems. Heidelberg: Springer.
• [10] Lambert-Lacroix, S. and Zwald, L. (2011). Robust regression through the Huber’s criterion and adaptive lasso penalty., Electron. J. Stat. 5 1015–1053.
• [11] Rosset, S. and Zhu, J. (2007). Piecewise linear regularized solution paths., Ann. Statist. 35 1012–1030.
• [12] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso., J. Roy. Statist. Soc. Ser. B 58 267–288.
• [13] Sun, T. and Zhang, C. H. (2012). Scaled sparse linear regression., Biometrika 99 879–898.
• [14] van de Geer, S. (2007). The deterministic Lasso. In, JSM Proceedings, 2007 140. American Statistical Association.
• [15] van de Geer, S. (2008). High-dimensional generalized linear models and the lasso., Ann. Statist. 36 614–645.
• [16] van de Geer, S. and Bühlmann, P. (2009). On the conditions used to prove oracle results for the Lasso., Electron. J. Stat. 3 1360–1392.
• [17] Wang, H., Li, G. and Jiang, G. (2007). Robust regression shrinkage and consistent variable selection through the LAD-Lasso., J. Bus. Econom. Statist. 25 347–355.