Open Access
Translator Disclaimer
2008 Honest variable selection in linear and logistic regression models via 1 and 1+2 penalization
Florentina Bunea
Electron. J. Statist. 2: 1153-1194 (2008). DOI: 10.1214/08-EJS287

Abstract

This paper investigates correct variable selection in finite samples via 1 and 1+2 type penalization schemes. The asymptotic consistency of variable selection immediately follows from this analysis. We focus on logistic and linear regression models. The following questions are central to our paper: given a level of confidence 1δ, under which assumptions on the design matrix, for which strength of the signal and for what values of the tuning parameters can we identify the true model at the given level of confidence? Formally, if is an estimate of the true variable set I*, we study conditions under which ℙ(=I*)1δ, for a given sample size n, number of parameters M and confidence 1δ. We show that in identifiable models, both methods can recover coefficients of size $\frac{1}{\sqrt{n}}$, up to small multiplicative constants and logarithmic factors in M and $\frac{1}{\delta}$. The advantage of the 1+2 penalization over the 1 is minor for the variable selection problem, for the models we consider here. Whereas the former estimates are unique, and become more stable for highly correlated data matrices as one increases the tuning parameter of the 2 part, too large an increase in this parameter value may preclude variable selection.

Citation

Download Citation

Florentina Bunea. "Honest variable selection in linear and logistic regression models via 1 and 1+2 penalization." Electron. J. Statist. 2 1153 - 1194, 2008. https://doi.org/10.1214/08-EJS287

Information

Published: 2008
First available in Project Euclid: 16 December 2008

zbMATH: 1320.62170
MathSciNet: MR2461898
Digital Object Identifier: 10.1214/08-EJS287

Subjects:
Primary: 62J07
Secondary: 62G08 , 62J02

Keywords: consistent , Elastic net , generalized linear models , high dimensions , ℓ_1 and ℓ_1+ℓ_2 regularization , Lasso , logistic regression , Penalty , regression , sparse , Variable selection

Rights: Copyright © 2008 The Institute of Mathematical Statistics and the Bernoulli Society

JOURNAL ARTICLE
42 PAGES


SHARE
Back to Top