The Annals of Statistics
- Ann. Statist.
- Volume 41, Number 3 (2013), 1111-1141.
A lasso for hierarchical interactions
We add a set of convex constraints to the lasso to produce sparse interaction models that honor the hierarchy restriction that an interaction only be included in a model if one or both variables are marginally important. We give a precise characterization of the effect of this hierarchy constraint, prove that hierarchy holds with probability one and derive an unbiased estimate for the degrees of freedom of our estimator. A bound on this estimate reveals the amount of fitting “saved” by the hierarchy constraint.
We distinguish between parameter sparsity—the number of nonzero coefficients—and practical sparsity—the number of raw variables one must measure to make a new prediction. Hierarchy focuses on the latter, which is more closely tied to important data collection concerns such as cost, time and effort. We develop an algorithm, available in the R package hierNet, and perform an empirical study of our method.
Ann. Statist., Volume 41, Number 3 (2013), 1111-1141.
First available in Project Euclid: 13 June 2013
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 62J07: Ridge regression; shrinkage estimators
Bien, Jacob; Taylor, Jonathan; Tibshirani, Robert. A lasso for hierarchical interactions. Ann. Statist. 41 (2013), no. 3, 1111--1141. doi:10.1214/13-AOS1096. https://projecteuclid.org/euclid.aos/1371150895
- Supplementary material: Supplement to “A lasso for hierarchical interactions”. We include proofs of Property 1 and of the statement in Remark 3. Additionally, we show that the algorithm for the logistic regression case is nearly identical and give more detail on Algorithm 2.