Open Access
July 2007 Piecewise linear regularized solution paths
Saharon Rosset, Ji Zhu
Ann. Statist. 35(3): 1012-1030 (July 2007). DOI: 10.1214/009053606000001370


We consider the generic regularized optimization problem β̂(λ)=arg minβL(y, )+λJ(β). Efron, Hastie, Johnstone and Tibshirani [Ann. Statist. 32 (2004) 407–499] have shown that for the LASSO—that is, if L is squared error loss and J(β)=‖β1 is the 1 norm of β—the optimal coefficient path is piecewise linear, that is, ∂β̂(λ)/∂λ is piecewise constant. We derive a general characterization of the properties of (loss L, penalty J) pairs which give piecewise linear coefficient paths. Such pairs allow for efficient generation of the full regularized coefficient paths. We investigate the nature of efficient path following algorithms which arise. We use our results to suggest robust versions of the LASSO for regression and classification, and to develop new, efficient algorithms for existing problems in the literature, including Mammen and van de Geer’s locally adaptive regression splines.


Download Citation

Saharon Rosset. Ji Zhu. "Piecewise linear regularized solution paths." Ann. Statist. 35 (3) 1012 - 1030, July 2007.


Published: July 2007
First available in Project Euclid: 24 July 2007

zbMATH: 1194.62094
MathSciNet: MR2341696
Digital Object Identifier: 10.1214/009053606000001370

Primary: 62J07
Secondary: 62F35 , 62G08 , 62H30

Keywords: ℓ_1-norm penalty , polynomial splines , regularization , solution paths , Sparsity , Total variation

Rights: Copyright © 2007 Institute of Mathematical Statistics

Vol.35 • No. 3 • July 2007
Back to Top