Open Access
August 2012 Regularization of Case-Specific Parameters for Robustness and Efficiency
Yoonkyung Lee, Steven N. MacEachern, Yoonsuh Jung
Statist. Sci. 27(3): 350-372 (August 2012). DOI: 10.1214/11-STS377


Regularization methods allow one to handle a variety of inferential problems where there are more covariates than cases. This allows one to consider a potentially enormous number of covariates for a problem. We exploit the power of these techniques, supersaturating models by augmenting the “natural” covariates in the problem with an additional indicator for each case in the data set. We attach a penalty term for these case-specific indicators which is designed to produce a desired effect. For regression methods with squared error loss, an $\ell_{1}$ penalty produces a regression which is robust to outliers and high leverage cases; for quantile regression methods, an $\ell_{2}$ penalty decreases the variance of the fit enough to overcome an increase in bias. The paradigm thus allows us to robustify procedures which lack robustness and to increase the efficiency of procedures which are robust.

We provide a general framework for the inclusion of case-specific parameters in regularization problems, describing the impact on the effective loss for a variety of regression and classification problems. We outline a computational strategy by which existing software can be modified to solve the augmented regularization problem, providing conditions under which such modification will converge to the optimum solution. We illustrate the benefits of including case-specific parameters in the context of mean regression and quantile regression through analysis of NHANES and linguistic data sets.


Download Citation

Yoonkyung Lee. Steven N. MacEachern. Yoonsuh Jung. "Regularization of Case-Specific Parameters for Robustness and Efficiency." Statist. Sci. 27 (3) 350 - 372, August 2012.


Published: August 2012
First available in Project Euclid: 5 September 2012

zbMATH: 1331.62349
MathSciNet: MR3012431
Digital Object Identifier: 10.1214/11-STS377

Keywords: Case indicator , large margin classifier , Lasso , leverage point , outlier , penalized method , Quantile regression

Rights: Copyright © 2012 Institute of Mathematical Statistics

Vol.27 • No. 3 • August 2012
Back to Top