- Bayesian Anal.
- Volume 12, Number 3 (2017), 753-778.
Selection of Tuning Parameters, Solution Paths and Standard Errors for Bayesian Lassos
Penalized regression methods such as the lasso and elastic net (EN) have become popular for simultaneous variable selection and coefficient estimation. Implementation of these methods require selection of the penalty parameters. We propose an empirical Bayes (EB) methodology for selecting these tuning parameters as well as computation of the regularization path plots. The EB method does not suffer from the “double shrinkage problem” of frequentist EN. Also it avoids the difficulty of constructing an appropriate prior on the penalty parameters. The EB methodology is implemented by efficient importance sampling method based on multiple Gibbs sampler chains. Since the Markov chains underlying the Gibbs sampler are proved to be geometrically ergodic, Markov chain central limit theorem can be used to provide asymptotically valid confidence band for profiles of EN coefficients. The practical effectiveness of our method is illustrated by several simulation examples and two real life case studies. Although this article considers lasso and EN for brevity, the proposed EB method is general and can be used to select shrinkage parameters in other regularization methods.
Bayesian Anal., Volume 12, Number 3 (2017), 753-778.
First available in Project Euclid: 7 September 2016
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Roy, Vivekananda; Chakraborty, Sounak. Selection of Tuning Parameters, Solution Paths and Standard Errors for Bayesian Lassos. Bayesian Anal. 12 (2017), no. 3, 753--778. doi:10.1214/16-BA1025. https://projecteuclid.org/euclid.ba/1473276258
- Supplementary Material: Supplementary Material for “Selection of Tuning Parameters, Solution Paths and Standard Errors for Bayesian Lassos”. The online supplementary materials contain the proofs of lemmas. Also a summary of the steps involved in the estimation of the tuning parameters and the solution paths is given in the supplementary materials.