Institute of Mathematical Statistics Collections

Orthogonalized smoothing for rescaled spike and slab models

Hemant Ishwaran and Ariadni Papana

Full-text: Open access


Rescaled spike and slab models are a new Bayesian variable selection method for linear regression models. In high dimensional orthogonal settings such models have been shown to possess optimal model selection properties. We review background theory and discuss applications of rescaled spike and slab models to prediction problems involving orthogonal polynomials. We first consider global smoothing and discuss potential weaknesses. Some of these deficiencies are remedied by using local regression. The local regression approach relies on an intimate connection between local weighted regression and weighted generalized ridge regression. An important implication is that one can trace the effective degrees of freedom of a curve as a way to visualize and classify curvature. Several motivating examples are presented.

Chapter information

Bertrand Clarke and Subhashis Ghosal, eds., Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2008), 267-281

First available in Project Euclid: 28 April 2008

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Primary: 62J07: Ridge regression; shrinkage estimators
Secondary: 62J05: Linear regression

effective degrees of freedom penalization selective shrinkage

Copyright © 2008, Institute of Mathematical Statistics


Ishwaran, Hemant; Papana, Ariadni. Orthogonalized smoothing for rescaled spike and slab models. Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, 267--281, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2008. doi:10.1214/074921708000000192.

Export citation


  • [1] Ishwaran, H. and Rao, J. S. (2005). Spike and slab variable selection: frequentist and Bayesian strategies. Ann. Statist. 33 730–773.
  • [2] Ishwaran, H. and Rao, J. S. (2005). Spike and slab gene selection for multigroup microarray data. J. Amer. Statist. Assoc. 100 764–780.
  • [3] Ishwaran, H. and Rao, J. S. (2003). Detecting differentially expressed genes in microarrays using Bayesian model selection. J. Amer. Statist. Assoc. 98 438–455.
  • [4] Lempers, F. B. (1971). Posterior Probabilities of Alternative Linear Models. Rotterdam Univ. Press, Rotterdam.
  • [5] Mitchell, T. J. and Beauchamp, J. J. (1988). Bayesian variable selection in linear regression. J. Amer. Statist. Assoc. 83 1023–1036.
  • [6] George, E. I. and McCulloch, R. E. (1993). Variable selection via Gibbs sampling. J. Amer. Statist. Assoc. 88 881–889.
  • [7] George, E. I. and McCulloch, R. E. (1997). Approaches for Bayesian variable selection. Statist. Sinica 7 339–373.
  • [8] Chipman, H. (1996). Bayesian variable selection with related predictors. Canad. J. Statist. 24 17–36.
  • [9] Clyde, M., DeSimone, H. and Parmigiani, G. (1996). Prediction via orthogonalized model mixing. J. Amer. Statist. Assoc. 91 1197–1208.
  • [10] Geweke, J. and Meese, R. (1981). Estimating regression models of finite but unknown order. International Econ. Rev. 22 55–70.
  • [11] Chipman, H. A., George, E. I. and McCulloch R. E. (2001). The practical implementation of Bayesian model selection (with discussion). In IMS Lecture Notes Monogr. Ser. 38. Model Selection (P. Lahiri, ed.) 63–134. IMS, Beachwood, OH.
  • [12] Bachrach, L. K., Hastie, T., Wang, M.-C. and Narasimhan, B. (1999). Bone mineral acquisition in healthy asian, hispanic, black and caucasian youth. A longitudinal study. J. Clin. Endocrinol. Metab. 84 4702–4712.
  • [13] Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall, London.
  • [14] Hastie, T. and Loader, C. (1993). Local regression: automatic kernel carpentry. Statist. Sci. 8 120–143.
  • [15] Stone, C. J. (1977). Consistent nonparametric regression. Ann. Statist. 5 595–620.
  • [16] Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. J. Amer. Statist. Assoc. 74 829–836.
  • [17] Fan, J. (1992). Design-adaptive nonparametric regression. J. Amer. Statist. Assoc. 87 998–1004.
  • [18] Fan, J. and Gijbels, I. (1995). Data-driven bandwidth selection in local polynomial fitting: variable bandwidth and spatial adaptation. J. Roy. Statist. Soc. Ser. B 57 371–394.
  • [19] Rice, J. (1984). Bandwidth choice for nonparametric regression. Ann. Statist. 12 1215–1230.
  • [20] Genovese, C. R., Miller, C. J., Nichol, R. C., Arjunwadkar, M. and Wasserman, L. (2004). Nonparametric inference for the cosmic microwave background. Statist. Sci. 19 308–321.
  • [21] Li, X., Gentleman, R., Lu, X., Shi, Q., Iglehart, J. D., Harris, L. and Miron, A. (2005). SELDI-TOF mass spectometry protein data. In Bioinformatics and Computational Biology Solutions Using R and Bioconductor (R. Gentleman et al., eds.) 91–108. Springer, New York.