The Annals of Statistics

The Admissible Bayes Character of Subset Selection Techniques Involved in Variable Selection, Outlier Detection, and Slippage Problems

Ronald W. Butler

Full-text: Open access

Abstract

We demonstrate the admissible Bayes character of two residual error criteria for subset selection of independent variables in normal multivariate regression models. In particular, suppose a linear model includes the independent variable list $X$, and suppose all additional independent variable subsets of size $s$ (fixed) from list $Z$ are under consideration for inclusion in the model. Let $\mathbf{\Sigma}$ be the regression error covariance matrix and $\hat{\mathbf{\Sigma}}(\hat{\mathbf{\Sigma}}_\mathscr{J})$ the usual unbiased estimator of $\Sigma$ which assumes a model fitting the variables in list $X (X \cup \mathscr{J}$, where $\mathscr{J} \subset Z$ of size $s$). Then two best subsets of $Z$ of size $s$ may be characterized as minimizing $\operatorname{tr} \hat{\mathbf{\Sigma}}^{-1} \hat{\mathbf{\Sigma}}_\mathscr{J}$ and $|\hat{\mathbf{\Sigma}}_\mathscr{J}|/|\hat{\mathbf{\Sigma}}|$ over all subsets $\mathscr{J}$ of size $s$ in $Z$. We show that the significance tests for including-excluding these two best subsets are admissible proper Bayes rules for fixed effect variables in list $Z$. If $Z$ is allowed to encompass random and mixed effects, then the latter test is admissible proper Bayes in a class of location and scale invariant tests. Special cases of the general selection problem include multiple outlier detection and slippage tests where the best subset criteria above lead to Studentized residual outlier and slippage detection criteria. These tests are derived using models which explain outliers and slippage as locational biases and/or inflated variances.

Article information

Source
Ann. Statist., Volume 9, Number 5 (1981), 960-973.

Dates
First available in Project Euclid: 12 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aos/1176345576

Digital Object Identifier
doi:10.1214/aos/1176345576

Mathematical Reviews number (MathSciNet)
MR628752

Zentralblatt MATH identifier
0484.62011

JSTOR
links.jstor.org

Subjects
Primary: 62C25: Compound decision problems
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures 62J05: Linear regression 62G35: Robustness

Keywords
Admissibility Bayesian decision rule variable selection outliers slippage tests multivariate regression Models I and II invariance Studentized residuals

Citation

Butler, Ronald W. The Admissible Bayes Character of Subset Selection Techniques Involved in Variable Selection, Outlier Detection, and Slippage Problems. Ann. Statist. 9 (1981), no. 5, 960--973. doi:10.1214/aos/1176345576. https://projecteuclid.org/euclid.aos/1176345576


Export citation