The Admissible Bayes Character of Subset Selection Techniques Involved in Variable Selection, Outlier Detection, and Slippage Problems

Abstract

We demonstrate the admissible Bayes character of two residual error criteria for subset selection of independent variables in normal multivariate regression models. In particular, suppose a linear model includes the independent variable list $X$, and suppose all additional independent variable subsets of size $s$ (fixed) from list $Z$ are under consideration for inclusion in the model. Let $\mathbf{\Sigma}$ be the regression error covariance matrix and $\hat{\mathbf{\Sigma}}(\hat{\mathbf{\Sigma}}_\mathscr{J})$ the usual unbiased estimator of $\Sigma$ which assumes a model fitting the variables in list $X (X \cup \mathscr{J}$, where $\mathscr{J} \subset Z$ of size $s$). Then two best subsets of $Z$ of size $s$ may be characterized as minimizing $\operatorname{tr} \hat{\mathbf{\Sigma}}^{-1} \hat{\mathbf{\Sigma}}_\mathscr{J}$ and $|\hat{\mathbf{\Sigma}}_\mathscr{J}|/|\hat{\mathbf{\Sigma}}|$ over all subsets $\mathscr{J}$ of size $s$ in $Z$. We show that the significance tests for including-excluding these two best subsets are admissible proper Bayes rules for fixed effect variables in list $Z$. If $Z$ is allowed to encompass random and mixed effects, then the latter test is admissible proper Bayes in a class of location and scale invariant tests. Special cases of the general selection problem include multiple outlier detection and slippage tests where the best subset criteria above lead to Studentized residual outlier and slippage detection criteria. These tests are derived using models which explain outliers and slippage as locational biases and/or inflated variances.