Adaptive Multivariate Ridge Regression

Abstract

A multivariate version of the Hoerl-Kennard ridge regression rule is introduced. The choice from among a large class of possible generalizations is guided by Bayesian considerations; the result is implicitly in the work of Lindley and Smith although not actually derived there. The proposed rule, in a variety of equivalent forms is discussed and the choice of its ridge matrix considered. As well, adaptive multivariate ridge rules and closely related empirical Bayes procedures are presented, these being for the most part formal extensions of certain univariate rules. Included is the Efron-Morris multivariate version of the James-Stein estimator. By means of an appropriate generalization of a result of Morris (see Thisted) the mean square error of these adaptive and empirical Bayes rules are compared.