Selection of model selection criteria for multivariate ridge regression

Abstract

In the present study, we consider the selection of model selection criteria for multivariate ridge regression. There are several model selection criteria for selecting the ridge parameter in multivariate ridge regression, e.g., the $C_p$ criterion and the modified $C_p$ ($MC_p$) criterion. We propose the generalized $C_p$ ($GC_p$) criterion, which includes $C_p$ and $MC_p$ criteria as special cases. The $GC_p$ criterion is specified by a non-negative parameter $\lambda$, which is referred to as the penalty parameter. We attempt to select an optimal penalty parameter such that the predicted mean square error (PMSE) of the predictor of ridge regression after optimizing the ridge parameter is minimized. Through numerical experiments, we verify that the proposed optimization methods exhibit better performance than conventional optimization methods, i.e., optimizing only the ridge parameter by minimizing the $C_p$ or $MC_p$ criterion.