A Comparison of Generalized Cross Validation and Modified Maximum Likelihood for Estimating the Parameters of a Stochastic Process

Abstract

Wahba compared the performance of generalized cross validation (GCV) and modified maximum likelihood (MML) procedures for choosing the smoothing parameter of a smoothing spline. This work makes a more careful study of the two procedures when the stochastic model motivating the modified maximum likelihood estimate is correct. In particular, it is shown that in the case of the linear smoothing spline with equally spaced observations, both estimates are asymptotically normal with the GCV estimate having twice the asymptotic variance of the MML estimate. The impact of using these estimates on the subsequent predictions is also calculated. Conjectures on how these results should generalize to higher order smoothing splines are developed. These conjectures suggest that the penalty for using GCV instead of MML when the stochastic model is correct is greater for higher order smoothing splines, both in terms of the efficiency in estimating the smoothing parameter and the impact on subsequent predictions.