On the Consistency of Cross-Validation in Kernel Nonparametric Regression

Abstract

For the nonparametric regression model $Y(t_i) = \theta(t_i) + \varepsilon(t_i)$ where $\theta$ is a smooth function to be estimated, $t_i$'s are nonrandom, $\varepsilon(t_i)$'s are i.i.d. errors, this paper studies the behavior of the kernel regression estimate $\hat{\theta}(t) = \big\lbrack \sum^n_{j=1}K \big(\frac{t_j - t}{\lambda}\big) Y(t_j) \big\rbrack / \big\lbrack\ sum^n_{j=1} K \big(\frac{t_j - t}{\lambda}\big) \big\rbrack$ when $\lambda$ is chosen by cross-validation on the average squared error. Strong consistency in terms of the average squared error is established for uniform spacing, compact kernel and finite fourth error moment.