Distribution-Free Pointwise Consistency of Kernel Regression Estimate

Abstract

An estimate $\sum^n_{i=1} Y_iK((x - X_i)/h)/\sum^n_{j=1} K((x - X_j)/h)$, calculated from a sequence $(X_1, Y_1), \cdots, (X_n, Y_n)$ of independent pairs of random variables distributed as a pair $(X, Y)$, converges to the regression $E\{Y\mid X = x\}$ as $n$ tends to infinity in probability for almost all $(\mu) x \in R^d$, provided that $E|Y| < \infty, h \rightarrow 0$ and $nh^d \rightarrow \infty$ as $n \rightarrow \infty$. The result is true for all distributions $\mu$ of $X$. If, moreover, $|Y| \leq \gamma < \infty$ and $nh^d/\log n \rightarrow \infty$ as $n \rightarrow \infty$, a complete convergence holds. The class of applicable kernels includes those having unbounded support.