Regression Models with Infinitely Many Parameters: Consistency of Bounded Linear Functionals

Abstract

Consider a linear model with infinitely many parameters given by $y = \sum^\infty_{i = 1} x_i\theta_i + \varepsilon$ where $\mathbf{x} = (x_1, x_2, \cdots)'$ and $\theta = (\theta_1, \theta_2, \cdots)'$ are infinite dimensional vectors such that $\sum^\infty_{i = 1}x^2_i < \infty$ and $\sum^\infty_{i = 1} \theta^2_i < \infty$. Suppose independent observations $y_1, y_2, \cdots, y_n$ are observed at levels $\mathbf{x}_1, \mathbf{x}_2, \cdots, \mathbf{x}_n$. Under suitable conditions about the error distribution, the set of all bounded linear functionals $T(\theta)$ for which there exists an estimate $\hat{T}_n$ such that $\hat{T}_n \rightarrow T(\theta)$ in probability will be characterized. An application will be extended to the nonparametric regression problem where the response curve $f$ is smooth on the interval [0, 1] in the sense that $f$ has an $(m - 1)$th derivative that is absolutely continuous and $\int^1_0 f^{(m)}(t)^2 dt < \infty$.