The limiting behavior of isotonic and convex regression estimators when the model is misspecified

Abstract

We study the asymptotic behavior of the least squares estimators when the model is possibly misspecified. We consider the setting where we wish to estimate an unknown function $f_{*}:(0,1)^{d}\rightarrow \mathbb{R}$ from observations $(X,Y),(X_{1},Y_{1}),\cdots ,(X_{n},Y_{n})$; our estimator $\hat{g}_{n}$ is the minimizer of $\sum _{i=1}^{n}(Y_{i}-g(X_{i}))^{2}/n$ over $g\in \mathcal{G}$ for some set of functions $\mathcal{G}$. We provide sufficient conditions on the metric entropy of $\mathcal{G}$, under which $\hat{g}_{n}$ converges to $g_{*}$ as $n\rightarrow \infty $, where $g_{*}$ is the minimizer of $\|g-f_{*}\|\triangleq \mathbb{E}(g(X)-f_{*}(X))^{2}$ over $g\in \mathcal{G}$. As corollaries of our theorem, we establish $\|\hat{g}_{n}-g_{*}\|\rightarrow 0$ as $n\rightarrow \infty $ when $\mathcal{G}$ is the set of monotone functions or the set of convex functions. We also make a connection between the convergence rate of $\|\hat{g}_{n}-g_{*}\|$ and the metric entropy of $\mathcal{G}$. As special cases of our finding, we compute the convergence rate of $\|\hat{g}_{n}-g_{*}\|^{2}$ when $\mathcal{G}$ is the set of bounded monotone functions or the set of bounded convex functions.