Asymptotic Behavior of $M$-Estimators of $p$ Regression Parameters when $p^2/n$ is Large. I. Consistency

Abstract

Consider the general linear model $Y = x\beta + R$ with $Y$ and $R n$-dimensional, $\beta p$-dimensional, and $X$ an $n \times p$ matrix with rows $x'_i$. Let $\psi$ be given and let $\hat\beta$ be an $M$-estimator of $\beta$ satisfying $0 = \sum x_i\psi(Y_i - x'_i\hat\beta)$. Previous authors have considered consistency and asymptotic normality of $\hat\beta$ when $p$ is permitted to grow, but they have required at least $p^2/n \rightarrow 0$. Here the following result is presented: in typical regression cases, under reasonable conditions if $p(\log p)/n \rightarrow 0$ then $\|\hat{\beta} - \beta\|^2 = \mathscr{O}_p(p/n)$. A subsequent paper will show that $\hat{\beta}$ has a normal approximation in $R^p$ if $(p \log p)^{3/2}/n \rightarrow 0$ and that $\max_i|x'_i(\hat{\beta} - \beta)| \rightarrow_p 0$ (which would not follow from norm consistency if $p^2/n \rightarrow \infty$). In ANOVA cases, $\hat{\beta}$ is not norm consistent, but it is shown here that $\max|x'_i(\hat{\beta} - \beta)| \rightarrow_p 0$ if $p \log p/n \rightarrow 0$. A normality result for arbitrary linear combinations $a'(\hat{\beta} - \beta)$ is also presented in this case.