The Annals of Statistics

Asymptotic Behavior of $M$-Estimators of $p$ Regression Parameters when $p^2/n$ is Large. I. Consistency

Stephen Portnoy

Full-text: Open access

Abstract

Consider the general linear model $Y = x\beta + R$ with $Y$ and $R n$-dimensional, $\beta p$-dimensional, and $X$ an $n \times p$ matrix with rows $x'_i$. Let $\psi$ be given and let $\hat\beta$ be an $M$-estimator of $\beta$ satisfying $0 = \sum x_i\psi(Y_i - x'_i\hat\beta)$. Previous authors have considered consistency and asymptotic normality of $\hat\beta$ when $p$ is permitted to grow, but they have required at least $p^2/n \rightarrow 0$. Here the following result is presented: in typical regression cases, under reasonable conditions if $p(\log p)/n \rightarrow 0$ then $\|\hat{\beta} - \beta\|^2 = \mathscr{O}_p(p/n)$. A subsequent paper will show that $\hat{\beta}$ has a normal approximation in $R^p$ if $(p \log p)^{3/2}/n \rightarrow 0$ and that $\max_i|x'_i(\hat{\beta} - \beta)| \rightarrow_p 0$ (which would not follow from norm consistency if $p^2/n \rightarrow \infty$). In ANOVA cases, $\hat{\beta}$ is not norm consistent, but it is shown here that $\max|x'_i(\hat{\beta} - \beta)| \rightarrow_p 0$ if $p \log p/n \rightarrow 0$. A normality result for arbitrary linear combinations $a'(\hat{\beta} - \beta)$ is also presented in this case.

Article information

Source
Ann. Statist., Volume 12, Number 4 (1984), 1298-1309.

Dates
First available in Project Euclid: 12 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aos/1176346793

Digital Object Identifier
doi:10.1214/aos/1176346793

Mathematical Reviews number (MathSciNet)
MR760690

Zentralblatt MATH identifier
0584.62050

JSTOR
links.jstor.org

Subjects
Primary: 62G35: Robustness
Secondary: 62E20: Asymptotic distribution theory 62J05: Linear regression

Keywords
$M$-estimators general linear model asymptotic normality consistency regression robustness

Citation

Portnoy, Stephen. Asymptotic Behavior of $M$-Estimators of $p$ Regression Parameters when $p^2/n$ is Large. I. Consistency. Ann. Statist. 12 (1984), no. 4, 1298--1309. doi:10.1214/aos/1176346793. https://projecteuclid.org/euclid.aos/1176346793


Export citation

See also

  • Part II: Stephen Portnoy. Asymptotic Behavior of $M$ Estimators of $p$ Regression Parameters when $p^2 / n$ is Large; II. Normal Approximation. Ann. Statist., Volume 13, Number 4 (1985), 1403--1417.