Bootstrapping $M$-Estimators of a Multiple Linear Regression Parameter

Abstract

Consider a multiple linear regression model $Y_i = x'_i\beta + \varepsilon_i$, where the $\varepsilon_i$'s are independent random variables with common distribution $F$ and the $x_i$'s are known design vectors. Let $\bar\beta_n$ be the $M$-estimator of $\beta$ corresponding to a score function $\psi$. Under some conditions on $F, \psi$ and the $x_i$'s, two-term Edgeworth expansions for the distributions of standardized and studentized $\bar\beta_n$ are obtained. Furthermore, it is shown that the bootstrap method is second order correct in the studentized case when the bootstrap samples are drawn from some suitable weighted empirical distribution or from the ordinary empirical distribution of the residuals.