The Annals of Statistics

A class of robust and fully efficient regression estimators

Daniel Gervini and Victor J. Yohai

Full-text: Open access


This paper introduces a new class of robust estimators for the linear regression model. They are weighted least squares estimators, with weights adaptively computed using the empirical distribution of the residuals of an initial robust estimator. It is shown that under certain general conditions the asymptotic breakdown points of the proposed estimators are not less than that of the initial estimator, and the finite sample breakdown point can be at most $1/n$ less. For the special case of the least median of squares as initial estimator, hard rejection weights and normal errors and carriers, the maximum bias function of the proposed estimators for point-mass contaminations is numerically computed, with the result that there is almost no worsening of bias. Moreover–and this is the original contribution of this paper–if the errors are normally distributed and under fairly general conditions on the design the proposed estimators have full asymptotic efficiency. A Monte Carlo study shows that they have better behavior than the initial estimators for finite sample sizes.

Article information

Ann. Statist., Volume 30, Number 2 (2002), 583-616.

First available in Project Euclid: 14 May 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J05: Linear regression
Secondary: 62F35: Robustness and adaptive procedures

Adaptive estimation efficient estimation maximum breakdown point weighted least squares


Gervini, Daniel; Yohai, Victor J. A class of robust and fully efficient regression estimators. Ann. Statist. 30 (2002), no. 2, 583--616. doi:10.1214/aos/1021379866.

Export citation


  • ADROVER, J., BIANCO, A. and YOHAI, V. J. (2001). Approximate -estimates for linear regression based on subsampling of elemental sets. In Statistics in Genetics and in the Environmental Sciences (L. T. Fernholz, S. Morgenthaler and W. Stahel, eds.) 141-151. Birkäuser, Basel.
  • AGOSTINELLI, C. and MARKATOU, M. (1998). A one-step robust estimator for regression based on the weighted likelihood reweighting scheme. Statist. Probab. Lett. 37 341-350.
  • DAVIES, L. (1990). The asymptotics of S-estimators in the linear regression model. Ann. Statist. 18 1651-1675.
  • DONOHO, D. L. and HUBER, P. J. (1983). The notion of breakdown point. In A Festschrift for Erich L. Lehmann (P. J. Bickel, K. A. Doksum and J. L. Hodges, Jr., eds.) 157-184. Wadsworth, Belmont, CA.
  • GERVINI, D. and YOHAI, V. J. (2000). A class of robust and fully efficient regression estimators. Unpublished manuscript.
  • HAMPEL, F. R. (1975). Beyond location parameters: robust concepts and methods (with discussion). Bull. ISI 46 375-391.
  • HAMPEL, F. R., RONCHETTI, E. M., ROUSSEEUW, P. J. and STAHEL, W. A. (1986). Robust Statistics: The Approach Based on Influence Functions. Wiley, New York.
  • HE, X. and PORTNOY, S. (1992). Reweighted LS estimators converge at the same rate as the initial estimator. Ann. Statist. 20 2161-2167.
  • HÖSSJER, O. (1992). On the optimality of S-estimators. Statist. Probab. Lett. 14 413-419.
  • MARTIN, R. D., YOHAI, V. J. and ZAMAR, R. H. (1989). Min-max bias robust regression. Ann. Statist. 17 1608-1630.
  • POLLARD, D. (1990). Empirical Processes: Theory and Applications. IMS, Hayward, CA.
  • ROUSSEEUW, P. J. (1984). Least median of squares regression. J. Amer. Statist. Assoc. 79 871-880.
  • ROUSSEEUW, P. J. and LEROY, A. (1987). Robust Regression and Outlier Detection. Wiley, New York.
  • ROUSSEEUW, P. J. and YOHAI, V. (1984). Robust regression by means of S-estimators. Robust and Nonlinear Time Series Analysis. Lecture Notes in Statist. 26 256-272. Springer, New York.
  • SIMPSON, D. G. and YOHAI, V. J. (1998). Functional stability of one-step GM-estimators in approximately linear regression. Ann. Statist. 26 1147-1169.
  • YOHAI, V. J. (1987). High breakdown-point and high efficiency robust estimates for regression. Ann. Statist. 15 642-656.
  • YOHAI, V. J. and ZAMAR, R. H. (1988). High breakdown-point estimates of regression by means of the minimization of an efficient scale. J. Amer. Statist. Assoc. 83 406-413.
  • YOHAI, V. J. and ZAMAR, R. H. (1993). A minimax-bias property of the least -quantile estimates. Ann. Statist. 21 1824-1842.