The Annals of Statistics

Robust estimation in structured linear regression

Lamine Mili and Clint W. Coakley

Full-text: Open access


A structured linear regression model is one in which there are permanent dependencies among some p row vectors of the $n \times p$ design matrix. To study structured linear regression, we introduce a new class of robust estimators, called D-estimators, which can be regarded as a generalization of the least median of squares and least trimmed squares estimators. They minimize a dispersion function of the ordered absolute residuals up to the rank h. We investigate their breakdown point and exact fit point as a function of h in structured linear regression. It is found that the D- and S-estimators can achieve the highest possible breakdown point for h appropriately chosen. It is shown that both the maximum breakdown point and the corresponding optimal value of h, $h_{\mathrm{op}}$, are sample dependent. They hinge on the design but not on the response. The relationship between the breakdown point and the design vanishes when h is strictly larger than $h_{\mathrm{op}}$. However, when h is smaller than $h_{\mathrm{op}}$, the breakdown point depends in a complicated way on the design as well as on the response.

Article information

Ann. Statist., Volume 24, Number 6 (1996), 2593-2607.

First available in Project Euclid: 16 September 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G35: Robustness
Secondary: 62J05: Linear regression 62K99: None of the above, but in this section 62N99: None of the above, but in this section

Robust estimation structured regression general position reduced position breakdown point exact fit point $D$-estimators


Mili, Lamine; Coakley, Clint W. Robust estimation in structured linear regression. Ann. Statist. 24 (1996), no. 6, 2593--2607. doi:10.1214/aos/1032181171.

Export citation


  • BASSETT, G. W., JR. 1991. Equivariant, monotonic, 50% breakdown estimators. Amer. Statist. 45 135 137. Z.
  • COAKLEY, C. W. 1991. Advances in the study of breakdown and resistance. Ph.D. dissertation, Dept. Statistics, Pennsy lvania State Univ. Z.
  • COAKLEY, C. W. and HETTMANSPERGER, T. P. 1993. A bounded influence, high breakdown, efficient regression estimator. J. Amer. Statist. Assoc. 88 872 880. Z.
  • COAKLEY, C. W. and MILI, L. 1993. Exact fit points under simple regression with replication. Statist. Probab. Lett. 17 265 271. Z.
  • COAKLEY, C. W., MILI, L. and CHENIAE, M. G. 1994. Effect of leverage on the finite sample efficiencies of high breakdown estimators. Statist. Probab. Lett. 19 399 408. Z.
  • CROUX, C., ROUSSEEUW, P. J. and HOSSJER, O. 1994. Generalized S-estimators. J. Amer. ¨ Statist. Assoc. 89 1271 1281. Z.
  • DAVIES, P. L. 1993. Aspects of robust linear regression. Ann. Statist. 21 1843 1899. Z.
  • DONOHO, D. L. and HUBER, P. J. 1983. The notion of breakdown point. In A Festschrift for Z. Erich L. Lehman P. J. Bickel, K. A. Doksum and J. L. Hodges, Jr., eds. 157 184. Wadsworth, Belmont, CA. Z.
  • DONOHO, D. L., JOHNSTONE, I., ROUSSEEUW, P. J. and STAHEL, W. 1985. Comment on ``Projection pursuit'' by P. J. Huber. Ann. Statist. 13 496 500. Z.
  • ELLIS, S. P. and MORGENTHALER, S. 1992. Leverage and breakdown in L regression. J. Amer. 1 Statist. Assoc. 87 143 148. Z.
  • HAMPEL, F. R. 1971. A general qualitative definition of robustness. Ann. Math. Statist. 42 1887 1896.
  • HAMPEL, F. R., RONCHETTI, E. M., ROUSSEEUW, P. J. and STAHEL, W. A. 1986. Robust Statistics: The Approach Based on Influence Functions. Wiley, New York. Z.
  • HODGES, J. L., JR. 1967. Efficiency in normal samples and tolerance of extreme values for some estimates of location. Proc. Fifth Berkeley Sy mp. Math. Statist. Probab. 1 163 168. Univ. California Press, Berkeley. Z.
  • MARTIN, R. D., YOHAI, V. J. and ZAMAR, R. H. 1989. Min-max bias robust regression. Ann. Statist. 17 1608 1630. Z.
  • MILI, L., CHENIAE, M. G. and ROUSSEEUW, P. J. 1994. Robust state estimation of electric power sy stems. IEEE Trans. Circuits and Sy stems 41 349 358. Z.
  • MILI, L. and COAKLEY, C. W. 1993. Robust estimation in structured linear regression. Technical Report 93-13, Dept. Statistics, Virginia Poly technic Institute and State Univ., Blacksburg. Z.
  • MILI, L., PHANIRAJ, V. and ROUSSEEUW, P. J. 1990. Robust estimation theory for bad data diagnostics in electric power sy stems. In Control and Dy namic Sy stems: Advances in Z. Theory and Applications C. T. Leondes, ed.. Advances in Industrial Sy stems 37 271 325. Academic Press, New York. Z.
  • MILI, L., PHANIRAJ, V. and ROUSSEEUW, P. J. 1991. Least median of squares estimation in power sy stems. IEEE Trans. Power Sy stems 6 511 523. Z.
  • MULLER, CH. H. 1995. Breakdown points for designed experiments. J. Statist. Plann. Inference ¨ 45 413 427. Z.
  • My ERS, R. H. and MONTGOMERY, D. C. 1995. Response Surface Methodology: Process and Product Optimization Using Designed Experiments. Wiley, New York. Z.
  • NETER, J., WASSERMAN, W. and KUTNER, M. H. 1985. Applied Linear Statistical Models, 2nd ed. Irwin, Homewood, IL. Z.
  • ROUSSEEUW, P. J. 1984. Least median of squares regression. J. Amer. Statist. Assoc. 79 871 880. Z.
  • ROUSSEEUW, P. J. and LEROY, A. M. 1987. Robust Regression and Outlier Detection. Wiley, New York. Z.
  • ROUSSEEUW, P. J. and YOHAI, V. 1984. Robust regression by means of S-estimators. In Robust and Nonlinear Time Series Analy sis. Lecture Notes in Statist. 26 256 272. Springer, New York. Z.
  • RUCKSTUHL, A. F. 1995. Analy sis of the T emission spectrum by robust estimation techniques. 2 Ph.D. thesis, Seminar fur Statistik, Swiss Federal Institute of Technology, Zurich. ¨ ¨ Z.
  • RUCKSTUHL, A. F., STAHEL, W. A. and DRESSLER, K. 1993. Robust estimation of term values in high-resolution spectroscopy: application to the e3 a3 spectrum of T. Jouru g 2 nal of Molecular Spectroscopy 160 434 445. Z.
  • SIMPSON, D. G., RUPPERT, D. and CARROLL, R. J. 1992. On one-step GM estimates and stability of inferences in linear regression. J. Amer. Statist. Assoc. 87 439 450. Z.
  • STROMBERG, A. J. 1992. Personal communication. Z. Z.
  • TABLEMAN, M. 1994. The asy mptotics of the least trimmed absolute deviations LTAD estimator. Statist. Probab. Lett. 17 387 398. Z.
  • YOHAI, V. J. 1987. High breakdown point and high efficiency robust estimates for regression. Ann. Statist. 15 642 656. Z.
  • YOHAI, V. J. and ZAMAR, R. 1988. High breakdown-point estimates of regression by means of the minimization of an efficient scale. J. Amer. Statist. Assoc. 83 406 413.