The Singularities of Fitting Planes to Data

Abstract

Plane-fitting, for example, linear regression, principal components or projection pursuit, is treated from a general perspective. It is shown that any method of plane-fitting satisfying very mild hypotheses must have singularities, that is, data sets near which the procedure is unstable. The well-known collinearity phenomenon in least squares regression is a special case. Severity of singularities is also discussed. The results, which are applications of algebraic topology, may be viewed as putting limits on how much can be done through robustification to stabilize plane-fitting.