Dimension of the Singular Sets of Plane-Fitters

Abstract

Let $n > p > k > 0$ be integers. Let $\delta$ be any technique for fitting $k$-planes to $p$-variate data sets of size $n$, for example, linear regression, principal components or projection pursuit. Let $\mathscr{Y}$ be the set of data sets which are (1) singularities of $\delta$, that is, near them $\delta$ is unstable (for example, collinear data sets are singularities of least squares regression) and (2) nondegenerate, that is, their rank, after centering, is at least $k$. It is shown that the Hausdorff dimension, $\dim_H(\mathscr{Y})$, of $\mathscr{Y}$ is at least $nk + (k + 1)(p - k) - 1$. This bound is tight. Under hypotheses satisfied by some projection pursuits (including principal components), $\dim_H(\mathscr{Y}) \geq np - 2$, that is, once singularity is taken into account, only two degrees of freedom remain in the problem! These results have implications for multivariate data description, resistant plane-fitting and jackknifing and bootstrapping plane-fitting.