## The Annals of Mathematical Statistics

- Ann. Math. Statist.
- Volume 36, Number 5 (1965), 1579-1582.

### An Optimal Property of Principal Components

#### Abstract

Let $x' = (x_1, x_2, \cdots, x_p)$ be a random vector and let $E\lbrack x\rbrack = 0, E\lbrack xx'\rbrack = \Sigma = (\sigma_{ij})$ where we assume that $\Sigma$ is non-singular. Further let \[ \Sigma = T\Lambda T' = (t_1, t_2, \cdots, t_p) \left[ \begin{array}{c|c|c|c|c|c|c}\lambda_1 & 0 & \cdot & \cdot & \cdot & 0 \\ 0 & \lambda_2 & \cdot & \cdot & \cdot & 0 \\ \vdots & \vdots & \vdots & \vdots \\0 & 0 & \cdot & \cdot & \cdot & \lambda_p\end{array} \right] \] \[ \left[ \begin{array}{|c}t'_1 \\ t'_2 \\ \vdots \\ t'_p\end{array}\right] \] where $TT' = I$ and where we suppose that the eigenvalues of $\Sigma$ are in order of decreasing magnitude, that is $\lambda_1 \geqq \lambda_2 \geqq \cdots \geqq \lambda_p > 0$. The principal components of $x$, namely $u_1 = t'_1 x, u_2 = t'_2 x, \cdots, u_p = t_p' x$ were introduced by Hotelling (1933) who characterised them by certain optimal properties. Since then Girshick (1936), Anderson (1958) and Kullback (1959) have characterised the principal components by slightly different sets of optimal properties. Thus Anderson shows that $u_1$ is the linear function $\alpha_1' x$ having maximum variance subject to $\alpha_1' \alpha_1 = 1; u_2$ is the linear function $\alpha_2'x$ which is uncorrelated with $u_1$ and has maximum variance subject to $\alpha_2'\alpha_2 = 1$; and so on. The above mentioned characterisations have two properties in common; they introduce the principal components one by one and, more importantly, the optimal properties hold only with-in the class of linear functions of $x_1, x_2, \cdots, x_p$. In the following theorem the first $k$ principal components are characterized by an optimal property within the class of all random variables.

#### Article information

**Source**

Ann. Math. Statist., Volume 36, Number 5 (1965), 1579-1582.

**Dates**

First available in Project Euclid: 27 April 2007

**Permanent link to this document**

https://projecteuclid.org/euclid.aoms/1177699920

**Digital Object Identifier**

doi:10.1214/aoms/1177699920

**Mathematical Reviews number (MathSciNet)**

MR183073

**Zentralblatt MATH identifier**

0141.34401

**JSTOR**

links.jstor.org

#### Citation

Darroch, J. N. An Optimal Property of Principal Components. Ann. Math. Statist. 36 (1965), no. 5, 1579--1582. doi:10.1214/aoms/1177699920. https://projecteuclid.org/euclid.aoms/1177699920