Projection pursuit is concerned with "interesting" projections of high dimensional data sets, with finding such projections by machine, and with using them for nonparametric fitting and other data-analytic purposes. This survey attempts to put the fascinating problems and ramifications of projection pursuit--which range from principal components, multidimensional scaling, factor analysis, nonparametric regression, density estimation and deconvolution of time series to computer tomography and problems in pure mathematics--into a coherent perspective

Let $X$ be an $n \times k$ random matrix whose coordinates are independently normally distributed with common variance $\sigma^2$ and means given by $EX = e\mu' + \theta\lambda',$ where $e$ is the vector in $R^n$ having all coordinates equal to $1, \theta \in R^n,$ and $\mu, \lambda \in R^k$ with $\sum^k_{j = 1} \lambda^2_j = 1.$ The problem is to estimate $\lambda$, say by $\hat{\lambda},$ with loss function $1 - (\lambda'\hat{\lambda})^2$ when $\mu, \theta,$ and $\sigma^2$ are unknown. It is shown that the largest principal component of $X'X - (1/n)X'ee'X$ is the best estimator invariant under rotations in $R^k$ and rotations in $R^n$ leaving $e$ invariant and is admissible.

The paper deals with approximate ancillarity as discussed by Efron and Hinkley (1978). In the multivariate i.i.d. case we derive the second-order Edgeworth expansion of the MLE given a normalized version of the second derivative of the log-likelihood at its maximum. The expansion agrees with the one derived by Amari (1982a) for curved exponential families, but holds for any family satisfying the regularity conditions given in the paper. It is shown that the Fisher information lost by reducing the data to the MLE is recovered by the conditioning, and it is sketched how the loss of information relates to the deficiency as defined by LeCam. Finally, we investigate some properties of three test statistics, proving a conjecture by Efron and Hinkley (1978) concerning the conditional null-distribution of the likelihood ratio test statistic, and establishing a kind of superiority of the observed Fisher information over the expected one as estimate of the inverse variance of the MLE.

The problem of testing the underlying distribution in a general regression model is considered when the data are right censored. It is proposed that tests be based on efficient scores from a certain class of parametric alternatives. The connection between tests based on the scores and more traditional approaches using empirical distribution functions is explored. Neyman's smooth test is extended to allow for censored data.

Many population processes in demography, epidemiology and other fields can be represented by a time-continuous Markov chain model with a finite state space. If we have complete information on the life history of a cohort, the intensities of the Markov model may be estimated by the occurrence/exposure rates or by nonparametric techniques. In many situations, however, we have only incomplete information. In this paper we consider the special, but important, case where the occurrences and the total exposure are known, but not the distribution of the latter over the various separate statuses. Methods for handling such data, so-called demographic incidence rates, and methods for estimating the intensities from this kind of data, are known in the literature. However, their statistical properties are only vaguely known. The present paper gives a thorough presentation of the theory of these methods, and provide rigorous proofs of their statistical properties using stochastic process theory.

The problem of testing for a constant failure rate against alternatives with failure rates involving a single change-point is considered. The asymptotic significance level for tests based on maximal score statistics are shown to involve the solution to a first passage time problem for an Ornstein-Uhlenbeck process. An example illustrates the methodology.

A histogram type estimator of the hazard rate of lifetimes which are randomly right-censored is studied. This estimator is based on random spacings of the order statistics of uncensored observations and extends the idea of the histogram density estimation suggested by Van Ryzin (1973) to censored data. In this paper we establish pointwise large sample properties of the estimator including strong consistency, strong uniform consistency on a bounded interval, and two asymptotic distribution results. Of particular interest is the distribution result obtained by imposing extra "symmetry" conditions on the interval covering the given point. In fact, it yields the best attainable rate of convergence among all nonnegative estimators. Comparisons of our results with the kernel type estimators proposed in the literature are also given.

A new two-sample randomization test is proposed for testing that the joint distribution of two samples is invariant under permutations. The $p$-value of the test has a finite sample minimaxity property over neighborhoods of completely specified alternative distributions. Asymptotically, the test has minimax Bahadur slope against the neighborhoods, which remain fixed as the sample sizes increase. The proposed test also offers the best compromise between robustness against departures from a model alternative and optimality at the model alternative in the sense that no other test with the same gross-error-sensitivity has larger slope at the model. Some modifications of the test are proposed for testing the nonparametric null hypothesis against neighborhoods of models that have a shared nuisance location-scale parameter. These nuisance-parameter-free versions of the test are justified for large samples from exponential families, and an example of their use is given.

We consider a standard Bayes decision situation except that with small probability the data may be irrelevant to the parameter of interest (i.e. the experiment is "biased"). The minimax solution under quite general assumptions is described and discussed.

This paper considers the asymptotic minimax property of the sequential probability ratio test (SPRT) when the given distributions $P_{\pm \varepsilon}$ contain a small amount of contamination. Let $\mathscr{P}_{\pm \varepsilon}$ be the neighborhoods of $P_{\pm \varepsilon}.$ Suppose that $P_\varepsilon$ and $P_{-\varepsilon}$ approach each other as $\varepsilon \downarrow 0$ and that $\mathscr{P}_{\pm \varepsilon}$ shrink at an appropriate rate. We prove (under regularity assumptions) that the SPRT based on the least favorable pair of distributions $(Q^\ast_{-\varepsilon}, Q^\ast_\varepsilon)$ given by Huber (1965) is asymptotically least favorable for expected sample size and is asymptotically minimax, provided that the limiting maximum error probabilities do not exceed $1/2.$

Under an age replacement policy, a stochastically failing unit is replaced at failure or after being in service for $t$ units of time, whichever comes first. An important problem is the estimation of $\phi^\ast,$ the optimal replacement time when the form of the failure distribution is unknown. Here, $\phi^\ast$ is optimal in the sense that it is the replacement time that achieves the smallest long-run expected cost. It is shown that substantial cost savings can be effected by estimating $\phi^\ast$ sequentially. The sequential methodology employed here is stochastic approximation (SA). When suitably standardized, convergence in distribution of the SA estimator to $\phi^\ast$ is established. This gives precise information about the rate of convergence. A sequential methodology introduced by Bather (1977) has roughly the same aims as ours, but it is not of the SA type. Rates of convergence apparently have not been established for Bather's procedure.

An estimator $\hat{\beta}$ of $\beta$ is accurate with accuracy $A$ and confidence $\gamma, 0 < \gamma < 1,$ if $P(\hat{\beta} - \beta \in A) \geq \gamma$ for all $\beta.$ Given a sequence $Y_1, Y_2, \cdots$ of independent vector-valued homoscedastic normally-distributed random variables generated via the general linear model $Y_i = X_i\beta + \varepsilon,$ the $k$-dimensional parameter $\beta$ is accurately estimated using a sequential version of the maximum probability estimator developed by L. Weiss and J. Wolfowitz. The procedure given also generalizes C. Stein's fixed-width confidence sets to several dimensions.

Let $X_1, X_2, \cdots$ be i.i.d. random variables with mean $\theta$ and finite, positive variance $\sigma^2,$ depending on unknown parameters $\omega\in\Omega.$ The problem addressed is that of finding a stopping time $t$ for which the risk $R_A(t, \omega) = E_\omega\{A \gamma^2_0(\omega)(\bar{X}_t - \theta)^2 + t\}$ is as small as possible (in a suitable sense), where $A > 0, \gamma_0$ is a positive function on $\Omega$, and $\bar{X}_t = (X_1 + \cdots + X_t)/t.$ For fixed (nonrandom) sample sizes, $2 \sqrt{A}(\gamma_0\sigma)$ is a lower bound for $R_A(n, \omega), n \geq 1$; and the regret of a stopping time $t$ is defined to be $r_A(t, \omega) = R_A(t, \omega) - 2\sqrt{A}(\gamma_0 \sigma).$ The main results determine an asymptotic lower bound, as $A \rightarrow\infty,$ for the minimax regret $M_A(\Omega_0) = \inf_t\sup_{\omega\in\Omega_0}r_A(t, \omega)$ for neighborhoods $\Omega_0$ of arbitrary parameter points $\omega_0 \in \Omega.$ The bound is obtained for multiparameter exponential families and the nonparametric case. The bound is attained asymptotically by an intuitive procedure in several special cases.

Let $(X, Y)$ be a pair of random variables such that $X = (X_1, \cdots, X_J)$ and let $f$ by a function that depends on the joint distribution of $(X, Y).$ A variety of parametric and nonparametric models for $f$ are discussed in relation to flexibility, dimensionality, and interpretability. It is then supposed that each $X_j \in \lbrack 0, 1\rbrack,$ that $Y$ is real valued with mean $\mu$ and finite variance, and that $f$ is the regression function of $Y$ on $X.$ Let $f^\ast,$ of the form $f^\ast(x_1, \cdots, x_J) = \mu + f^\ast_1(x_1) + \cdots + f^\ast_J(x_J),$ be chosen subject to the constraints $Ef^\ast_j = 0$ for $1 \leq j \leq J$ to minimize $E\lbrack(f(X) - f^\ast(X))^2\rbrack.$ Then $f^\ast$ is the closest additive approximation to $f,$ and $f^\ast = f$ if $f$ itself is additive. Spline estimates of $f^\ast_j$ and its derivatives are considered based on a random sample from the distribution of $(X, Y).$ Under a common smoothness assumption on $f^\ast_j, 1 \leq j \leq J,$ and some mild auxiliary assumptions, these estimates achieve the same (optimal) rate of convergence for general $J$ as they do for $J = 1.$

Basic decision theory for discrete random variables of the multivariate geometric (power series) type is developed. Some properties of Bayes estimators that carry over in the limit to admissible estimators are obtained. A stepwise generalized Bayes representation of admissible estimators is developed with estimation of the mean of a multivariate Poisson random variable in mind. The development carries over to estimation of the mean of a multivariate negative Binomial random variable. Due to the natural boundary of the parameter space there is an interesting pathology illustrated to some extent by the examples given. Examples include one to show that admissible estimators with somewhere infinite risk do exist in two or more dimensions.

The classical problem of choice of number of classes in testing goodness of fit is considered for a class of alternatives, for the chi-square and likelihood ratio statistics. Pitman and Bahadur efficiencies are used to compare the two statistics and also to analyse the effect for each statistic of changing the number of classes for the case where the number of classes increases asymptotically with the number of observations. Overall, the results suggest that if the class of alternatives is suitably restricted the number of classes should not be very large.

It is shown that Hill's estimator (1975) for the exponent of regular variation is asymptotically normal if the number $k_n$ of extreme order statistics used to construct it tends to infinity appropriately with the sample size $n.$ As our main result, we derive a general condition which can be used to determine the optimal $k_n$ explicitly, provided that some prior knowledge is available on the underlying distribution function with regularly varying upper tail. This condition is simplified under appropriate assumptions and then applied to several examples.

$A$-optimal designs for comparing each of $\nu$ test treatments simultaneously with a control, in $b$ blocks of size $k$ each are considered. It is shown that several families of BIB designs in the test treatments augmented by $t$ replications of a control in each block are $A$-optimal. In particular these designs with $t = 1$ are optimal whenever $(k - 2)^2 + 1 \leq \nu \leq (k - 1)^2$ irrespective of the number of blocks. This includes BIB designs associated with finite projective and Euclidean geometries.

Let $(\mathscr{X, A})$ be a space with a $\sigma$-field, $M = \{P_s; s \in \Theta\}$ be a family of probability measures on $\mathscr{A}$ with $\Theta$ arbitrary, $X_1, \cdots, X_n$ i.i.d. observations on $P_\theta.$ Define $\mu_n(A) = (1/n) \sum^n_{i = 1} I_A(X_i),$ the empirical measure indexed by $A \in \mathscr{A}.$ Assume $\Theta$ is totally bounded when metrized by the $L_1$ distance between measures. Robust minimum distance estimators $\hat{\theta}_n$ are constructed for $\theta$ and the resulting rate of convergence is shown naturally to depend on an entropy function for $\Theta$.

The number of local maxima of the Cauchy location likelihood function which are not global maxima is asymptotically Poisson distributed with mean parameter $1/\pi.$

In a clinical comparison of responses to two treatments, patients are admitted sequentially and given one of the two treatments. The allocation is determined randomly, to decrease the possibility of personal bias in the selection of subjects for the test. To balance the assignments, the probability of receiving one treatment is a function of the proportion of patients previously assigned to that treatment. A local limit theorem for the distribution of the number of patients assigned to the first treatment is developed.

Consider a sequential probability ratio test comparing two treatments, where each subject receives only one of the treatments. Each subject's treatment assignment is determined by the flip of a biased coin, where the bias serves to balance the number of patients assigned to each treatment. The asymptotic properties of this test are studied, as the sample size approaches infinity. A renewal theorem is given for the joint distribution of the sample size, the imbalance in treatment assignment at the end of the experiment, and the excess over the stopping boundary. This theorem is used to calculate asymptotic expressions for the test's error probabilities.

The method of maximum likelihood leads to an ill-posed optimization problem in the case of a mixture of normal distributions. Estimation in the univariate case is reformulated using simple constraints into an optimization problem having a strongly consistent, global solution.

A framework is presented for deciding among functional forms when constructing confidence bands in simple linear regression. Using the concept of tautness, definitions of admissibility and completeness are developed. These lead to a characterization of a minimal complete class of band forms. A type of average width optimality within this class is briefly discussed.

Given a random sample from an unknown distribution $F,$ which is assumed to belong to some nonparametric family of distributions, consider the problem of estimating $\gamma(F),$ some function of $F.$ When the loss function is squared error, admissible estimators are exhibited for a large class of $\gamma$'s. A relationship between these estimators and similar ones in finite population sampling is demonstrated.

In a recent paper, Tanner (1983) proves pointwise consistency of a variable bandwidth kernel estimator for the hazard function. In the present note, a simplified proof of uniform consistency of a data-adaptive kernel estimator with certain additional advantages is given.

Using an argument developed in Siegmund (1982), we give a bound for the tail probability of Kolmogorov-Smirnov statistics in the following form $P(\inf_x(F_n(x) - F(x)) > \zeta) \leq 2\sqrt{2} e^{-2n\zeta^2}.$

In this paper we consider asymptotic approximations to the distribution function $F(x)$ of a linear combination of an estimate in a multivariate linear model. A method is given for obtaining an asymptotic expansion $F_{s - 1}(x)$ of $F(x)$ up to $O(n^{-s + 1})$ and a bound $c_s$ such that $|F(x) - F_{s - 1}(x)| \leq c_s$ uniformly in $x$ and $c_s = O(n^{-s})$.

We consider finite state gambling problems with the Dubins and Savage payoff and with the $\lim\inf$ payoff. For these models we show that the optimal return function with respect to all stationary strategies can be characterized similarly to the optimal return function. This enables us then to characterize those stationary strategies which are optimal within the set of all stationary strategies in the same way as it was done for optimal strategies by Dubins and Savage.