This paper on estimating linear statistical relationships includes three lectures on linear functional and structural relationships, factor analysis, and simultaneous equations models. The emphasis is on relating the several models by a general approach and on the similarity of maximum likelihood estimators (under normality) in the different models. In the first two lectures the observable vector is decomposed into a "systematic part" and a random error; the systematic part satisfies the linear relationships. Estimators are derived for several cases and some of their properties given. Estimation of the coefficients of a single equation in a simultaneous equations model is shown to be equivalent to estimation of linear functional relationships.

The impact of survey design on standard multinomial-based methods for a multiway contingency table is studied, under nested loglinear models. The asymptotic null distribution of the Pearson chi-squared test statistic, $X^2$ (or the likelihood ratio test statistic, $G^2$) is obtained as a weighted sum of independent $\chi^2_1$ random variables, and the weights are then related to the familiar design effects (deffs) used by survey samplers. A simple correction to $X^2$ (or $G^2$) is also obtained which requires the knowledge of only the cell deffs and the deffs for collapsed tables (marginals), whenever the model admits a direct solution of likelihood equations under multinomial sampling. Finally, an example on the relative performance of $X^2$ and some corrected $X^2$ statistics in a three-way table is given, using some data from the Canada Health Survey, 1978-1979.

In this paper we present a first general proof of Tukey's (1953) conjecture concerning the extension of the Tukey multiple comparisons procedure to the case of unequal sample sizes, thereby proving that the so-called Tukey-Kramer procedure is conservative in all cases. Also a brief history of the conjecture is given and some extensions and further problems concerning the procedure are discussed.

A class of statistics generalizing $U$-statistics and $L$-statistics, and containing other varieties of statistic as well, such as trimmed $U$-statistics, is studied. Using the differentiable statistical function approach, differential approximations are obtained and the influence curves of these generalized $L$-statistics are derived. These results are employed to establish asymptotic normality for such statistics. Parallel generalizations of $M$- and $R$-statistics are noted. Strong convergence, Berry-Esseen rates, and computational aspects are discussed.

This paper focuses on two aspects of the errors in variables problem--variance estimation of the classical estimators of slope and intercept, and the detection of influential observations. The behaviour of the jackknife, bootstrap, normal theory and influence function estimators of variability is examined under a number of sampling situations by Monte Carlo methods. In the multivariate case, perturbation analysis is used to calculate the influence function of the estimator of Gleser (1981). The connection to estimation in linear regression models is discussed. The role of the influence function in the detection of influential observations is considered and an illustration is given by a numerical example.

Let $\hat{T}_n$ be an estimate of the form $\hat{T}_n = T(\hat{F}_n)$, where $\hat{F}_n$ is the sample $\operatorname{cdf}$ of $n \operatorname{iid}$ observations and $T$ is a locally quadratic functional defined on $\operatorname{cdf's}$. Then, the normalized jackknife estimates for bias, skewness, and variance of $\hat{T}_n$ approximate closely their bootstrap counterparts. Each of these estimates is consistent. Moreover, the jackknife and bootstrap estimates of variance are asymptotically normal and asymptotically minimax. The main results: the first-order Edgeworth expansion estimate for the distribution of $n^{1/2}(\hat{T}_n - T(F))$, with $F$ being the actual $\operatorname{cdf}$ of each observation and the expansion coefficients being estimated by jackknifing, is asymptotically equivalent to the corresponding bootstrap distribution estimate, up to and including terms of order $n^{-1/2}$. Both distribution estimates are asymptotically minimax. The jackknife Edgeworth expansion estimate suggests useful corrections for skewness and bias to upper and lower confidence bounds for $T(F)$.

The finite sample breakdown properties of $M$-estimators, defined by $\sum\rho(x_i - T) = \min!$, and of the associated Pitman-type or $P$-estimators, defined by $T = \frac{\int \exp\{-\Sigma \rho(x_i - \theta)\}\theta d\theta}{\int \exp\{-\Sigma\rho(x_i - \theta)\} d\theta},$ are investigated. If $\rho$ is symmetric, and $\psi = \rho'$ is monotone and bounded, then the breakdown point of either estimator is $\varepsilon^\ast = 1/2$. If $\psi$ decreases to 0 for large $x$ ("redescending estimators"), the same result remains true if $\rho$ is unbounded. For bounded $\rho$, the $P$-estimator is undefined, and the breakdown point of the $M$-estimator typically is slightly less than $1/2$; it is calculated in explicit form.

Huber (1981b) and Li and Chen (1981) have proposed the use of projection pursuit techniques to construct and estimate measures of location and dispersion for multivariate distributions. In this paper we show that such measures do not ordinarily commute with affine coordinate reexpressions unless the distribution is centrosymmetric (location case) or elliptically contoured (dispersion case).

A class of semi-Markov models, those which have proportional hazards and which are forward-going (if state $j$ can be reached from $i$, then $i$ cannot be reached from $j$), are shown to fit into the multiplicative intensity model of counting processes after suitable random time changes. Standard large-sample results for counting processes following this multiplicative model can therefore be used to make inferences on the above class of semi-Markov models, including the case where observations may be censored. Large-sample results for a four-state model used in clinical trials are presented.

A conditional limit theorem is derived for a certain class of stochastic processes whose distributions constitute a nonergodic family. The limit theorem allows us to study the asymptotic behaviour under the conditional model of some standard statistical procedures by making use of results for ergodic families. Explosive Gaussian autoregressive processes are studied in some detail. Here the conditional process is shown to be a nonexplosive Gaussian autoregression bearing a simple relation to the original process. Some optimality results under the conditional model are given for estimators and tests based on the unconditional likelihood.

We solve here the general nonstationary multivariate MA spectral factorization problem, i.e. the problem of obtaining all the possible MA models (with time-dependent coefficients) corresponding to a given (time-dependent) autocovariance function. Our result (Theorem 8) relies on a symbolic generalization (Theorem 1) of the classical factorization property of the characteristic polynomial associated with stationary autocovariance functions, and is obtained by means of a matrix extension of ordinary continued fractions. We also give necessary and sufficient conditions for an autocovariance function to be an MA autocovariance function and for a process to be an MA one (Theorems 6 and 7).

Two statistics are proposed for testing the homogeneity of signals, when these are noisy and repeatedly recorded. The distribution of the statistics is well approximated by the $\chi^2$ and the normal distribution, respectively, as shown by theoretical and simulation results. Power and insensitivity to other types of variation are discussed. The approach is developed in the context of analysing recordings of electric brain potentials being evoked by some stimulation, as traditionally investigated in neurophysiology and psychophysiology. In particular, applications to visual potentials evoked by light flashes of a group of 41 children are given.

In recent years, nonparametric curve estimates have been extensively explored in theoretical work. There has, however, been a certain lack of convincing applications, in particular involving comparisons with parametric techniques. The present investigation deals with the analysis of human height growth, where longitudinal measurements were collected for a sample of boys and a sample of girls. Evidence is presented that kernel estimates of acceleration and velocity of height, and of height itself, might offer advantages over a parametric fitting via functional models recently introduced. For the specific problem considered, both approaches are biased, but the parametric one shows qualitative and quantitative distortion which both are not easily predictable. Data-analytic problems involved with kernel estimation concern the choice of kernels, the choice of the smoothing parameter, and also whether the smoothing parameter should be chosen uniformly for all subjects or individually.

Under suitable conditions, we show that the cross-validated nearest neighbor estimates for the unknown smooth regression function in $R^p$ is asymptotically consistent.

Weak laws of large numbers and central limit theorems are proved for integrated square error of kernel estimators of regression functions. The regressor is assumed to take values in $\mathbb{R}^p$, and the regressand, $X$, to be real valued. It is shown that in many cases, integrated square error is asymptotically normally distributed and independent of the $X$-sample. As an application, a test for the regression function (such as that proposed by Konakov) is seen to be asymptotically independent of an arbitrary test based on the $X$-sample. The proofs involve martingale methods.

In this paper the asymptotic relative deficiency of the sample $q$-quantile with respect to kernel type estimators of the $q$-quantile is evaluated. The comparison is based on the mean square errors of the estimators. The result suggests a purely analytic measure of performance within the class of kernels. It is notable that a similar situation occurs when kernel estimators of a distribution function are studied.

We consider the nearly linear regression problem when the assumed first degree model is contaminated by some small constant, and the design space consists of finitely many points, symmetrically distributed on the interval $\lbrack -1/2, 1/2\rbrack$. Under the usual squared loss function for the estimation of the slope and the intercept, and with the use of the least squares estimators, the problem is to find the designs which are optimal in the sense of minimizing the maximum risk among symmetric designs. The results turn out to be quite different from those obtained by Li and Notz (1981) in a setup that is similar except that the design space is the whole interval $\lbrack -1/2, 1/2\rbrack$. In many cases the optimal solution has a support containing more than two points.

Optimal Bayesian experimental designs for estimation and prediction in linear models are discussed. The designs are optimal for estimating a linear combination of the regression parameters $\mathbf{c}^T\theta$ or prediction at a point where the expected response is $\mathbf{c}^T\mathbf{\theta}$ under squared error loss. A distribution on $\mathbf{c}$ is introduced to represent the interest in particular linear combinations of the parameters. In the usual notation for linear models minimizing the preposterior expected loss leads to minimizing the quantity $\mathrm{tr}\psi(R + XX^T)^{-1}$. The matrix $\psi$ is defined to be $E(\mathbf{cc}^T)$ and the matrix $R$ is the prior precision matrix of $\theta$. A geometric interpretation of the optimal designs is given which leads to a parallel of Elfving's theorem for $\mathbf{c}$-optimality. A bound is given for the minimum number of points at which it is necessary to take observations. Some examples of optimal Bayesian designs are given and optimal designs for prediction in polynomial regression are derived. The optimality of rounding non-integer designs to integer designs is discussed.

Let $\mathbf{X}_1, \mathbf{X}_2,\cdots$ be independent $p$-variate normal vectors with $E \mathbf{X}_\alpha \equiv \beta \mathbf{Y}_\alpha, \alpha = 1,2,\cdots$ and same p.d. dispersion matrix $\Sigma$. Here $\beta: p \times q$ and $\Sigma$ are unknown parameters and $\mathbf{Y}_\alpha$'s are known $q \times 1$ vectors. Writing $\beta = (\beta'_1 \beta'_2)' = (\beta_{(1)}\beta_{(2)})$ with $\beta_i: p_i \times q(p_1 + p_2 = p)$ and $\beta_{(i)}: p \times q_i(q_1 + q_2 = q)$, we have constructed invariant confidence sequences for (i) $\beta$, (ii) $\beta_{(1)}$, (iii) $\beta_1$ when $\beta_2 = 0$ and (iv) $\sigma^2 = |\Sigma|$. This uses the basic ideas of Robbins (1970) and generalizes some of his and Lai's (1976) results. In the process alternative simpler solutions of some of Khan's results (1978) are obtained.

In the location estimation problem, translation equivariant estimators are considered. It is shown that under a mild regularity condition the distribution of such estimators is more spread out than a particular distribution which is defined in terms of the sample size and the density of the i.i.d. observations. Some consequences of this so-called spread-inequality are discussed, namely the Cramer-Rao inequality, an asymptotic minimax inequality and the efficiency of the maximum likelihood estimator in some nonregular cases.

Simultaneous estimation of $p(p \geq 3)$ location parameters are considered under quadratic loss. Explicit estimators which dominate the best invariant one are given mainly when coordinates of the best invariant one are independently, identically and symmetrically distributed. Effectiveness of integration by parts in evaluating the risk function of the dominating estimator is shown for three typical continuous distributions (uniform, double exponential and $t$). Further explicit dominating estimators are given in terms of second and fourth moments of the best invariant one.

Let $\pi_1,\cdots, \pi_k$ be given populations which are associated with unknown real parameters $\theta_1,\cdots, \theta_k$ from a common underlying exponential family $\mathscr{F}$. Permutation invariant sequential selection procedures are considered to find good populations (i.e. those which have large parameters), where inferior populations are intended to be screened out at the earlier stages. The natural terminal decisions, i.e. decisions which are made in terms of largest sufficient statistics, are shown to be optimum in terms of the risk, uniformly in $(\theta_1,\cdots, \theta_k)$, under fairly general loss assumptions. Similar results with respect to subset selections within stages are established under the additional assumption that $\mathscr{F}$ is strongly unimodal (i.e. $\log$-concave). The results are derived in the Bayes approach under symmetric priors. Backward induction as well as the concept of decrease in transposition (DT) by Hollander, Proschan and Sethuraman (1977) are the main tools which are used in the proofs.

Given a positive, normalized kernel and a finite measure on an Euclidean space, we construct a random density by convoluting the kernel with the Dirichlet random probability indexed by the finite measure. The posterior distribution of the random density given a sample is classified. The Bayes estimator of the density function is given.

For two linear experiments $d_1 = L(X_1\beta, V_1)$ and $d_2 = L(X_2\beta, V_2)$ where the covariances $V_1$ and $V_2$ are known and can be singular or nonsingular, we characterize the following relations: $d_1$ at least as good as $d_2, d_1$ better than $d_2$, and $d_1$ equivalent to $d_2$. Sometimes only a subset of parameters is of interest to the experimenter. We extend the above relations between $d_1$ and $d_2$ to estimation of a common subset of parameters and give analogous characterizations. Three examples are given.

In the theory of competing risks, the nonparametric Kaplan-Meier estimator plays an important role. In this paper, a bivariate nonparametric estimator for the competing risk problem is given, which for the special case of independent causes gives the Kaplan-Meier estimator. This paper also introduces a matrix $w$, through which dependent models for the competing problem can be studied. These results also indicate the special case on the matrix $w$ for which the bounds to the survival function given by Peterson [4] can be obtained.

Three approaches to a nonparametric maximum likelihood problem are considered. One, based on the method of "sieves", is shown to include the other two. The sieve considered is a double exponential convolution sieve. A closed form solution is given for certain values of the sieve parameter.

The problem of sequentially testing two simple hypotheses for a stochastic process is considered. It is shown that, for arbitrary distributions $P_0$ and $P_1$, the following optimality holds for an SPRT which stops on its boundaries: If $\alpha$ and $\beta$ represent the error probabilities of the SPRT and a competing test has error probabilities $\alpha' \leq \alpha$ and $\beta' \leq \beta$ then $E_0g(D_{\tau'}) \geq E_0g(D_\tau)$ for any convex function $g$ satisfying some minor requirement, provided $P_1(\tau' < \infty) = 1$ for the competing test. Here $D_\tau$ and $D_{\tau'}$ denote the terminal likelihood ratios under the SPRT and the competitor. An analogous statement holds for expectation under $P_1$, and several applications of this optimality result are given.

Let $\mathbf{X} = (X_1,\cdots, X_n)$ have a density $g(\mathbf{x}, \lambda)$ which is decreasing in transposition, where $\lambda = (\lambda_1,\cdots, \lambda_n)$. Suppose one wishes to select a subset of $\{1,\cdots, n\}$ containing the subscripts associated with the largest values of the $\lambda_i$'s. Let $S$ be a permutation invariant selection rule which is more likely to select a subset associated with the largest values of the $X_i$'s. Let $A = \{i(1),\cdots, i(k)\} \subset \{1,\cdots, n\}$ and $B = \{j(1),\cdots, j(k)\} \subset \{1,\cdots, n\}$ be such that $\lambda_{i(s)} \geq \lambda_{j(s)}, s = 1,\cdots, k$. The following five inequalities are proved for nonrandomized selection rules. $(|D|$ denotes the number of elements in $D$. $D^c$ denotes the complement of $D$.) $P_\lambda(|S \cap A| \geq (>)m) \geq P(|S \cap B| \geq (>)m)$ for every $m \in R^1, P_\lambda(|S \cap A^c| \leq (<)m) \geq P_\lambda(|S \cap B^c| \leq (<)m)$ for every $m \in R^1$, and $P_\lambda(S = A) \geq P_\lambda(S = B)$. Inequalities for randomized selection rules are also obtained. These generalized monotonicity properties are derived using a unified approach. The results apply to selection rules proposed under several formulations of the selection problem.