We study linear smoothers and their use in building nonparametric regression models. In the first part of this paper we examine certain aspects of linear smoothers for scatterplots; examples of these are the running-mean and running-line, kernel and cubic spline smoothers. The eigenvalue and singular value decompositions of the corresponding smoother matrix are used to describe qualitatively a smoother, and several other topics such as the number of degrees of freedom of a smoother are discussed. In the second part of the paper we describe how linear smoothers can be used to estimate the additive model, a powerful nonparametric regression model, using the "back-fitting algorithm." We show that backfitting is the Gauss-Seidel iterative method for solving a set of normal equations associated with the additive model. We provide conditions for consistency and nondegeneracy and prove convergence for the backfitting and related algorithms for a class of smoothers that includes cubic spline smoothers.

Given data $y_i = (Kg)(u_i) + \varepsilon_i$ where the $\varepsilon$'s are random errors, the $u$'s are known, $g$ is an unknown function in a reproducing kernel space with kernel $r$ and $K$ is a known integral operator, it is shown how to calculate convergence rates for the regularized solution of the equation as the evaluation points $\{u_i\}$ become dense in the interval of interest. These rates are shown to depend on the eigenvalue asymptotics of $KRK^\ast$, where $R$ is the integral operator with kernel $r$. The theory is applied to Abel's equation and the estimation of particle size densities in stereology. Rates of convergence of regularized histogram estimates of the particle size density are given.

We construct a tractable mathematical model for kernel-based projection pursuit regression approximation. The model permits computation of explicit formulae for bias and variance of estimators. It is shown that the bias of an orientation estimate dominates error about the mean--indeed, the latter is asymptotically negligible in comparison with bias. However, bias and error about the mean are of the same order in the case of projection pursuit curve estimates. Implications of our formulae for bias and variance are discussed.

We develop asymptotic theory for two polynomial-based methods of estimating orientation in projection pursuit density approximation. One of the techniques uses Legendre polynomials and has been proposed and implemented by Friedman [1]. The other employs Hermite functions. Issues of smoothing parameter choice and robustness are addressed. It is shown that each method can be used to construct $\sqrt n$-consistent estimates of the projection which maximizes distance from normality, although the former can only be employed in that manner when the underlying distribution has extremely light tails. The former can be used very generally to measure "low-frequency" departure from normality.

The desire to make nonparametric regression robust leads to the problem of conditional median function estimation. Under appropriate regularity conditions, a sequence of local median estimators can be chosen to achieve the optimal rate of convergence $n^{-1/(2+d)}$ both pointwise and in the $L^q (1 \leq q < \infty)$ norm restricted to a compact. It can also be chosen to achieve the optimal rate of convergence $(n^{-1} \log n)^{1/(2+d)}$ in the $L^\infty$ norm restricted to a compact. These results also constitute an answer to an open question of Stone.

In a recent paper of Gaffke an example was given regarding the $\phi_p$-optimal designs for the highest two coefficients in a one dimensional polynomial regression. The purpose of this paper is to supply a direct proof of this result using the theory of canonical moments and orthogonal polynomials.

When one uses the unbalanced, mixed linear model $\mathbf{y}_i = \mathbf{X}_i\mathbf{\alpha} + \mathbf{Z}_i\mathbf{\beta}_i + \varepsilon_i, i = 1, \cdots, n$ to analyze data from longitudinal experiments with continuous outcomes, it is customary to assume $\varepsilon_i \sim_{\operatorname{ind}} \mathscr{N}(\mathbf{0}, \sigma^2\mathbf{I}_i)$ independent of $\mathbf{\beta}_i \sim_{\operatorname{iid}} \mathscr{N}(\mathbf{0,\Delta})$, where $\sigma^2$ and the elements of an arbitrary $\mathbf{\Delta}$ are unknown variance and covariance components. In this paper, we describe a method for checking model adequacy and, in particular, the distributional assumption on the random effects $\mathbf{\beta}_i$. We generalize the weighted normal plot to accommodate dependent, nonidentically distributed observations subject to multiple random effects for each individual unit under study. One can detect various departures from the normality assumption by comparing the expected and empirical cumulative distribution functions of standardized linear combinations of estimated residuals for each of the individual units. Through application of distributional results for a certain class of estimators to our context, we adjust the estimated covariance of the empirical cumulative distribution function to account for estimation of unknown parameters. Several examples of our method demonstrate its usefulness in the analysis of longitudinal data.

Randomized experiments and observational studies both attempt to estimate the effects produced by a treatment, but in observational studies, subjects are not randomly assigned to treatment or control. A theory of observational studies would closely resemble the theory for randomized experiments in all but one critical respect: In observational studies, the distribution of treatment assignments is not known. The problems that are special to observational studies revolve around our uncertainty about how treatments were assigned. In this connection, tools are needed for describing distributions of treatment assignments that do not assign equal probabilities to all assignments. Two such tools are a lattice of treatment assignments first studied by Savage and an inequality due to Holley for probability distributions on a lattice. Using these tools, it is shown that certain permutation tests are unbiased as tests of the null hypothesis that the distribution of treatment assignments resembles a randomization distribution against the alternative hypothesis that subjects with higher responses are more likely to receive the treatment. In particular, these tests are unbiased against alternatives formulated in terms of a model previously used in connection with sensitivity analyses.

A directed graph with identified nodes is defined to represent a set of conditional independence (c.i.) statements. It is shown how new c.i. statements can be read from the graph of an influence diagram and results of Howard and Matheson are rigorised and generalized. A new decomposition theorem, analogous to Kiiveri, Speed and Carlin and requiring no positivity condition, is proved. Connections between influence diagrams and Markov field networks are made explicit. Because all results depend on only three properties of c.i., the theorems proved here can be restated as theorems about other structures like second order processes.

This article shows how to modify an approximate lack-of-fit test proposed by Neill and Johnson (1985) to obtain an exact $F$ test. The modified test is shown to be consistent and to yield uniformly most powerful invariant tests for two specific types of lack of fit. Pure types of lack of fit are (1) lack of fit that exists between clusters of near replicates and (2) lack of fit that is contained within clusters of near replicates. Lack of fit that is a mixture of these two types can be difficult or impossible to find depending on the nature of the mixture.

It is proved that $EX^2 < \infty$ is necessary for a very mild form of the bootstrap of the mean to work a.s. and that $X$ must be in the domain of attraction of the normal law if a.s. is weakened to "in probability."

Recent attention has focussed on possible improvements in performance of estimators which might flow from using the smoothed bootstrap. We point out that in a great many problems, such as those involving functions of vector means, any such improvements will be only second-order effects. However, we argue that substantial and significant improvements can occur in problems where local properties of underlying distributions play a decisive role. This situation often occurs in estimating the variance of an estimator defined in an $L^1$ setting; we illustrate in the special case of the variance of a quantile estimator. There we show that smoothing appropriately can improve estimator convergence rate from $n^{-1/4}$ for the unsmoothed bootstrap to $n^{-(1/2) + \varepsilon}$, for arbitrary $\varepsilon > 0$. We provide a concise description of the smoothing parameter which optimizes the convergence rate.

It is shown that the Bayesian bootstrap approximation to the posterior distribution of the unknown mean (with respect to a Dirichlet process prior) is more accurate than both the standard normal approximation and the bootstrap approximation. It is also shown that the Bayesian bootstrap approximation to the sampling distribution of the sample average is not as accurate as the bootstrap approximation.

An investigation is carried out in the behavior of the determinants of certain moment matrices, for which the $(i, j)$ entry is the $(i + j)$th moment of a distribution $F$. The determinant can be represented as the expected value of a $U$-statistic type kernel. The structure of the kernel illustrates how the determinant carries information about the number of support points of the distribution $F$. The kernel representation can be extended to the determinant of a matrix of moment generating function derivatives, where the $(i, j)$ entry is the $i + j$th derivative of the moment generating function of $F$. When done, this reveals that this determinant is itself, as a function of $t$, a moment generating function. When this somewhat surprising result is applied to members of the quadratic variance exponential family, one obtains the result that they are closed under this two-step operation of taking derivatives, then computing determinants. This results in an elementary recursion for the values of the moment determinants. The final result gives the convergence of the moment determinants to the normal theory values under central limit theorem conditions.

The use of moment matrices and their determinants are shown to elucidate the structure of mixture estimation as carried out using the method of moments. The setting is the estimation of a discrete finite support point mixing distribution. In the important class of quadratic variance exponential families it is shown for any sample there is an integer $\hat{\nu}$ depending on the data which represents the maximal number of support points that one can put in the estimated mixing distribution. From this analysis one can derive an asymptotically normal statistic for testing the true number of points in the mixing distribution. In addition, one can construct consistent nonparametric estimates of the mixing distribution for the case when the number of points is unknown or even infinite. The normal model is then examined in more detail, and in particular the case when $\sigma^2$ is unknown is given a comprehensive solution. It is shown how to estimate the parameters in a direct way for every hypothesized number of support points in the mixing distribution, and it is shown how the structure of the problem yields a decomposition of variance into model and error components very similar to the traditional analysis of variance.

Exponential dispersion models play an important role in the context of generalized linear models, where error distributions, other than the normal, are considered. Any statistical model expressible in terms of a variance-mean relation $(V, \Omega)$ leads to an exponential dispersion model provided that $(V, \Omega)$ is a variance function of a natural exponential family: Here $\Omega$ is the domain of means and $V$ is the variance function of the natural exponential family. Therefore, it is of a particular interest to examine whether a pair $(V, \Omega)$ can serve as the variance function of a natural exponential family. In this study we consider the case where $\Omega$ is bounded and examine whether $V$ can be the restriction to $\Omega$ of a rational function vanishing at the boundary points of $\Omega$. The class of such functions is large and contains the important subclass of polynomials. It is shown that, apart from the binomial family (possessing a quadratic variance function) and affine transformations thereof, there exists no natural exponential family with variance function belonging to this class. Such a result implies, in particular, that the only variance functions of natural exponential families among polynomials of at least third degree are those restricted to unbounded domains $\Omega$.

To assess the dependence structure in a stationary bivariate point process the second-order distribution can be very useful. We prove that the natural estimates of this distribution, based on a realization $A_1 < A_2 < \cdots < A_{n_A}, B_1 < B_2 < \cdots < B_{n_B}$ are asymptotically normal and we present a method for constructing approximate confidence intervals for this distribution.

Let $F$ denote the life distribution of a coherent structure of independent components. Suppose that we have a sample of independent systems, all having the same structure. Each system is continuously observed until it fails. For every component in each system, either a failure time or a censoring time is recorded. A failure time is recorded if the component fails before or at the time of system failure; otherwise a censoring time is recorded. We introduce a method for finding estimates for $F(t)$, quantiles and other functionals of $F$, based on the censorship of the component lives by system failure. We use the theory of counting processes and stochastic integrals to obtain limit theorems that enable the construction of confidence intervals for large samples. Our approach extends and gives a novel application of censoring methodology.

Methods are described which permit one to work with continuous-time optimal stopping problems, using the heat equation, even when the prior placed on the drift parameter of a Wiener process is not normal. The details of the method are worked out for Chernoff's problem of testing the sign of the drift parameter when the prior is "smooth."

We consider dynamic procedures for sampling from a process (a Brownian motion) and stopping it after a change is detected. The basic idea is to conduct a sequence of similar SPRT's, each one of them done in negligible time, while not sampling at all between them. The procedures detect the change point much faster than the standard procedures with the same sampling rate and time to false alarm, but hold the sampling rate constant.

Motivated by a scheduling problem arising from serial sacrifice experiments, the asymptotic efficiency of irreversible adaptive allocation rules is studied. The asymptotic lower bound for the regret of an adaptive allocation rule is characterized by the minimum of a linear program. Based on a class of one-sided sequential tests, asymptotically efficient rules which achieve the lower bound are constructed. The conditions necessary for this construction are verified in the serial sacrifice scheduling problem.

Suppose $\{X_n\}$ is a $p$th order autoregressive process with innovations in the domain of attraction of a stable law and the true order $p$ unknown. The estimate $\hat{p}$ of $p$ is chosen to minimize Akaike's information criterion over the integers $0, 1, \cdots, K$. It is shown that $\hat{p}$ is weakly consistent and the consistency is retained if $K \rightarrow \infty$ as $N \rightarrow \infty$ at a certain rate depending on the index of the stable law.

The aim of this article is to investigate the asymptotic behaviour of estimators of the optimal value and optimal solutions of a stochastic program. These estimators are closely related to the $M$-estimators introduced by Huber (1964). The parameter set of feasible solutions is supposed to be defined by a number of equality and inequality constraints. It will be shown that in the presence of inequality constraints the estimators are not asymptotically normal in general. Maximum likelihood and robust regression methods will be discussed as examples.

Arbitrary, possibly randomized estimators of a one-dimensional parameter are considered. It is shown that suitable averages of their distribution functions are more spread out than particular distribution functions, which are defined in terms of the weight functions by which the averages are taken over the parameter space and in terms of the family of distributions for the random quantity on which the estimators are based. In this way bounds are provided for the performance of arbitrary estimators of the parameter. As consequences of this nonasymptotic spread inequality, which will be proved under mild regularity conditions, a local asymptotic minimax inequality and a generalization of the classical results on superefficiency can be derived, thus showing the strength of our spread inequality.

We consider the problem of robustness or sensitivity of given Bayesian posterior criteria to specification of the prior distribution. Criteria considered include the posterior mean, variance and probability of a set (for credible regions and hypothesis testing). Uncertainty in an elicited prior, $\pi_0$, is modelled by an $\varepsilon$-contamination class $\Gamma = \{\pi = (1 - \varepsilon)\pi_0 + \varepsilon q, q \in Q\}$, where $\varepsilon$ reflects the amount of probabilistic uncertainty in $\pi_0$, and $Q$ is a class of allowable contaminations. For $Q = \{$all unimodal distributions$\}$ and $Q = \{\text{all symmetric unimodal distributions}\}$, we determine the ranges of the various posterior criteria as $\pi$ varies over $\Gamma$.

In estimation of a $p$-variate normal mean with identity covariance matrix, Stein-type estimators can offer significant gains over the $\operatorname{mle}$ in terms of risk with respect to sum of squares error loss. Their maximum risk is still equal to $p$, however, which will typically be their "reported loss." In this paper we consider use of data-dependent "loss estimators." Two conditions that are attractive for such a loss estimator are that it be an improved loss estimator under some scoring rule and that it have a type of frequentist validity. Loss estimators with these properties are found for several of the most important Stein-type estimators. One such estimator is a generalized Bayes estimator, and the corresponding loss estimator is its posterior expected loss. Thus Bayesians and frequentists can potentially agree on the analysis of this problem.

Conditions are given for the formal posterior of an improper prior to be coherent and applied to translation models. An example is given of a proper countably additive statistical model and a finitely additive prior for which there is no posterior.

This paper reformulates a result of Hora and Buehler on best equivariant estimators to treat a model admitting an ancillary statistic. The approach itself was established by Pitman, Girshick and Savage and Kiefer, and expanded by Zidek. The model considered in this paper is assumed to be generated as an orbit under a group acting on the parameter space. The general result obtained here is applied to a model in the Nile problem, a model with a known variation coefficient, a circle model and the GMANOVA model, and best equivariant estimators (BEE's) are derived. In the first two models, the BEE's dominate the MLE's uniformly.

Certain polynomials of a skew-symmetric matrix are considered. These polynomials can be expressed in terms of the zonal polynomials on the Hermitian matrices, and they are used to obtain a series expansion for the density of the non-null distribution of the maximal invariant corresponding to the problem of testing for reality of the covariance matrix of a complex multivariate normal distribution.

Consider the problem of estimating the ordered means $\mu_1 \leq \mu_2 \leq \cdots \leq \mu_k$ of independent normal random variables, $Y_1, Y_2, \cdots, Y_k$. It is shown that the absolute error of each component $\hat{\mu}_i$ of the isotonic regression estimator is stochastically smaller than that of the usual estimator $Y_i$. Thus $\hat{\mu}_i$ is superior to $Y_i$ under any nonconstant loss which is a nondecreasing function of absolute error.

This note concerns the problem of order determination for autoregressive models. Rissanen's "Predictive least squares principle" prescribes that one should choose as order estimate $\hat{k}(n)$ at time $n$ the order of the model which has given the least mean square prediction error up to that time. We show that this procedure is strongly consistent, that is, that $\hat{k}(n) \rightarrow p$ a.s. as $n \rightarrow \infty$ when the data are generated by an AR process of order $p$, given an upper bound $p^\ast$.