Variable window width kernel density estimators, with the width varying proportionally to the square root of the density, have been thought to have superior asymptotic properties. The rate of convergence has been claimed to be as good as those typical for higher-order kernels, which makes the variable width estimators more attractive because no adjustment is needed to handle the negativity usually entailed by the latter. However, in a recent paper, Terrell and Scott show that these results can fail in important cases. In this paper, we characterize situations where the fast rate is valid, and also give rates for a variety of cases where they are slower. In addition, a modification of the usual variable window width estimator is proposed, which does have the earlier claimed rates of convergence.

Let $\varphi$ be a smooth function of $k + 2$ variables. We shall investigate in this paper the rates of convergence of estimators of $T(f) = \int\varphi(f(x), f'(x), \ldots, f^{(k)}(x), x) dx$ when $f$ belongs to some class of densities of smoothness $s$. We prove that, when $s \geq 2k + \frac{1}{4}$, one can define an estimator $\hat{T}_n$ of $T(f)$, based on $n$ i.i.d. observations of density $f$ on the real line, which converges at the semiparametric rate $1/ \sqrt n$. On the other hand, when $s < 2k + \frac{1}{4}, T(f)$ cannot be estimated at a rate faster than $n^{-\gamma}$ with $\gamma = 4(s - k)/\lbrack 4s + 1\rbrack$. We shall also provide some extensions to the multidimensional case. Those results extend previous works of Levit, of Bickel and Ritov and of Donoho and Nussbaum on estimation of quadratic functionals.

In the usual linear regression model we investigate the geometric structure of a class of minimax optimality criteria containing Elfving's minimax and Kiefer's $\phi_p$-criteria as special cases. It is shown that the optimal designs with respect to these criteria are also optimal for $A'\theta$, where $A$ is any inball vector (in an appropriate norm) of a generalized Elfving set. The results explain the particular role of the $A$- and $E$-optimality criterion and are applied for determining the optimal design with respect to Elfving's minimax criterion in polynomial regression up to degree 9.

In 1987 Cheng determined $\phi_p$-optimal designs for linear regression (without intercept) over the $n$-dimensional unit cube $\lbrack 0, 1\rbrack^n$ for $-\infty \leq p \leq 1$. These are uniform distributions on the vertices with a fixed number of entries equal to unity, and mixtures of neighboring such designs. In 1989 Pukelsheim showed that this class of designs is essentially complete and that the corresponding class of moment matrices is minimally complete, with respect to what he called Kiefer ordering. In this paper, these results are extended to general permutation-invariant design regions.

Let $\mathbf{X}$ have a multivariate, $p$-dimensional normal distribution $(p \geq 2)$ with unknown mean $\mathbf{\mu}$ and known, nonsingular covariance $\mathbf{\Sigma}$. Consider testing $H_0: \mathbf{b}'_i\mathbf{\mu} \leq 0$, for some $i = 1, \ldots, k$, versus $H_1: \mathbf{b}'_i\mathbf{\mu} > 0$, for all $i = 1, \ldots, k$, where $\mathbf{b}_1, \ldots, \mathbf{b}_k, k \geq 2$, are known vectors that define the hypotheses. For any $0 < \alpha < 1/2$, we construct a size-$\alpha$ test that is uniformly more powerful than the size-$\alpha$ likelihood ratio test (LRT). The proposed test is an intersection-union test. Other authors have presented uniformly more powerful tests under restrictions on the covariance matrix and on the hypothesis being tested. Our new test is uniformly more powerful than the LRT for all known nonsingular covariance matrices and all hypotheses. So our results show that, in a very general class of problems, the LRT can be uniformly dominated.

Fisher's linear discriminant analysis (LDA) is a popular data-analytic tool for studying the relationship between a set of predictors and a categorical response. In this paper we describe a penalized version of LDA. It is designed for situations in which there are many highly correlated predictors, such as those obtained by discretizing a function, or the grey-scale values of the pixels in a series of images. In cases such as these it is natural, efficient and sometimes essential to impose a spatial smoothness constraint on the coefficients, both for improved prediction performance and interpretability. We cast the classification problem into a regression framework via optimal scoring. Using this, our proposal facilitates the use of any penalized regression technique in the classification setting. The technique is illustrated with examples in speech recognition and handwritten character recognition.

The $k$ principal points of a $p$-variate random vector $\mathbf{X}$ are those points $\xi_1, \ldots, \xi_k \in \mathbb{R}^p$ which approximate the distribution of $\mathbf{X}$ by minimizing the expected squared distance of $\mathbf{X}$ from the nearest of the $\xi_j$. Any set of $k$ points $\mathbf{y}_1, \ldots, \mathbf{y}_k$ partitions $\mathbb{R}^p$ into "domains of attraction" $D_1, \ldots, D_k$ according to minimal distance; following Hastie and Stuetzle we call $\mathbf{y}_1, \ldots, \mathbf{y}_k$ self-consistent if $E\lbrack\mathbf{X}\mid\mathbf{X} \in D_j\rbrack = \mathbf{y}_j$ for $j = 1, \ldots, k$. Principal points are a special case of self-consistent points. In this paper we study principal points and self-consistent points of $p$-variate elliptical distributions. The main results are the following: (1) If $k$ self-consistent points of $\mathbf{X}$ span a subspace of dimension $q < p$, then this subspace is also spanned by $q$ principal components, that is, self-consistent points of elliptical distributions exist only in principal component subspaces. (2) The subspace spanned by $k$ principal points of $\mathbf{X}$ is identical with the subspace spanned by the principal components associated with the largest roots. This proves a conjecture of Flury. We also discuss implications of our results for the computation and estimation of principal points.

We consider Bahadur-Kiefer representations for smoothed quantile processes. We prove that the asymptotics of the distance between smoothed empirical and quantile processes can be completely different from that of the unsmoothed ones. We obtain a complete characterization of the possible limits.

Suppose we observe a uniformly ergodic Markov chain with unknown transition distribution. The empirical estimator for a linear functional of the (invariant) joint distribution of two successive observations is defined using the pairs of successive observations. Its efficiency is proved using a martingale approximation. As a corollary we show efficiency of the empirical joint distribution function in the sense of a functional convolution theorem.

Robillard's approach to obtaining an expression for the cumulant generating function of the null distribution of Kendall's $S$-statistic, when one ranking is tied, is extended to the general case where both rankings are tied. An expression is obtained for the cumulant generating function and it is used to provide a direct proof of the asymptotic normality of the standardized score, $S/ \sqrt{\operatorname{Var}(S)}$, when both rankings are tied. The third cumulant of $S$ is derived and an expression for exact evaluation of the fourth cumulant is given. Significance testing in the general case of tied rankings via a Pearson type I curve and an Edgeworth approximation to the null distribution of $S$ is investigated and compared with results obtained under the standard normal approximation as well as the exact distribution obtained by enumeration.

We consider a semiparametric estimation method for general regression models when some of the predictors are measured with error. The technique relies on a kernel regression of the "true" covariate on all the observed covariates and surrogates. This requires a nonparametric regression in as many dimensions as there are covariates and surrogates. The usual theory copes with such higher-dimensional problems by using higher-order kernels, but this is unrealistic for most problems. We show that the usual theory is essentially as good as one can do with this technique. Instead of regression with higher-order kernels, we propose the use of dimension reduction techniques. We assume that the "true" covariate depends only on a linear combination of the observed covariates and surrogates. If this linear combination were known, we could apply the one-dimensional versions of the semiparametric problem, for which standard kernels are applicable. We show that if one can estimate the linear directions at the root-$n$ rate, then asymptotically the resulting estimator of the parameters in the main regression model behaves as if the linear combination were known. Simulations lend some credence to the asymptotic results.

The frailty model is a generalization of Cox's proportional hazards model which includes a random effect. Nielsen, Gill, Andersen and Sorensen (1992) proposed an EM algorithm to estimate the cumulative baseline hazard and the variance of the random effect. Here the asymptotic distribution of the estimators is given along with a consistent estimator of the asymptotic variance.

We consider the problem of estimating a pdf $f$ from samples $X_1, X_2, \ldots, X_n$ of a random variable with pdf $\mathscr{K}f$, where $\mathscr{K}$ is a compact integral operator. We employ a maximum smoothed likelihood formalism inspired by a nonlinearly smoothed version of the EMS algorithm of Silverman, Jones, Wilson and Nychka. We show that this nonlinearly smoothed algorithm is itself an EM algorithm, which helps explain the strong convergence properties of the algorithm. For the case of (standard) density estimation, that is, the case where $\mathscr{K}$ is the identity, the method yields the standard kernel density estimators. The maximum smoothed likelihood density estimation technique is a regularization technique. We prove an inequality which implies the stability and convergence of the regularization method for the large sample asymptotic problem. Under minimal assumptions it also implies the a.s. convergence of the finite sample density estimate via a uniform version of the strong law of large numbers. Under extra regularity conditions we get a.s. convergence rates via a uniform version of the law of the iterated logarithm (under stronger conditions than usual).

In finite mixture models, we establish the best possible rate of convergence for estimating the mixing distribution. We find that the key for estimating the mixing distribution is the knowledge of the number of components in the mixture. While a $\sqrt n$-consistent rate is achievable when the exact number of components is known, the best possible rate is only $n^{-1/4}$ when it is unknown. Under a strong identifiability condition, it is shown that this rate is reached by some minimum distance estimators. Most commonly used models are found to satisfy the strong identifiability condition.

Asymptotic normality of the difference between the number of subjects assigned to a treatment and the desired number to be assigned is established for allocation rules which use Eisele's biased coin design. Subject responses are assumed to be independent random variables from standard exponential families. In the proof, it is shown that the difference may be magnified by appropriate constants so that the magnified difference is nearly a martingale. An application to the Behrens-Fisher problem in the normal case is described briefly.

We study sequential detection of a change-point using the generalized likelihood ratio statistic. For the special case of detecting a change in a normal mean with known variance, we give approximations to the average run lengths and compare our procedure to standard CUSUM tests and combined CUSUM-Shewhart tests. Several examples indicating extensions to problems involving multiple parameters are discussed.

A new proof is given to a known result on the average run length to false alarm of the Shiryayev-Roberts change-point detection policy when the observations are nonlattice. Via the approach of this new proof, the average run length to false alarm can be calculated in the lattice case and for the mixed-type Shiryayev-Roberts scheme.

The problem of testing whether or not a change has occurred in the parameter values and order of an autoregressive model is considered. It is shown that if the white noise in the AR model is weakly stationary with finite fourth moments, then under the null hypothesis of no changepoint, the normalized Gaussian likelihood ratio test statistic converges in distribution to the Gumbel extreme value distribution. An asymptotically distribution-free procedure for testing a change of either the coefficients in the AR model, the white noise variance or the order is also proposed. The asymptotic null distribution of this test is obtained under the assumption that the third moment of the noise is zero. The proofs of these results rely on Horvath's extension of Darling-Erdos' result for the maximum of the norm of a $k$-dimensional Ornstein-Uhlenbeck process and an almost sure approximation to partial sums of dependent random variables.

We consider a standard ARMA process of the form $\phi(B)X_t = \theta(B)Z_t$, where the innovations $Z_t$ belong to the domain of attraction of a stable law, so that neither the $Z_t$ nor the $X_t$ have a finite variance. Our aim is to estimate the coefficients of $\phi$ and $\theta$. Since maximum likelihood estimation is not a viable possibility (due to the unknown form of the marginal density of the innovation sequence), we adopt the so-called Whittle estimator, based on the sample periodogram of the $X$ sequence. Despite the fact that the periodogram does not, a priori, seem like a logical object to study in this non-$\mathscr{L}^2$ situation, we show that our estimators are consistent, obtain their asymptotic distributions and show that they converge to the true values faster than in the usual $\mathscr{L}^2$ case.

A stationary exponential family is defined using transition densities which take the form of exponentiated symmetric $k$-linear forms on $\mathbf{R}^d$. Estimation is based on a mean value parametrization through a convex function on a finite-dimensional vector space. A consistency theorem and a central limit theorem are presented.