A. N. Kolmogorov contributed much to mathematical statistics, both through his own statistical works and through the researches of his students and disciples in statistics. But the case will be made that he influenced statistics still more through his general mathematical concepts and approaches. This paper gives some examples of unexpected applications of Kolmogorov's ideas in topics of nonparametric statistics.

The paper deals with upper and lower bounds for the quality of (probability) density estimation. Connections are established between these problems and the theory of approximation of functions. Particularly, it is demonstrated how some of Kolmogorov's concepts work.

In this paper Kolmogorov's work in mathematical statistics is reviewed. The main areas under discussion are: the idea of sufficiency, linear models and unbiased estimators. The relationship of Kolmogorov's contributions with modern statistical theory, in particular, with Bayesian analysis, is analyzed.

A bibliography is given of works by Kolmogorov of statistical interest and of statistical papers by others that have cited his work. Also listed separately are bibliographies of Kolmogorov's works in all fields.

We discuss properties of some data-analytic methods which are intimately related to each other: alternating least squares (ALS), correspondence analysis and more recently Breiman and Friedman's ACE algorithm. The application of these methods to regression produces nonparametric estimators of nonlinear transformations, both of the response and the predictors. These procedures are among the most powerful tools for data analysis, but missing awareness of some artifacts could lead to inappropriate interpretations. We point out some anomalies as well as some curiosities in the mathematics of these methods, and we relate them to some areas in computer-aided tomography, projection pursuit regression and nonlinear devices in the theory of noise.

Let $l(\theta) = n^{-1} \log p(x, \theta)$ be the log likelihood of an $n$-dimensional $X$ under a $p$-dimensional $\theta$. Let $\hat{\theta}_j$ be the mle under $H_j: \theta^1 = \theta^1_0, \ldots, \theta^j = \theta^j_0$ and $\hat{\theta}_0$ be the unrestricted mle. Define $T_j$ as $\lbrack 2n\{l(\hat{\theta}_{j - 1}) - l(\hat{\theta}_j)\}\rbrack^{1/2} \operatorname{sgn}(\hat{\theta}^j_{j - 1} - \theta^j_0).$ Let $T = (T_1, \ldots, T_p)$. Then under regularity conditions, the following theorem is proved: Under $\theta = \theta_0, T$ is asymptotically $N(n^{-1/2}a_0 + n^{-1}a, J + n^{-1}\sum) + O(n^{-3/2})$ where $J$ is the identity matrix. The result is proved by first establishing an analogous result when $\theta$ is random and then making the prior converge to a degenerate distribution. The existence of the Bartlett correction to order $n^{-3/2}$ follows from the theorem. We show that an Edgeworth expansion with error $O(n^{-2})$ for $T$ involves only polynomials of degree less than or equal to 3 and hence verify rigorously Lawley's (1956) result giving the order of the error in the Bartlett correction as $O(n^{-2})$.

The membranes of cells, such as those found in heart and nerve tissue, contain structures known as ion channels. The current produced by an ion channel in a cell membrane has been modelled as a finite state-space Markov process. Statistical analysis of ion channels has been largely limited to the case where a single channel is observed, or where the stimulus to the channel is so slight that only one channel at a time is emitting current. In many experiments, however, several channels are producing current simultaneously. Therefore only the superposition of the currents from an unknown number of channels can be observed. A hypothesis test is established which tests whether these channels operate independently and identically. If the null hypothesis is not rejected, confidence intervals for the mean emitting time and mean nonemitting time of individual channels and estimates of the distribution functions of the sojourns in emitting states and nonemitting states are obtained.

Suppose $z(\cdot)$ is a random process defined on a bounded set $R \subset \mathbb{R}^1$ with finite second moments. Consider the behavior of linear predictions based on $z(t_1), \ldots, z(t_n)$, where $t_1, t_2, \cdots$ is a dense sequence of points in $R$. Stein showed that if the second-order structure used to generate the predictions is incorrect but compatible with the correct second-order structure, the obtained predictions are uniformly asymptotically optimal as $n \rightarrow \infty$. In the present paper, a general method is described for obtaining rates of convergence when the covariance function is misspecified but compatible with the correct covariance function. When $z(\cdot)$ is Gaussian, these bounds are related to the entropy distance (the symmetrized Kullback divergence) between the measures for the random field under the actual and presumed covariance functions. Explicit bounds are given when $R = \lbrack 0, 1\rbrack$ and $z(\cdot)$ is stationary with spectral density of the form $f(\lambda) = (a^2 + \lambda^2)^{-p}$, where $p$ is a known positive integer and $a$ is the parameter that is misspecified. More precise results are given in the case $p = 1$. An application of this result implies that equally spaced observations are asymptotically optimal in the sense used by Sacks and Ylvisaker in terms of maximizing the Kullback divergence between the actual and presumed models when $z(\cdot)$ is Gaussian.

Wahba compared the performance of generalized cross validation (GCV) and modified maximum likelihood (MML) procedures for choosing the smoothing parameter of a smoothing spline. This work makes a more careful study of the two procedures when the stochastic model motivating the modified maximum likelihood estimate is correct. In particular, it is shown that in the case of the linear smoothing spline with equally spaced observations, both estimates are asymptotically normal with the GCV estimate having twice the asymptotic variance of the MML estimate. The impact of using these estimates on the subsequent predictions is also calculated. Conjectures on how these results should generalize to higher order smoothing splines are developed. These conjectures suggest that the penalty for using GCV instead of MML when the stochastic model is correct is greater for higher order smoothing splines, both in terms of the efficiency in estimating the smoothing parameter and the impact on subsequent predictions.

Consider an $m$-dimensional diffusion process $(X_t)$ with unknown drift and small known variance observed on a time interval $\lbrack 0, T\rbrack$. We derive here a general condition ensuring the asymptotic sufficiency, in the sense of Le Cam, of incomplete observations of $(X_t)_{0 \leq t \leq T}$ with respect to the complete observation of the diffusion as the variance goes to 0. We then construct estimators based on these partial observations which are consistent, asymptotically Gaussian and asymptotically equivalent to the maximum likelihood estimator based on the observation of the complete sample path on $\lbrack 0, T\rbrack$. Finally, we study when this condition is satisfied for various incomplete observations which often arise in practice: discrete observations, observations of a smoothed diffusion, observation of the first hitting times and positions of concentric spheres, complete or partial observation of the record process for one-dimensional diffusions.

Martingale and counting process techniques are applied to the problem of inference for general conditional hazard functions. This problem was first studied by Beran, who introduced a class of estimators for the conditional cumulative hazard and survival functions in the special case of time-independent covariates. Here the covariate can be time dependent; the classical i.i.d. assumptions are relaxed by replacing them with certain asymptotic stability assumptions, and models involving recurrent failures are included. This is done within the framework of a general nonparametric counting process regression model. Important examples of the model include right-censored survival data, semi-Markov processes, an illness-death process with duration dependence, and age-dependent birth and death processes.

Bahadur has obtained an asymptotic almost sure representation for empirical quantiles of independent and identically distributed random variables. In this paper we present an analogous result for a large class of stationary linear processes.

Many of the popular nonparametric test statistics for censored survival data used in two-sample, $s$-sample trend and single continuous covariate situations are special cases of a general statistic, differing only in the choice of covariate-based label and weight function. Formulated in terms of counting processes and martingales this general statistic, standardized by the square root of its consistent variance estimator, is shown to be asymptotically normal under the null hypothesis and under a sequence of contiguous hazard alternatives that includes both relative and excess risk models. As an application to some specific cases of the general statistic, the asymptotic relative efficiencies of the Brown, Hollander and Korwar modification of the Kendall rank statistic, the Cox score statistic and the generalized logrank statistic of Jones and Crowley are investigated under relative and excess risk models. Finally, an example is given in which the Cox score test is not as efficient as the generalized logrank test in the presence of outliers in the covariate space.

To check the validity of an assumed parametric model for survival data, one may compare $\hat{A}(t)$, the nonparametric Nelson-Aalen plot of the cumulative hazard rate, with $A(t, \hat{\theta})$, the estimated parametric cumulative hazard rate, $\hat{\theta}$ being for example the maximum likelihood estimator. Convergence in distribution of $\sqrt n (\hat{A}(t) - A(t, \hat{\theta}))$ and more general processes is studied in the present paper, employing the general framework of counting processes, which allows for quite general models for life history data and for quite general censoring schemes. The results are applied to the construction of $\chi^2$-type statistics for goodness of fit. Cramer-von Mises and Kolmogorov-Smirnov type tests are presented in the case where the unknown parameter is one-dimensional. Power considerations are also included, and some optimality results are reached. Finally tests are constructed for the hypothesis that the unspecified hazard rate part in Cox's regression model follows a parametric form.

Several authors have constructed nonparametric Bayes estimators for a cumulative distribution function based on (possibly right-censored) data. The prior distributions have, for example, been Dirichlet processes or, more generally, processes neutral to the right. The present article studies the related problem of finding Bayes estimators for cumulative hazard rates and related quantities, w.r.t. prior distributions that correspond to cumulative hazard rate processes with nonnegative independent increments. A particular class of prior processes, termed beta processes, is introduced and is shown to constitute a conjugate class. To arrive at these, a nonparametric time-discrete framework for survival data, which has some independent interest, is studied first. An important bonus of the approach based on cumulative hazards is that more complicated models for life history data than the simple life table situation can be treated, for example, time-inhomogeneous Markov chains. We find posterior distributions and derive Bayes estimators in such models and also present a semiparametric Bayesian analysis of the Cox regression model. The Bayes estimators are easy to interpret and easy to compute. In the limiting case of a vague prior the Bayes solution for a cumulative hazard is the Nelson-Aalen estimator and the Bayes solution for a survival probability is the Kaplan-Meier estimator.

Lower bounds on Bayes factors in favor of the null hypothesis in multinomial tests of point null hypotheses are developed. These are then applied to derive lower bounds on Bayes factors in both exact and asymptotic chi-squared testing situations. The general conclusion is that the lower bounds tend to be substantially larger than $P$-values, raising serious questions concerning the routine use of moderately small $P$-values (e.g., 0.05) to represent significant evidence against the null hypothesis.

A $k$-sided die is thrown $n$ times, to estimate the probabilities $\theta_1, \ldots, \theta_k$ of landing on the various sides. The MLE of $\theta$ is the vector of empirical proportions $p = (p_1, \ldots, p_k)$. Consider a set of Bayesians that put uniformly positive prior mass on all reasonable subsets of the parameter space. Their posterior distributions will be uniformly concentrated near $p$. Sharp bounds are given, using entropy. These bounds apply to all sample sequences: There are no exceptional null sets.

We give an upper bound for the posterior probability of a measurable set $A$ when the prior lies in a class of probability measures $\mathscr{P}$. The bound is a rational function of two Choquet integrals. If $\mathscr{P}$ is weakly compact and is closed with respect to majorization, then the bound is sharp if and only if the upper prior probability is 2-alternating. The result is used to compute bounds for several sets of priors used in robust Bayesian inference. The result may be regarded as a characterization of 2-alternating Choquet capacities.

Observations $y_i$ are made at points $x_i$ according to the model $y_i = F(x_i) + e_i$, where the $e_i$ are independent normals with constant variance. In order to decide whether or not $F(x)$ is constant, a likelihood ratio test is constructed, comparing $F(x) \equiv \mu$ with $F(x) = \mu + Z(x)$, where $Z(x)$ is a Brownian motion. The ratio of error variance to Brownian motion variance is chosen to maximize the likelihood, and the resulting maximum likelihood statistic $B$ is used to test departures from constant mean. Its asymptotic distribution is derived and its finite sample size behavior is compared with five other tests. The $B$-statistic is comparable or superior to each of the tests on the five alternatives considered.

The problem of designing an experiment to estimate the difference between the means of two normal populations with unit variances is considered, when the cost of drawing a sample from either population may depend on unknown parameters. A quasi-Bayesian approach is adopted in which the mean difference is estimated by its maximum likelihood estimator, but the design (allocation rule) is evaluated in Bayesian, decision-theoretic terms. A three-stage procedure is proposed and its risk evaluated, up to terms which are small compared to the cost of a single observation. This procedure is shown to be optimal to second order for squared error loss.

Time-sequential point estimation is studied in the model of fully parametric censored data and Cox's regression model. Both are investigated in the context of counting processes through estimating equations defined by martingales. The concept of information and a related inequality developed in estimating function theory by Godambe are adapted to these models. These suggest some optimality criteria for choosing stopping times as well as estimators. These lead naturally to some sequential procedures, which are shown to be asymptotically efficient with respect to the above criteria.

In inverse regression problems (or more generally, the estimation of ratios of regression parameters) and errors-in-variables models, it has been shown by Gleser and Hwang that the length of any confidence interval with positive confidence level is infinite with positive probability. Therefore the confidence sets derived using asymptotic theory, although having correct asymptotic coverage probability, typically have zero confidence level when the sample size is fixed. Is it possible to construct a sequential confidence interval with finite length and $1 - \alpha > 0$ confidence level? The answer is no for any finite stage sequential sampling. The answer is, however, yes for a fully sequential scheme, as demonstrated by Hwang and Liu. For the inverse regression problem, and more generally the set estimation of a ratio of regression parameters, we construct a $(1 - \alpha)$ confidence sequence. Applying such a confidence sequence, we can construct a $(1 - \alpha)$ sequential confidence interval with the length less than a prespecified quantity.

Let $(X_1, Z_1), (X_2, Z_2), \ldots, (X_n, Z_n)$ be iid as $(X, Z), Z$ taking values in $R^1$, and for $0 < p < 1$, let $\xi_p(x)$ denote the conditional $p$-quantile of $Z$ given $X = x,$ i.e., $P(Z \leq \xi_p(x)\mid X = x) = p$. In this paper, kernel and nearest-neighbor estimators of $\xi_p(x)$ are proposed. In order to study the asymptotics of these estimates, Bahadur-type representations of the sample conditional quantiles are obtained. These representations are used to examine the important issue of choosing the smoothing parameter by a local approach (for a fixed $x$) based on weak convergence of these estimators with varying $k$ in the $k$-nearest-neighbor method and with varying $h$ in the kernel method with bandwidth $h$. These weak convergence results lead to asymptotic linear models which motivate certain estimators.

Consider estimating the mean of a standard Gaussian shift when that mean is known to lie in an orthosymmetric quadratically convex set in $l_2$. Such sets include ellipsoids, hyperrectangles and $l_p$-bodies with $p > 2$. The minimax risk among linear estimates is within 25% of the minimax risk among all estimates. The minimax risk among truncated series estimates is within a factor 4.44 of the minimax risk. This implies that the difficulty of estimation--a statistical quantity--is measured fairly precisely by the $n$-widths--a geometric quantity. If the set is not quadratically convex, as in the case of $l_p$-bodies with $p < 2$, things change appreciably. Minimax linear estimators may be out-performed arbitrarily by nonlinear estimates. The (ordinary, Kolmogorov) $n$-widths still determine the difficulty of linear estimation, but the difficulty of nonlinear estimation is tied to the (inner, Bernstein) $n$-widths, which can be far smaller. Essential use is made of a new heuristic: that the difficulty of the hardest rectangular subproblem is equal to the difficulty of the full problem.

We study the asymptotic normality of the jackknife histogram. For one sample mean, it holds if and only if $r$, the number of observations retained, and $d (= n - r)$, the number of observations deleted, both diverge to infinity. The best convergence rate $n^{-1/2}$ is obtained when $r = O(n)$ and $d = O(n)$. For $U$ statistics of degree 2 and nonlinear statistics admitting the expansion (3.1), similar results are obtained under conditions on $r$ and $d$. A second order approximation based on the Edgeworth expansion is discussed briefly.

Three types of asymptotic $\chi^2$ goodness-of-fit tests derived under the normal assumption have been used widely in factor analysis. Asymptotic behavior of the test statistics is investigated here for the factor analysis model with linearly or nonlinearly restricted factor loadings under weak assumptions on the factor vector and the error vector. In particular the limiting $\chi^2$ result for the three tests is shown to hold for the factor vector, either fixed or random with any distribution having finite second-order moments, and for the error vector with any distribution having finite second-order moments, provided that the components of the error vector are independent, not just uncorrelated. As special cases the result holds for exploratory and confirmatory factor analysis models and for certain nonnormal structural equation (LISREL) models.

Suppose $X_1, X_2, \ldots$ are independent random variables such that for some unknown $\nu$, each of $X_1, \ldots, X_{\nu - 1}$ is distributed according to $F_0$, while $X_\nu, X_{\nu + 1}, \ldots$ are all distributed according to $F_1$. We prove a result of Moustakides that claims that the CUSUM procedures are optimal in the sense of Lorden. We do that by proving that the procedures are Bayes for some stochastic mechanism of generating $\nu$.

The problem of testing linear hypotheses about the parameter vector of an autoregressive process with finite order is considered. Based on the property of local asymptotic normality, we derive asymptotically optimal statistical tests. Additionally, we define and investigate so-called residual rank tests. For these tests we obtain under the null hypothesis an asymptotic distribution which does not depend on the distribution of the innovation.

The Berry-Esseen-type bounds of order $N^{-1/2}$ for the rate of convergence to normality are derived for the signed linear rank statistics under the hypothesis of symmetry. The results are obtained with a broad range of regression constants and scores (allowed to be generated by discontinuous score generating functions, but not necessarily) restricted by only mild conditions, while almost all previous results are obtained with continuously differentiable score generating functions. Furthermore, the proof is very short and elementary, based on the conditioning argument.

Let $\alpha_n$ be the classical empirical process and $T: D\lbrack 0, 1\rbrack \rightarrow R$. Assume $T$ satisfies the Lipschitz condition. Using the Komlos-Major-Tusnady inequality, bounds for $P(T(\alpha_n) \geq x_n \sqrt n)$ are obtained for every $n$ and $x_n > 0$. Hence expansions for large deviations, as well as some moderate and Cramer-type large-deviations results for $T(\alpha_n)$, are derived.

For any sequence of empirical probability measures $\{\mu_n\}$ on the Borel sets of the real line and any $\delta > 0$, there exists a singular continuous probability measure $\mu$ such that $\inf_n \sup_A |\mu_n(A) - \mu(A)| \geq \frac{1}{2} - \delta \quad\text{almost surely}.$