In many nonparametric problems, such as density estimation, nonparametric regression and so on, all the existing informative estimators are biased (asymptotic or finite sample). There has long been a suspicion that either informative unbiased estimators do not exist for such problems or they must be quite complicated. In this paper, we clarify the nonexistence of informative unbiased estimators in all singular problems both for fixed sample size and asymptotically (this includes most problems with optimal rate of convergence slower than $n^{-1/2}$). We also discuss situations in regular problems where such nonexistences can occur.

Consider a class $\mathscr{P}={P_\theta:\theta\in\Theta}$ of probability measures on a measurable space $(\mathscr{X},\mathscr{A})$, dominated by a $\sigma$ -finite measure $\mu$. Let $f_\theta=dP_\theta/d_\mu$, $\theta\ in\Theta$, and let $\theta_n$ be a maximum likelihood estimator based on n independent observations from $P_{\theta_0}$, $\theta_0\in\Theta$. We use results from empirical process theory to obtain convergence for the Hellinger distance $h(f_{\hat{\theta}_n}, f_{\theta_0})$, under certain entropy conditions on the class of densities ${f_\theta:\theta\in\Theta}$ The examples we present are a model with interval censored observations, smooth densities, monotone densities and convolution models. In most examples, the convexity of the class of densities is of special importance.

Let $(X_1,Y_1),\cdots,(X_n,Y_n)$ be independent replicates of the random vector $(X,Y)\in \mathbb{R}^{d+m}$, where X is $\mathbb{R}^d$-valued and Y is $\mathbb{R}^m$-valued. We assume that the conditional distribution $P(Y\in\cdot|X=x)=Q_\vartheta(\cdot)$ of Y given X = x is a member of a parametric family, where the parameter space $\Theta$ is an open subset of $\mathbb{R}^k$ with $0\in\Theta$. Under suitable regularity conditions we establish upper bounds for the power functions of asymptotic level-$\infty$ tests for the problem $\vartheta=0$ against a sequence of contiguous alternatives, as well as asymptotically optimal tests which attain these bounds. Since the testing problem involves the joint density of (X,Y) as an infinite dimensional nuisance parameter, its solution is not standard. A Monte Carlo simulation exemplifies the influence of this nuisance parameter. As a main tool we establish local asymptotic normality (LAN) of certain Poisson point processes which approximately describe our initial sample.

In statistical applications the unknown parameter of interest can frequently be defined as a functional $\theta=T(F)$, where F is an unknown population. Statistical inferences about $\theta$ are usually made based on the statistic $T(F_n)$, where $F_n$ is the empirical distribution. Assessing $T(F_n)$ (as an estimator of $\theta$) or making large sample inferences usually requires a consistent estimator of the asymptotic variance of $T(F_n)$. Asymptotic behavior of the jackknife variance estimator is closely related to the smoothness of the functional T. This paper studies the smoothness of T through the differentiability of T and establishes some general results for the consistency of the jackknife variance estimators. The results are applied to some examples in which the statistics $T(F_n)$ are L-, M-estimators and some test statistics.

Large sample approximations are developed to establish asymptotic linearity of the commonly used linear rank estimating functions, defined as stochastic integrals of counting processes over the whole line, for censored regression data. These approximations lead to asymptotic normality of the resulting rank estimators defined as solutions of the linear rank estimating equations. A second kind of approximations is also developed to show that the estimating functions can be uniformly approximated by certain more manageable nonrandom functions, resulting in a simple condition that guarantees consistency of the rank estimators. This condition is verified for the two-sample problem, thereby extending earlier results by Louis and Wei and Gail, as well as in the case when the underlying error distribution has increasing failure rate, which includes most parametric regression models in survival analysis. Techniques to handle the delicate tail fluctuations are provided and discussed in detail.

A Bayesian bootstrap for a censored data model is introduced. Its small sample distributional properties are discussed and found to be similar to Efron's bootstrap for censored data. In the absence of censoring, the Bayesian bootstrap for censored data reduces to Rubin's Bayesian bootstrap for complete data. A first-order large-sample theory is developed. This theory shows that both censored data bootstraps are consistent bootstraps for approximating the sampling distribution of the Kaplan-Meier estimator. It also shows that both bootstraps are consistent bootstraps for approximating a posterior distribution of the survival function with respect to each member of the class of conjugate beta-neutral process priors.

Nonparametric estimation of the relative risk in a generalized Cox model with multivariate time dependent covariates is considered. Estimation is based on a penalized partial likelihood. Using techniques from Andersen and Gill, and Cox and O'Sullivan, upper bounds on rate of convergence in a variety of norms are obtained. These upper bounds match the optimal rates available for linear nonparametric regression and density estimation. The results are uniform in the smoothing parameter, which is an important step for the analysis of data dependent rules for the selection of the smoothing parameter.

In the left-truncation model, one observes data $(X_i,Y_i)$ only when $Y_i\leq X_i$. Let F denote the marginal d.f. of $X_i$ , the variable of interest. The nonparametric MLE $\hat{F}_n$ of F aims at reconstructing F from truncated data. In this paper an almost sure representation of $\hat{F}_n$ is derived with improved error bounds on the one hand and under weaker distributional assumptions on the other hand.

Single-index models generalize linear regression. They have applications to a variety of fields, such as discrete choice analysis in econometrics and dose response models in biometrics, where high-dimensional regression models are often employed. Single-index models are similar to the first step of projection pursuit regression, a dimension-reduction method. In both cases the orientation vector can be estimated root-n consistently, even if the unknown univariate function (or nonparametric link function) is assumed to come from a large smoothness class. However, as we show in the present paper, the similarities end there. In particular, the amount of smoothing necessary for root-n consistent orientation estimation is very different in the two cases. We suggest a simple, empirical rule for selecting the bandwidth appropriate to single-index models. This rule is studies in a small simulation study and an application in binary response models.

We introduce a new class of variable kernels which depend on the smoothing parameter b through a simple scaling operation, and which have good integrated mean square error (IMSE) convergence properties. These kernels deform "automatically" near the boundary, eliminating boundary bias. Computational formulas are given for all orders of kernel in terms of exponentially damped sines and cosines. The kernel is a computationally convenient approximation to a certain Green's function, with the resulting kernel estimate closely related to a smoothing spline estimate.

In this paper we introduce a smooth version of local linear regression estimators and address their advantages. The MSE and MISE of the estimators are computed explicitly. It turns out that the local linear regression smoothers have nice sampling properties and high minimax efficiency-they are not only efficient in rates but also nearly efficient in constant factors. In the nonparametric regression context, the asymptotic minimax lower bound is developed via the heuristic of the "hardest onedimensional subproblem" of Donoho and Liu. Connections of the minimax risk with the modulus of continuity are made. The lower bound is also applicable for estimating conditional mean (regression) and conditional quantiles for both fixed and random design regression problems.

In this article, a class of penalized likelihood probability density estimators is proposed and studied. The true log density is assumed to be a member of a reproducing kernel Hilbert space on a finite domain, not necessarily univariate, and the estimator is defined as the unique unconstrained minimizer of a penalized log likelihood functional in such a space. Under mild conditions, the existence of the estimator and the rate of convergence of the estimator in terms of the symmetrized Kullback-Leibler distance are established. To make the procedure applicable, a semiparametric approximation of the estimator is presented, which sits in an adaptive finite dimensional function space and hence can be computed in principle. The theory is developed in a generic setup and the proofs are largely elementary. Algorithms are yet to follow.

Observations $y_{ij}$ are made at points $(x_{1i},x_{2j})$ according to the model $y_{iy}=F(x_{1i},x_{2j})+e_{ij}$, where the $e_{ij}$ are independent normals with constant variance. In order to test that $F(x_1,x_2)$ is an additive function of $x_1$ and $x_2$, a likelihood ratio test is constructed comparing $F(x_1,x_2)=\mu+Z_1 (x_1)+Z_2(x_2)$ with $F(x_1,x_2)=\mu+Z_1(x_1)+Z_2(x_2)+Z(x_1,x_2)$, where $Z_1$, $Z_2$ are Brownian motions and Z is a Brownian sheet. The ratio of Brownian sheet variance to error variance $\infty$ is chosen by maximum likelihood and the likelihood ratio test statistic W of $H_0:\infty=0$ used to test for departures from additivity. The asymptotic null distribution of W is derived, and its finite sample size behaviour is compared with two standard tests in a simulation study. The W test performs well on the five alternatives considered.

In this paper two bootstrap procedures are considered for the estimation of the distribution of linear contrasts and of F-test statistics in high dimensional linear models. An asymptotic approach will be chosen where the dimension p of the model may increase for sample size $n\rightarrow\infty$. The range of validity will be compared for the normal approximation and for the bootstrap procedures. Furthermore, it will be argued that the rates of convergence are different for the bootstrap procedures in this asymptotic framework. This is in contrast to the usual asymptotic approach where p is fixed.

We show that the method of importance resampling, introduced by Vernon Johns and Anthony Davison, may be enhanced by balancing the resamples. It is demonstrated that "balanced importance resampling" improves on both "balanced uniform resampling" and "random importance resampling", from the viewpoint of statistical efficiency. Moreover, the range of applications for which efficient resampling methods may be applied is extended to include statistics which are smooth functions of solutions of estimating equations.

A natural extension of the simple leave-one-out cross validation (CV) method is to allow the deletion of more than one observations. In this article, several notions of the multifold cross validation (MCV) method have been discussed. In the context of variable selection under a linear regression model, we show that the delete-d MCV criterion is asymptotically equivalent to the well known FPE criterion. Two computationally more feasible methods, the r-fold cross validation and the repeated learning-testing criterion, are also studied. The performance of these criteria are compared with the simple leave-one-out cross validation method. Simulation results are obtained to gain some understanding on the small sample properties of these methods.

We study how robust estimators can be in parametric families, obtaining a lower bound on the contamination bias of an estimator that holds for a wide class of parametric families. This lower bound includes as a special case the bound used to establish that the median is bias minimax among location equivariant estimators, and it is tight or nearly tight in a variety of other settings such as scale estimation, discrete exponential families and multiple linear regression. The minimum variation distance estimator has contamination bias within a dimension-free factor of this bound. A second lower bound applies to locally linear estimates and implies that such estimates cannot be bias minimax among all Fisher-consistent estimates in higher dimensions. In linear regression this class of estimates includes the familiar M-estimates, GM-estimates and S-estimates. In discrete exponential families, yet another lower bound implies that the "proportion of zeros" estimates has minimax bias if the median of the distribution is zero, a common situation in some fields. This bound also implies that the information-standardized sensitivity of every Fisher consistent estimate of the Poisson mean and of the Binomial proportion is unbounded.

In 1964, P. Huber established the following minimax bias robustness result for estimating the location $\mu$ in the $\varepsilon$-contamination family $F(x)=(1-\varepsilon)\phi[(x-\mu)/s]+\varepsilon H(x)$, where $\phi$ is the standard normal distribution and H is an arbitrary distribution function: The median minimizes the maximum asymptotic bias among all translation equivariant estimates of location. However, the median efficiency of $2/\pi$ at the Gaussian model may be unacceptably low in some applications. This motivates one to solve the following problem for the above $\varepsilon$-contamination family: Among all location M-estimates, find the one which minimizes the maximum asymptotic bias subject to a constraint on efficiency at the Gaussian model. This problem is the dual form analog of Hampel's optimality problem of minimizing the asymptotic variance at the nominal model (e.g., the Gaussian model) subject to a bound on the gross-error sensitivity. We solve the global problem completely for the case of a known scale parameter. The main conclusion is that Hampel's heuristic is essentially correct: The resulting M-estimate is based on a $\psi$ function which is amazingly close, but not exactly equal, to the Huber/Hampel optimal $\psi$. It turns out that one pays only a relatively small price in terms of increase in maximal bias for increasing efficiency from 64% to the range 90-95%. We also present a conjectured solution to the problem, based on heuristic arguments and numerical calculations, when the nuisance scale parameter is unknown.

The so-called method of invariants is a technique in the field of molecular evolution for inferring phylogenetic relations among a number of species on the basis of nucleotide sequence data. An invariant is a polynomial function of the probability distribution defined by a stochastic model for the observed nucleotide sequence. This function has the special property that it is identically zero for one possible phylogeny and typically nonzero for another possible phylogeny. Thus it is possible to discriminate statistically between two competing phylogenies using an estimate of the invariant. The advantage of this technique is that it enables such inferences to be made without the need for estimating nuisance parameters that are related to the specific mechanisms by which the molecular evolution occurs. For a wide class of models found in the literature, we present a simple algebraic formalism for recognising whether or not a function is an invariant and for generating all possible invariants. Our work is based on recognising an uderlying group structure and using discrete Fourier analysis.

In this paper we consider the problem of estimating the loss of point estimators and study admissibility of such estimators of loss. We adapt and extend the "unified admissibility proof" idea of Brown and Hwang to this problem. We first present the result in the Gaussian setup. We then generalize the procedure to general exponential family distributions and apply it to the Poisson distribution. The result for the gamma distribution is also stated. The role played by the "polydisc transform" (cf. Johnstone and MacGibbon) in making explicit the relationship between the Gaussian and Poisson cases is discussed.

We consider a group of k experts each having a subjective probability distribution for a parameter $\theta$. If the members of the group are allowed to know the others' opinions and they appreciate the others' skills, it is likely that each expert will modify his distribution to account for this new information. This process can be continued indefinitely leading to an iterative pooling process. The main issue is whether the experts'distributions will converge towards a common limit or consensus. Several authors have considered this iterative process when the experts' distributions at a given stage are linear opinion pools of the distributions at the previous stage. In this paper we extend the model for the specific case where the experts use logarithmic opinion pools and, more broadly, for pools in a wide class that generalizes both the linear and the logarithmic pools. It is shown that the consensus properties in the logarithmic pool case are essentially the same as in the linear pool case, and that this fact uniquely characterizes both pools in the wide class mentioned above.

E-optmal designs for the full mean parameter vector, and for many subsets in univariate polynomial regression models are determined. The derivation is based on the interplay between E-optimality and scalar optimality. The scalar parameter systems are obtained as transformations of the coefficient vector c of the Chebyshev polynomial.

In the usual linear model $y=\theta'f(x)$ we consider the E-optimal design problem. A sequence of generalized Elfving sets $\mathscr{R}_k\subseteq\ mathbb{R}^{n \times k}$ (where n is the number of regression functions) is introduced and the corresponding in-ball radii are investigated. It is shown that the E-Optimal design is an optimal desing for $A'\theta$, where $A\in\mathbb{R}^{n \times n}$ is any in-ball vector of a generalized Elfving set $\mathscr{R}_n\subseteq\mathbb{R}^{n \times n}$. The minimum eigenvalue of the E-optimal design can be identified as the corresponding squared in-ball radius of $\mathscr{R}_n$. A necessary condition for the support points of the E-optimal design is given by a consideration of the supporting hyperplanes corresponding to the in-ball vectors of $\mathscr{R}_n$. The results presented allow the determination of E-optimal designs by an investigation of the geometric properties of a convex symmetric subset $\mathscr{R}_n$ of $\mathbb{R}^{n \times n}$ without using any equivalence theorems. The application is demonstrated in several examples solving elementary geometric problems for the determination of the E-optimal design. In particular we give a new proof of the E-optimal spring balance and chemical balance weighing (approximate) designs.

Model robustness in optimal regression design is studied by introducing a family of nonparametric models, which are defined as neighborhoods of classical parametric models in terms of the uniform norm. Optimal designs are sought under a minimax criterion for estimating linear functionals on such models that may be put as integrals using measures of finite support. A set of conditions equivalent to design optimality is derived using a Lagrangian principle applicable when the dimension is infinite and the function is not everywhere differentiable. From these conditions various optimal designs follow. Among them is the classical extrapolation design of Kiefer and Wolfowitz for Chebyshev regression, which is therefore model-robust against uniform departure. The conditions also shed light on other classical results of Kiefer and Wolfowitz and of others.

This paper investigates the effect of prior information on the design of experiments for detecting the potential impact of an event which is to occur at a specified time, in the knowledge of possible overall changes to the population as a whole. It is assumed that an F test of interaction is to be used to decide if an impact has occurred. Maximizing the power of this test, or rather the simpler, closely related goal of maximizing the noncentrality parameter is taken to be the designer's objective. Some of the results obtained are qualitative. For example, for certain fairly realistic general models of how a subregional impact might distribute itself, it is shown that it is never optimal to place more than 50% of the monitoring sites in any one of the homogeneous subregions in which the impact might occur. Another qualitative, more intuitively obvious result is that it is essential to monitor subregions where the impact is not likely to occur ("quasicontrols"); this would maximize the contrast created by the potential impact. A very general solution to the optimal design problem is given in a form which could be readily implemented in practice with the aid of a computer. Explicit solutions are also given for certain realistic impact models.

A model for sequential clinical trials involving three treatments is proposed and various error probabilities are defined. A specific procedure involving a concatenation of repeated significance tests is studied in detail. The first phase of testing proceeds until at least one treatment is eliminated; and then a second phase is used to compare the remaining treatments. Analytic approximations to the operating characteristics of the test are derived and compared with the results of a Monte Carlo experiment.

We consider the problem of sequentially estimating one parameter in a class of two-parameter exponential family of distributions. We assume a weighted squared error loss with a fixed cost of estimation error. The stopping rule, based on the maximum likelihood estimate of the nuisance parameter, is shown to be independent of the terminal estimate. The asymptotic normality of the stopping variable is established and approximations to its mean and to the regret associated with it are also provided. The general results are exemplified by the normal, gamma and the inverse Gaussian densities.

Let $S_1$, $S_2,\cdots$ denote a driftless random walk with values in an inner product space $\mathscr{W}$; let $Z_1$, $Z_2,\cdots$ denote a perturbed random walk of the form $Z_n=n+\langle c,S_n \rangle+\xi_n$, $n = 1, 2,\cdots$, where $\xi_1,\xi_2,\cdots$ are slowly changing, $\langle\centerdot,\centerdot\rangle$ denotes the inner product, and $c\in\mathscr{W}$; and let $t=t_a=inf{n\geq1:Z_n>a}$ for $0\leq a<\infty$. Conditions are developed under which the first four moments of $X_t:=S_t/t$ have asymptotic expansions, and the expansions are found. Stopping times of this form arise naturally in sequential estimation problems, and the main results may be used to find asymptotic expansions for risk functions in such problems. Examples of such applications are included.

It is shown that, under some regularity conditions, the least squares estimator of a stationary ergodic threshold autoregressive model is strongly consistent. The limiting distribution of the least squares estimator is derived. It is shown that the estimator of the threshold parameter is N consistent and its limiting distribution is related to a compound Poisson Process.

In an AR(p) model, R-estimation of a subset of parameters is considered when the complementary subset is possibly redundant. Along with the rank test of the full hypothesis and the subhypothesis of the parameters, both preliminary test and shrinkage R-estimators are considered. In the light of asymptotic distributional risk, the relative asymptotic risk-efficiency results are given. Though, the shrinkage R-estimator may dominate their classical versions, they do not in general dominate the preliminary test R-estimators.

It is shown that serial correlation can produce striking effects in distributions of change-point statistics. Failure to account for these effects is shown to invalidate change-point tests, either through increases in the type 1 error rates if low frequency spectral mass predominates in the spectrum of the noise process, or through diminution of the power of the tests when high frequency mass predominates. These effects are characterized by the expression ${2\pi f(0)/\int^\pi_-\pi f(\lambda)d\lambda}$, where $f(\centerdot)$ is the spectral density of the noise process; in sample survey work this is know as the design effect or "deff." Simple precise adjustments to change-point test statistics which account for serial correlation are provided. The same adjustment applies to all commonly used regression models. Residual processes are derived for both stationary time series satisfying a moment condition and for general linear regression models with stationary error structure.