Empirical likelihood is a nonparametric method of inference. It has sampling properties similar to the bootstrap, but where the bootstrap uses resampling, it profiles a multinomial likelihood supported on the sample. Its properties in i.i.d. settings have been investigated in works by Owen, by Hall and by DiCiccio, Hall and Romano. This article extends the method to regression problems. Fixed and random regressors are considered, as are robust and heteroscedastic regressions. To make the extension, three variations on the original idea are considered. It is shown that when some functionals of the distribution of the data are known, one can get sharper inferences on other functionals by imposing the known values as constraints on the optimization. The result is first order equivalent to conditioning on a sample value of the known functional. The use of a Euclidean alternative to the likelihood function is investigated. A triangular array version of the empirical likelihood theorem is given. The one-way ANOVA and heteroscedastic regression models are considered in detail. An example is given in which inferences are drawn on the parameters of both the regression function and the conditional variance model.

This paper introduces a hierarchical Bayes (HB) approach for prediction in general mixed linear models. The results find application in small area estimation. Our model unifies and extends a number of models previously considered in this area. Computational formulas for obtaining the Bayes predictors and their standard errors are given in the general case. The methods are applied to two actual data sets. Also, in a special case, the HB predictors are shown to possess some interesting frequentist properties.

We present a general procedure for obtaining an exact confidence set for the variance components in a mixed linear model. The procedure can be viewed as a generalization of the ANOVA method used with balanced models. Our procedure uses, as pivotal quantities, quadratic forms that are distributed independently as chi-squared variables. These quadratic forms are constructed with reference to spaces that are orthogonal with respect to the covariance matrix of the observation vector, which is a function of the variance components. For balanced models, these pivotal quantities simplify to multiples of the sums of squares used in the ANOVA method. An exact confidence set for the vector of ratios of the effect variances to the error variance is also presented, based on the same collection of quadratic forms. Computing formulas for calculating approximations to these confidence sets are presented, and the results of their application to several two-way data sets are given.

The estimation of the treatment effect in the two-sample problem with right censoring is of interest in survival analysis. In this article we consider both the location shift model and the scale change model. We establish the large-sample properties of a generalized Hodges-Lehmann type estimator. The strong consistency is established under the minimal possible conditions. The asymptotic normality is also obtained without imposing any conditions on the censoring mechanisms. As a by-product, we also establish a result for the oscillation behavior of the Kaplan-Meier process, which extends the Bahadur result for the empirical process to the censored case.

A sequence of independent nonnegative random variables with common distribution function $F$ is censored on the right by another sequence of independent identically distributed random variables. These two sequences are also assumed to be independent. We estimate the density function $f$ of $F$ by a sequence of kernel estimators $f_n(t) = (\int^\infty_{-\infty}K((t - x)/h(n))d\hat{F}_n(x))/h(n),$ where $h(n)$ is a sequence of numbers, $K$ is kernel density function and $\hat{F}_n$ is the product-limit estimator of $F.$ We prove central limit theorems for $\int^T_0|f_n(t) - f(t)|^p d\mu(t), 1 \leq p < \infty, 0 < T \leq \infty,$ where $\mu$ is a measure on the Borel sets of the real line. The result is tested in Monte Carlo trials and applied for goodness of fit.

Consider a $\mathscr{Y}$-valued response variable having a density function $f(\cdot\mid x)$ that depends on an $\mathscr{X}$-valued input variable $x.$ It is assumed that $\mathscr{X}$ and $\mathscr{Y}$ are compact intervals and that $f(\cdot\mid\cdot)$ is continuous and positive on $\mathscr{X} \times \mathscr{Y}.$ Let $F(\cdot\mid x)$ denote the distribution function of $f(\cdot\mid x)$ and let $Q(\cdot\mid x)$ denote its quantile function. A finite-parameter exponential family model based on tensor-product $B$-splines is constructed. Maximum likelihood estimation of the parameters of the model based on independent observations of the response variable at fixed settings of the input variable yields estimates of $f(\cdot \mid \cdot), F(\cdot \mid \cdot)$ and $Q(\cdot \mid \cdot).$ Under mild conditions, if the number of parameters suitably tends to infinity as $n \rightarrow \infty,$ these estimates have optimal rates of convergence. The asymptotic behavior of the corresponding confidence bounds is also investigated.

We consider interaction splines which model a multivariate regression function $f$ as a constant plus the sum of functions of one variable (main effects), plus the sum of functions of two variables (two-factor interactions), and so on. The estimation of $f$ by the penalized least squares method and the asymptotic properties of the models are studied in this article. It is shown that, under some regularity conditions on the data points, the expected squared error averaged over the data points converges to zero at a rate of $O(N^{-2m/(2m + 1)})$ as the sample size $N \rightarrow \infty$ if the smoothing parameters are appropriately chosen, where $m$ is a measure of the assumed smoothness of $f.$

In a nonparametric regression setup where the covariates are continuous, the problem of estimating the number of covariates will be discussed in this paper. The kernel method is used to estimate the regression function and the selection criterion is based on minimizing the cross-validation estimate of the mean squared prediction error. We consider choosing both the bandwidth and the number of covariates based on the data. Unlike the case of linear regression, it turns out that the selection is consistent and efficient even when the true model has only a finite number of covariates. In addition, we also observe the curse of dimensionality at work.

The problem of automatic bandwidth selection for a kernel density estimator is considered. It is well recognized that the bandwidth estimate selected by the least squares cross-validation is subject to large sample variation. This difficulty limits the application of the cross-validation estimate. Based on characteristic functions, an important expression for the cross-validation bandwidth estimate is obtained. The expression clearly points out the source of variation. To stabilize the variation, a simple bandwidth selection procedure is proposed. It is shown that the stabilized bandwidth selector gives a strongly consistent estimate of the optimal bandwidth. Under commonly used smoothness conditions, the stabilized bandwidth estimate has a faster convergence rate than the convergence rate of the cross-validation estimate. For sufficiently smooth density functions, it is shown that the stabilized bandwidth estimate is asymptotically normal with a relative convergence rate $n^{-1/2}$ instead of the rate $n^{-1/10}$ of the cross-validation estimate. A plug-in estimate and an adjusted plug-in estimate are also proposed, and their asymptotic distributions are obtained. It is noted that the plug-in estimate is asymptotically efficient. The adjusted plug-in bandwidth estimate and the stabilized bandwidth estimate are shown to be asymptotically equivalent. The simulation results verify that the proposed procedures perform much better than the cross-validation for finite samples.

For nonparametric regression, in the case of dependent observations, cross-validation is known to be severely affected by dependence. This effect is precisely quantified through a limiting distribution for the cross-validated bandwidth. The performance of two methods, the "leave-$(2l + 1)$-out" version of cross-validation and partitioned cross-validation, which adjust for the effect of dependence on bandwidth selection is investigated. The bandwidths produced by these two methods are analyzed by further limiting distributions which reveal significantly different characteristics. Simulations demonstrate that the asymptotic effects hold for reasonable sample sizes.

The asymptotically best bandwidth selectors for a kernel density estimator currently require the use of either unappealing higher order kernel pilot estimators or related Fourier transform methods. The point of this paper is to present a methodology which allows the fastest possible rate of convergence with the use of only nonnegative kernel estimators at all stages of the selection process. The essential idea is derived through careful study of factorizations of the pilot bandwidth in terms of the original bandwidth.

Central limit theorems are proven for $L_p$-norms $(1 \leq p < \infty)$ of multivariate density estimators.

Ridge regression is a well-known technique to estimate the coefficients of a linear model. The method of regularization is a similar approach commonly used to solve underdetermined linear equations with discrete noisy data. When applying such a technique, the choice of the smoothing (or regularization) parameter $h$ is crucial. Generalized cross-validation (GCV) and Mallows' $C_L$ are two popular methods for estimating a good value for $h,$ from the data. Their asymptotic properties, such as consistency and asymptotic optimality, have been largely studied [Craven and Wahba (1979); Golub, Heath and Wahba (1979); Speckman (1985)]. Very interesting convergence results for the actual (random) parameter given by GCV and $C_L$ have been shown by Li (1985, 1986). Recently, Girard (1987, 1989) has proposed fast randomized versions of GCV and $C_L.$ The purpose of this paper is to show that the above convergence results also hold for these new methods.

We examine the decision theoretic estimated confidence approach proposed by Kiefer, Robinson and Berger, and focus on results under the frequentist validity constraint previously described by Brown and by Berger. Our main result is that the usual constant coverage probability estimator for the usual confidence set of a linear model is admissible under the frequentist validity constraint. Note that it is inadmissible without the frequentist validity constraint when the dimension is at least 5. The criterion of admissibility under the frequentist validity constraint is shown to be quite a reasonable one. Therefore the constant coverage probability estimator which has been widely used is justifiable from the post-data point of view.

As does Woodroofe, we consider a Bayesian sequential allocation between two treatments that incorporates a covariate. The goal is to maximize the total discounted expected reward from an infinite population of patients. Although our model is more general than Woodroofe's, we are able to duplicate his main result: The myopic rule is asymptotically optimal.

One observes $n$ data points, $(\mathbf{t}_i, Y_i),$ with the mean of $Y_i$, conditional on the regression function $f,$ equal to $f(\mathbf{t}_i).$ The prior distribution of the vector $\mathbf{f} = (f(\mathbf{t}_1), \ldots, f(\mathbf{t}_n))^t$ is unknown, but lies in a known class $\Omega.$ An estimator, $\hat{\mathbf{f}},$ of $\mathbf{f}$ is found which minimizes the maximum $E\|\hat{\mathbf{f}} - \mathbf{f}\|^2.$ The maximum is taken over all priors in $\Omega$ and the minimum is taken over linear estimators of $\mathbf{f}.$ Asymptotic properties of the estimator are studied in the case that $\mathbf{t}_i$ is one-dimensional and $\Omega$ is the set of priors for which $f$ is smooth.

Confidence intervals for the variance of a normal distribution with unknown mean are constructed which improve upon the usual shortest interval based on the sample variance alone. These intervals have guaranteed coverage probability uniformly greater than a predetermined value $1-\alpha$ and have uniformly shorter length. Using information relating the size of the samples mean to that of the sample variance, we smoothly shift the usual minimum length interval closer to zero, simultaneously bringing the endpoints closer to each other. The gains in coverage probability and expected length are also investigated numerically. Lastly, we examine the posterior probabilities of the intervals, quantities which can be used as post-data confidence reports.

An attempt is made to determine the logically consistent rules for selecting a vector from any feasible set defined by linear constraints, when either all $n$-vectors or those with positive components or the probability vectors are permissible. Some basic postulates are satisfied if and only if the selection rule is to minimize a certain function which, if a "prior guess" is available, is a measure of distance from the prior guess. Two further natural postulates restrict the permissible distances to the author's $f$-divergences and Bregman's divergences, respectively. As corollaries, axiomatic characterizations of the methods of least squares and minimum discrimination information are arrived at. Alternatively, the latter are also characterized by a postulate of composition consistency. As a special case, a derivation of the method of maximum entropy from a small set of natural axioms is obtained.

We describe a general approach to the comparison of two stochastic specifications over a collection of random quantities and then extend the comparison to collections of stochastic specifications. This comparison derives from the eigenstructure of the belief transform, which we construct in full generality for partially specified belief structures. We describe an application of the methodology, namely the comparison of hypotheses. Given various competing probabilistic specifications for a collection of observable quantities, we use the belief transform to separate those quantities for which the different models make very different predictions (and therefore which may be used to distinguish between the models) from those quantities for which the different models make similar predictions (and therefore which may be used to assess the suitability of the general class of models under consideration). Finally, we describe a particular application of hypothesis comparison which relates to the modelling of a collection of time series derived from an aluminum smelting process.

The RC model has been proposed as a model for ordered contingency tables. It involves parametric scores that are assigned to the rows and columns of the table so that these scores reflect the ordinality of the row and column categories. Efficient estimation of these parameters subject to order constraints remained an open problem mainly due to severe difficulties in computing these estimates and difficulties in deriving an appropriate asymptotic goodness-of-fit test. A nonstandard yet very simple algorithm is derived which produces the desired order-restricted maximum likelihood estimates with probability converging to 1. Testing the order-restricted RC model for fit is also discussed.

A class of redescending $M$-estimates of multivariate location and scatter are investigated. Sufficient conditions are given to ensure the existence and uniqueness of the estimates. These results are applied to the multivariate $t$-distribution with degrees of freedom $\nu \geq 1.$

We derive necessary and sufficient conditions in order that each mixture of a given family of probability densities have no more than $s$ modal intervals, with special attention to ordinary unimodality and strong unimodality of such mixtures.

Suppose $X_i$ are i.i.d. random variables taking values in $\mathscr{X}, \Theta$ is a parameter space and $y: \mathscr{X} \times \Theta \rightarrow \mathbf{R}$ is a map. Define the averages $S_n(y, \theta) = (1/n)\sum^n_{i = 1}y(X_i, \theta)$ and the truncated expectations $T_m(y, \theta) = \mathbf{E} \max(y(X_1, \theta), - m)$. Under the hypothesis of global dominance [i.e., $\mathbf{E} \sup_\Theta y(X_1, \theta) < \infty$] and some regularity conditions, the main result of the paper characterizes the asymptotic suprema of $S_n$ as follows. For any subset $G$ of $\Theta$, with probability 1, $\lim_{n \rightarrow \infty} \sup_{\theta \in G} S_N(y, \theta) = \lim_{m \rightarrow \infty} \sup_{\theta \in G} T_m(y, \theta).$ This has immediate application to consistency of $M$-estimators. In particular, under global dominance, maxima of $S_n$ must converge to the same limit as the maxima of $T_m(y, \theta)$ almost surely. We also obtain necessary and sufficient conditions for consistency resembling Huber's in the case of local dominance [i.e., each $\theta \in \Theta$ has a neighborhood $N(\theta)$ such that $\mathbf{E} \sup_{\psi \in N(\theta)}y(X_1, \psi) < \infty\rbrack.$ In this case there must exist a function $b(\theta) \geq 1$ such that $y/b$ is globally dominated and maxima of $T_m(y/b,\theta)$ converge.

Optimal and highly efficient two-dimensional designs are constructed for correlated errors on the torus and in the plane. The technique uses the method of differences to produce series of connectable planar squares. Efficiency calculations for planner versions of the torus designs show that the torus approximation is very satisfactory.

A Bayesian version of Elfving's theorem is given for the $\mathbf{c}$-optimality criterion with emphasis on the inherent geometry. Conditions under which a one-point design is Bayesian $\mathbf{c}$-optimum are described. The class of prior precision matrices $R$ for which the Bayesian $\mathbf{c}$-optimal designs are supported by the points of the classical $\mathbf{c}$-optimal design is characterized. It is proved that the Bayesian $\mathbf{c}$-optimal design, for large $n,$ is always supported by the same support points as the classical one if the number of support points and the number of regression functions are equal. Examples and a matrix analog are discussed.

This paper considers the problem of constructing approximate confidence intervals for functional parameters in the nonparametric case. The approach based on transformation theory is applied to improve standard confidence intervals. The accelerated bias-corrected percentile interval introduced by Efron relies on the existence of a normalizing transformation with bias and skewness corrections, although calculation does not require explicit knowledge of its functional form. We formally construct such a transformation and estimate bias and skewness correction factors for nonparametric situations. The resulting interval is shown to be second-order accurate. To this end Edgeworth expansions for the distributions of transformed statistics are derived, using the von Mises expansion.

This paper concerns the distributions used to construct confidence intervals for the regression function in a nonparametric setup. Some rates of convergence for the normal limit, its plug-in approach and the wild bootstrap are obtained conditionally on the explanatory variable $X$ and also unconditionally. The bound found for the wild bootstrap approximation is slightly better (by a factor $n^{-1/45}$) than the bounds given by the plug-in approach or the CLT for the conditional probability. On the contrary, the unconditional bounds present a different feature: the rate obtained when approximating by the CLT improves the one given by the plug-in approach by a factor of $n^{-8/45},$ while this last one performs better than the wild bootstrap approximation and the corresponding ratio is $n^{-1/45}.$ It should be mentioned that these two sequences, especially the last one, tend to zero at an extremely slow rate.

For the critical and subcritical Galton-Watson processes with immigration, it is shown that if the data were collected according to an appropriate stopping rule, the natural sequential estimator of the offspring mean $m$ is asymptotically normally distributed for each fixed $m \in (0, 1\rbrack.$ Furthermore, the sequential estimator is shown to be asymptotically normally distributed uniformly over a class of offspring distributions with $m \in (0, 1\rbrack$ bounded variance and satisfying a mild condition. These results are to be contrasted with the nonsequential approach where drastically different limit distributions are obtained for the two cases: (a) $m < 1$ (normal) and (b) $m = 1$ (nonnormal), thus leading to a singularity problem at $m = 1.$ The sequential approach proposed here avoids this singularity and unifies the two cases. The proof of the uniformity result is based on a uniform version of the well-known Anscombe's theorem.

We present a general statistical model for data coarsening, which includes as special cases rounded, heaped, censored, partially categorized and missing data. Formally, with coarse data, observations are made not in the sample space of the random variable of interest, but rather in its power set. Grouping is a special case in which the degree of coarsening is known and nonstochastic. We establish simple conditions under which the possible stochastic nature of the coarsening mechanism can be ignored when drawing Bayesian and likelihood inferences and thus the data can be validly treated as grouped data. The conditions are that the data be coarsened at random, a generalization of the condition missing at random, and that the parameters of the data and the coarsening process be distinct. Applications of the general model and the ignorability condition are illustrated in a numerical example and described briefly in a variety of special cases.

Test statistics which are almost determined by $o(n)$ tail order statistics are shown to provide tests of asymptotic relative efficiency 0 against the usual type of contiguous alternative. The result is applied to several goodness-of-fit tests: the variance weighted Kolmogorov-Smirnov statistic, the Kolmogorov-Smirnov statistic in the stabilized probability plot and the correlation coefficient in a $Q - Q$ plot for a variety of distributions with exponential tails.

In this paper we shall discuss the estimation of the mean of a normal distribution with variance 1. The main question in this work is the existence and computation of a least favorable distribution among all the prior distributions satisfying a given set of constraints. In the following we show that if this distribution is bounded from above on some even moment, then the least favorable distribution exists and it is either normal or discrete. The support of the discrete distribution function does not have any accumulation point. The least favorable distribution is normal if and only if the second moment is bounded from above, without any other relevant constraint. These theorems shed light on the James-Stein estimator as the minimax estimator for a prior with unknown bounded variance.

In this paper we study the Kaplan-Meier estimator in the case where survival and censoring times are not all i.i.d. We prove several results which are analogous to those shown by van Zuijlen in the complete data case.

Krewski and Rao consider inference for a (nonlinear) function of a vector of finite population means $\theta = g(\bar{Y}).$ For a sequence of finite populations with increasing number of strata, they demonstrate that $\hat{\theta} = g(\bar{y})$ is asymptotically normal, where $\bar{y}$ is the usual unbiased stratified estimator of $\bar{Y}.$ Additionally, they demonstrate that $(\hat{\theta} - \theta)/\upsilon^{1/2}(\hat{\theta})$ is asymptotically a standard normal distribution, where $\upsilon(\hat{\theta})$ is a variance estimator obtained using linearization, jackknife or balanced repeated replication (BRR) methods. In this note we extend their results to when the partial first derivatives $(g_1(\mu), g_2(\mu),\ldots, g_p(\mu)) \equiv 0,$ where $\mu$ is the limit of $\bar{Y}$ with increasing number of strata. We explore the asymptotic distribution of $(\hat{\theta} - \theta)/\upsilon^{1/2}(\hat{\theta})$ and show (1) that it is no longer normal and (2) that it depends upon which variance estimator is used. We describe an application of these results to hypothesis testing using complex survey data.