The purpose of model selection algorithms such as All Subsets, Forward Selection and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be applied. Typically we have available a large collection of possible covariates from which we hope to select a parsimonious set for the efficient prediction of a response variable. Least Angle Regression (LARS), a new model selection algorithm, is a useful and less greedy version of traditional forward selection methods. Three main properties are derived: (1) A simple modification of the LARS algorithm implements the Lasso, an attractive version of ordinary least squares that constrains the sum of the absolute regression coefficients; the LARS modification calculates all possible Lasso estimates for a given problem, using an order of magnitude less computer time than previous methods. (2) A different LARS modification efficiently implements Forward Stagewise linear regression, another promising new model selection method; this connection explains the similar numerical results previously observed for the Lasso and Stagewise, and helps us understand the properties of both methods, which are seen as constrained versions of the simpler LARS algorithm. (3) A simple approximation for the degrees of freedom of a LARS estimate is available, from which we derive a C_p estimate of prediction error; this allows a principled choice among the range of possible LARS estimates. LARS and its variants are computationally efficient: the paper describes a publicly available algorithm that requires only the same order of magnitude of computational effort as ordinary least squares applied to the full set of covariates.

We describe here a framework for a certain class of multiscale likelihood factorizations wherein, in analogy to a wavelet decomposition of an L² function, a given likelihood function has an alternative representation as a product of conditional densities reflecting information in both the data and the parameter vector localized in position and scale. The framework is developed as a set of sufficient conditions for the existence of such factorizations, formulated in analogy to those underlying a standard multiresolution analysis for wavelets, and hence can be viewed as a multiresolution analysis for likelihoods. We then consider the use of these factorizations in the task of nonparametric, complexity penalized likelihood estimation. We study the risk properties of certain thresholding and partitioning estimators, and demonstrate their adaptivity and near-optimality, in a minimax sense over a broad range of function spaces, based on squared Hellinger distance as a loss function. In particular, our results provide an illustration of how properties of classical wavelet-based estimators can be obtained in a single, unified framework that includes models for continuous, count and categorical data types.

Starting from the observation of an ℝⁿ-Gaussian vector of mean f and covariance matrix σ²I_n (I_n is the identity matrix), we propose a method for building a Euclidean confidence ball around f, with prescribed probability of coverage. For each n, we describe its nonasymptotic property and show its optimality with respect to some criteria.

The minimax theory for estimating linear functionals is extended to the case of a finite union of convex parameter spaces. Upper and lower bounds for the minimax risk can still be described in terms of a modulus of continuity. However in contrast to the theory for convex parameter spaces rate optimal procedures are often required to be nonlinear. A construction of such nonlinear procedures is given. The results developed in this paper have important applications to the theory of adaptation.

This paper offers a new approach for estimating and forecasting the volatility of financial time series. No assumption is made about the parametric form of the processes. On the contrary, we only suppose that the volatility can be approximated by a constant over some interval. In such a framework, the main problem consists of filtering this interval of time homogeneity; then the estimate of the volatility can be simply obtained by local averaging. We construct a locally adaptive volatility estimate (LAVE) which can perform this task and investigate it both from the theoretical point of view and through Monte Carlo simulations. Finally, the LAVE procedure is applied to a data set of nine exchange rates and a comparison with a standard GARCH model is also provided. Both models appear to be capable of explaining many of the features of the data; nevertheless, the new approach seems to be superior to the GARCH method as far as the out-of-sample results are concerned.

Suppose we observe an invertible linear process with independent mean-zero innovations and with coefficients depending on a finite-dimensional parameter, and we want to estimate the expectation of some function under the stationary distribution of the process. The usual estimator would be the empirical estimator. It can be improved using the fact that the innovations are centered. We construct an even better estimator using the representation of the observations as infinite-order moving averages of the innovations. Then the expectation of the function under the stationary distribution can be written as the expectation under the distribution of an infinite series in terms of the innovations, and it can be estimated by a U-statistic of increasing order (also called an “infinite-order U-statistic”) in terms of the estimated innovations. The estimator can be further improved using the fact that the innovations are centered. This improved estimator is optimal if the coefficients of the linear process are estimated optimally. The variance reduction of our estimator over the empirical estimator can be considerable.

We propose a class of estimators for the parameters of a GARCH(p,q) sequence. We show that our estimators are consistent and asymptotically normal under mild conditions. The quasi-maximum likelihood and the likelihood estimators are discussed in detail. We show that the maximum likelihood estimator is optimal. If the tail of the distribution of the innovations is polynomial, even a quasi-maximum likelihood estimator based on exponential density performs better than the standard normal density-based quasi-likelihood estimator of Lee and Hansen and Lumsdaine.

Asymptotic properties of the local Whittle estimator in the nonstationary case (d>½) are explored. For ½<d≤1, the estimator is shown to be consistent, and its limit distribution and the rate of convergence depend on the value of d. For d=1, the limit distribution is mixed normal. For d>1 and when the process has a polynomial trend of order α>½, the estimator is shown to be inconsistent and to converge in probability to unity.

We consider the problem of choosing the optimal (in the sense of mean-squared prediction error) multistep predictor for an autoregressive (AR) process of finite but unknown order. If a working AR model (which is possibly misspecified) is adopted for multistep predictions, then two competing types of multistep predictors (i.e., plug-in and direct predictors) can be obtained from this model. We provide some interesting examples to show that when both plug-in and direct predictors are considered, the optimal multistep prediction results cannot be guaranteed by correctly identifying the underlying model”s order. This finding challenges the traditional model (order) selection criteria, which usually aim to choose the order of the true model. A new prediction selection criterion, which attempts to seek the best combination of the prediction order and the prediction method, is proposed to rectify this difficulty. When the underlying model is stationary, the validity of the proposed criterion is justified theoretically. To obtain this result, asymptotic properties of accumulated squares of multistep prediction errors are investigated. In addition to overcoming the above difficulty, some other advantages of the proposed criterion are also mentioned.

We derive information bounds for the regression parameters in Cox models when data are missing at random. These calculations are of interest for understanding the behavior of efficient estimation in case-cohort designs, a type of two-phase design often used in cohort studies. The derivations make use of key lemmas appearing in Robins, Rotnitzky and Zhao [J. Amer. Statist. Assoc. 89 (1994) 846–866] and Robins, Hsieh and Newey [J. Roy. Statist. Soc. Ser. B 57 (1995) 409–424], but in a form suited for our purposes here. We begin by summarizing the results of Robins, Rotnitzky and Zhao in a form that leads directly to the projection method which will be of use for our model of interest. We then proceed to derive new information bounds for the regression parameters of the Cox model with data Missing At Random (MAR). In the final section we exemplify our calculations with several models of interest in cohort studies, including an i.i.d. version of the classical case-cohort design of Prentice [Biometrika 73 (1986) 1–11] and Self and Prentice [Ann. Statist. 16 (1988) 64–81].

This paper provides further insight into the key concept of missing at random (MAR) in incomplete data analysis. Following the usual selection modelling approach we envisage two models with separable parameters: a model for the response of interest and a model for the missing data mechanism (MDM). If the response model is given by a complete density family, then frequentist inference from the likelihood function ignoring the MDM is valid if and only if the MDM is MAR. This necessary and sufficient condition also holds more generally for models for coarse data, such as censoring. Examples are given to show the necessity of the completeness of the underlying model for this equivalence to hold.

Finite sample properties of multiple imputation estimators under the linear regression model are studied. The exact bias of the multiple imputation variance estimator is presented. A method of reducing the bias is presented and simulation is used to make comparisons. We also show that the suggested method can be used for a general class of linear estimators.

We consider Gibbs and block Gibbs samplers for a Bayesian hierarchical version of the one-way random effects model. Drift and minorization conditions are established for the underlying Markov chains. The drift and minorization are used in conjunction with results from J. S. Rosenthal [J. Amer. Statist. Assoc. 90 (1995) 558–566] and G. O. Roberts and R. L. Tweedie [Stochastic Process. Appl. 80 (1999) 211–229] to construct analytical upper bounds on the distance to stationarity. These lead to upper bounds on the amount of burn-in that is required to get the chain within a prespecified (total variation) distance of the stationary distribution. The results are illustrated with a numerical example.

The term “empirical predictor” refers to a two-stage predictor of a linear combination of fixed and random effects. In the first stage, a predictor is obtained but it involves unknown parameters; thus, in the second stage, the unknown parameters are replaced by their estimators. In this paper, we consider mean squared errors (MSE) of empirical predictors under a general setup, where ML or REML estimators are used for the second stage. We obtain second-order approximation to the MSE as well as an estimator of the MSE correct to the same order. The general results are applied to mixed linear models to obtain a second-order approximation to the MSE of the empirical best linear unbiased predictor (EBLUP) of a linear mixed effect and an estimator of the MSE of EBLUP whose bias is correct to second order. The general mixed linear model includes the mixed ANOVA model and the longitudinal model as special cases.