Regression models to relate a scalar Y to a functional predictor X(t) are becoming increasingly common. Work in this area has concentrated on estimating a coefficient function, β(t), with Y related to X(t) through ∫β(t)X(t) dt. Regions where β(t)≠0 correspond to places where there is a relationship between X(t) and Y. Alternatively, points where β(t)=0 indicate no relationship. Hence, for interpretation purposes, it is desirable for a regression procedure to be capable of producing estimates of β(t) that are exactly zero over regions with no apparent relationship and have simple structures over the remaining regions. Unfortunately, most fitting procedures result in an estimate for β(t) that is rarely exactly zero and has unnatural wiggles making the curve hard to interpret. In this article we introduce a new approach which uses variable selection ideas, applied to various derivatives of β(t), to produce estimates that are both interpretable, flexible and accurate. We call our method “Functional Linear Regression That’s Interpretable” (FLiRTI) and demonstrate it on simulated and real-world data sets. In addition, non-asymptotic theoretical bounds on the estimation error are presented. The bounds provide strong theoretical motivation for our approach.

We derive sharp performance bounds for least squares regression with L₁ regularization from parameter estimation accuracy and feature selection quality perspectives. The main result proved for L₁ regularization extends a similar result in [Ann. Statist. 35 (2007) 2313–2351] for the Dantzig selector. It gives an affirmative answer to an open question in [Ann. Statist. 35 (2007) 2358–2364]. Moreover, the result leads to an extended view of feature selection that allows less restrictive conditions than some recent work. Based on the theoretical insights, a novel two-stage L₁-regularization procedure with selective penalization is analyzed. It is shown that if the target parameter vector can be decomposed as the sum of a sparse parameter vector with large coefficients and another less sparse vector with relatively small coefficients, then the two-stage procedure can lead to improved performance.

We consider the fundamental problem of estimating the mean of a vector y=Xβ+z, where X is an n×p design matrix in which one can have far more variables than observations, and z is a stochastic error term—the so-called “p>n” setup. When β is sparse, or, more generally, when there is a sparse subset of covariates providing a close approximation to the unknown mean vector, we ask whether or not it is possible to accurately estimate Xβ using a computationally tractable algorithm.

We show that, in a surprisingly wide range of situations, the lasso happens to nearly select the best subset of variables. Quantitatively speaking, we prove that solving a simple quadratic program achieves a squared error within a logarithmic factor of the ideal mean squared error that one would achieve with an oracle supplying perfect information about which variables should and should not be included in the model. Interestingly, our results describe the average performance of the lasso; that is, the performance one can expect in an vast majority of cases where Xβ is a sparse or nearly sparse superposition of variables, but not in all cases.

Our results are nonasymptotic and widely applicable, since they simply require that pairs of predictor variables are not too collinear.

This paper explores the following question: what kind of statistical guarantees can be given when doing variable selection in high-dimensional models? In particular, we look at the error rates and power of some multi-stage regression methods. In the first stage we fit a set of candidate models. In the second stage we select one model by cross-validation. In the third stage we use hypothesis testing to eliminate some variables. We refer to the first two stages as “screening” and the last stage as “cleaning.” We consider three screening methods: the lasso, marginal regression, and forward stepwise regression. Our method gives consistent variable selection under certain conditions.

We define a generalized index of jump activity, propose estimators of that index for a discretely sampled process and derive the estimators’ properties. These estimators are applicable despite the presence of Brownian volatility in the process, which makes it more challenging to infer the characteristics of the small, infinite activity jumps. When the method is applied to high frequency stock returns, we find evidence of infinitely active jumps in the data and estimate their index of activity.

We consider regression models with parametric (linear or nonlinear) regression function and allow responses to be “missing at random.” We assume that the errors have mean zero and are independent of the covariates. In order to estimate expectations of functions of covariate and response we use a fully imputed estimator, namely an empirical estimator based on estimators of conditional expectations given the covariate. We exploit the independence of covariates and errors by writing the conditional expectations as unconditional expectations, which can now be estimated by empirical plug-in estimators. The mean zero constraint on the error distribution is exploited by adding suitable residual-based weights. We prove that the estimator is efficient (in the sense of Hájek and Le Cam) if an efficient estimator of the parameter is used. Our results give rise to new efficient estimators of smooth transformations of expectations. Estimation of the mean response is discussed as a special (degenerate) case.

The problem we concentrate on is as follows: given (1) a convex compact set X in ℝⁿ, an affine mapping x↦A(x), a parametric family {p_μ(⋅)} of probability densities and (2) N i.i.d. observations of the random variable ω, distributed with the density p_A(x)(⋅) for some (unknown) x∈X, estimate the value g^Tx of a given linear form at x.

For several families {p_μ(⋅)} with no additional assumptions on X and A, we develop computationally efficient estimation routines which are minimax optimal, within an absolute constant factor. We then apply these routines to recovering x itself in the Euclidean norm.

We consider the problem of estimating a density f_X using a sample Y₁, …, Y_n from f_Y=f_X⋆f_ε, where f_ε is an unknown density. We assume that an additional sample ε₁, …, ε_m from f_ε is observed. Estimators of f_X and its derivatives are constructed by using nonparametric estimators of f_Y and f_ε and by applying a spectral cut-off in the Fourier domain. We derive the rate of convergence of the estimators in case of a known and unknown error density f_ε, where it is assumed that f_X satisfies a polynomial, logarithmic or general source condition. It is shown that the proposed estimators are asymptotically optimal in a minimax sense in the models with known or unknown error density, if the density f_X belongs to a Sobolev space $H_{\mathcal{p}}$ and f_ε is ordinary smooth or supersmooth.

This paper proves fixed domain asymptotic results for estimating a smooth invertible transformation f: ℝ²→ℝ² when observing the deformed random field Z○f on a dense grid in a bounded, simply connected domain Ω, where Z is assumed to be an isotropic Gaussian random field on ℝ². The estimate f̂ is constructed on a simply connected domain U, such that U̅⊂Ω and is defined using kernel smoothed quadratic variations, Bergman projections and results from quasiconformal theory. We show, under mild assumptions on the random field Z and the deformation f, that f̂→R_θf+c uniformly on compact subsets of U with probability one as the grid spacing goes to zero, where R_θ is an unidentifiable rotation and c is an unidentifiable translation.

We consider a class of doubly weighted rank-based estimating methods for the transformation (or accelerated failure time) model with missing data as arise, for example, in case-cohort studies. The weights considered may not be predictable as required in a martingale stochastic process formulation. We treat the general problem as a semiparametric estimating equation problem and provide proofs of asymptotic properties for the weighted estimators, with either true weights or estimated weights, by using empirical process theory where martingale theory may fail. Simulations show that the outcome-dependent weighted method works well for finite samples in case-cohort studies and improves efficiency compared to methods based on predictable weights. Further, it is seen that the method is even more efficient when estimated weights are used, as is commonly the case in the missing data literature. The Gehan censored data Wilcoxon weights are found to be surprisingly efficient in a wide class of problems.

In the analysis of cluster data, the regression coefficients are frequently assumed to be the same across all clusters. This hampers the ability to study the varying impacts of factors on each cluster. In this paper, a semiparametric model is introduced to account for varying impacts of factors over clusters by using cluster-level covariates. It achieves the parsimony of parametrization and allows the explorations of nonlinear interactions. The random effect in the semiparametric model also accounts for within-cluster correlation. Local, linear-based estimation procedure is proposed for estimating functional coefficients, residual variance and within-cluster correlation matrix. The asymptotic properties of the proposed estimators are established, and the method for constructing simultaneous confidence bands are proposed and studied. In addition, relevant hypothesis testing problems are addressed. Simulation studies are carried out to demonstrate the methodological power of the proposed methods in the finite sample. The proposed model and methods are used to analyse the second birth interval in Bangladesh, leading to some interesting findings.

We consider tests of hypotheses when the parameters are not identifiable under the null in semiparametric models, where regularity conditions for profile likelihood theory fail. Exponential average tests based on integrated profile likelihood are constructed and shown to be asymptotically optimal under a weighted average power criterion with respect to a prior on the nonidentifiable aspect of the model. These results extend existing results for parametric models, which involve more restrictive assumptions on the form of the alternative than do our results. Moreover, the proposed tests accommodate models with infinite dimensional nuisance parameters which either may not be identifiable or may not be estimable at the usual parametric rate. Examples include tests of the presence of a change-point in the Cox model with current status data and tests of regression parameters in odds-rate models with right censored data. Optimal tests have not previously been studied for these scenarios. We study the asymptotic distribution of the proposed tests under the null, fixed contiguous alternatives and random contiguous alternatives. We also propose a weighted bootstrap procedure for computing the critical values of the test statistics. The optimal tests perform well in simulation studies, where they may exhibit improved power over alternative tests.

In this paper we are interested in empirical likelihood (EL) as a method of estimation, and we address the following two problems: (1) selecting among various empirical discrepancies in an EL framework and (2) demonstrating that EL has a well-defined probabilistic interpretation that would justify its use in a Bayesian context. Using the large deviations approach, a Bayesian law of large numbers is developed that implies that EL and the Bayesian maximum a posteriori probability (MAP) estimators are consistent under misspecification and that EL can be viewed as an asymptotic form of MAP. Estimators based on other empirical discrepancies are, in general, inconsistent under misspecification.

Bayesian and frequentist methods differ in many aspects, but share some basic optimality properties. In practice, there are situations in which one of the methods is more preferred by some criteria. We consider the case of inference about a set of multiple parameters, which can be divided into two disjoint subsets. On one set, a frequentist method may be favored and on the other, the Bayesian. This motivates a joint estimation procedure in which some of the parameters are estimated Bayesian, and the rest by the maximum-likelihood estimator in the same parametric model, and thus keep the strengths of both the methods and avoid their weaknesses. Such a hybrid procedure gives us more flexibility in achieving overall inference advantages. We study the consistency and high-order asymptotic behavior of the proposed estimator, and illustrate its application. Also, the results imply a new method for constructing objective prior.

Mixture models have received considerable attention recently and Newton [Sankhyā Ser. A 64 (2002) 306–322] proposed a fast recursive algorithm for estimating a mixing distribution. We prove almost sure consistency of this recursive estimate in the weak topology under mild conditions on the family of densities being mixed. This recursive estimate depends on the data ordering and a permutation-invariant modification is proposed, which is an average of the original over permutations of the data sequence. A Rao–Blackwell argument is used to prove consistency in probability of this alternative estimate. Several simulations are presented, comparing the finite-sample performance of the recursive estimate and a Monte Carlo approximation to the permutation-invariant alternative along with that of the nonparametric maximum likelihood estimate and a nonparametric Bayes estimate.

Normal mixture distributions are arguably the most important mixture models, and also the most technically challenging. The likelihood function of the normal mixture model is unbounded based on a set of random samples, unless an artificial bound is placed on its component variance parameter. Moreover, the model is not strongly identifiable so it is hard to differentiate between over dispersion caused by the presence of a mixture and that caused by a large variance, and it has infinite Fisher information with respect to mixing proportions. There has been extensive research on finite normal mixture models, but much of it addresses merely consistency of the point estimation or useful practical procedures, and many results require undesirable restrictions on the parameter space. We show that an EM-test for homogeneity is effective at overcoming many challenges in the context of finite normal mixtures. We find that the limiting distribution of the EM-test is a simple function of the 0.5χ₀²+0.5χ₁² and χ₁² distributions when the mixing variances are equal but unknown and the χ₂² when variances are unequal and unknown. Simulations show that the limiting distributions approximate the finite sample distribution satisfactorily. Two genetic examples are used to illustrate the application of the EM-test.

Response-adaptive randomization has recently attracted a lot of attention in the literature. In this paper, we propose a new and simple family of response-adaptive randomization procedures that attain the Cramer–Rao lower bounds on the allocation variances for any allocation proportions, including optimal allocation proportions. The allocation probability functions of proposed procedures are discontinuous. The existing large sample theory for adaptive designs relies on Taylor expansions of the allocation probability functions, which do not apply to nondifferentiable cases. In the present paper, we study stopping times of stochastic processes to establish the asymptotic efficiency results. Furthermore, we demonstrate our proposal through examples, simulations and a discussion on the relationship with earlier works, including Efron’s biased coin design.

The research of developing a general methodology for the construction of good nonregular designs has been very active in the last decade. Recent research by Xu and Wong [Statist. Sinica 17 (2007) 1191–1213] suggested a new class of nonregular designs constructed from quaternary codes. This paper explores the properties and uses of quaternary codes toward the construction of quarter-fraction nonregular designs. Some theoretical results are obtained regarding the aliasing structure of such designs. Optimal designs are constructed under the maximum resolution, minimum aberration and maximum projectivity criteria. These designs often have larger generalized resolution and larger projectivity than regular designs of the same size. It is further shown that some of these designs have generalized minimum aberration and maximum projectivity among all possible designs.

This paper establishes a necessary and sufficient condition for the asymptotic normality of the nonparametric estimator of sample coverage proposed by Good [Biometrica 40 (1953) 237–264]. This new necessary and sufficient condition extends the validity of the asymptotic normality beyond the previously proven cases.