Let ℱ be a class of measurable functions f:S↦[0, 1] defined on a probability space (S, $\mathcal{A}$, P). Given a sample (X₁, …, X_n) of i.i.d. random variables taking values in S with common distribution P, let P_n denote the empirical measure based on (X₁, …, X_n). We study an empirical risk minimization problem P_nf→min , f∈ℱ. Given a solution f̂_n of this problem, the goal is to obtain very general upper bounds on its excess risk $$\mathcal{E}_{P}(\hat{f}_{n}):=P\hat{f}_{n}-\inf_{f\in \mathcal{F}}Pf,$$ expressed in terms of relevant geometric parameters of the class ℱ. Using concentration inequalities and other empirical processes tools, we obtain both distribution-dependent and data-dependent upper bounds on the excess risk that are of asymptotically correct order in many examples. The bounds involve localized sup-norms of empirical and Rademacher processes indexed by functions from the class. We use these bounds to develop model selection techniques in abstract risk minimization problems that can be applied to more specialized frameworks of regression and classification.

We propose a class of rank-based procedures for testing that the shape matrix V of an elliptical distribution (with unspecified center of symmetry, scale and radial density) has some fixed value V₀; this includes, for V₀={I}_k, the problem of testing for sphericity as an important particular case. The proposed tests are invariant under translations, monotone radial transformations, rotations and reflections with respect to the estimated center of symmetry. They are valid without any moment assumption. For adequately chosen scores, they are locally asymptotically maximin (in the Le Cam sense) at given radial densities. They are strictly distribution-free when the center of symmetry is specified, and asymptotically so when it must be estimated. The multivariate ranks used throughout are those of the distances—in the metric associated with the null value V₀ of the shape matrix—between the observations and the (estimated) center of the distribution. Local powers (against elliptical alternatives) and asymptotic relative efficiencies (AREs) are derived with respect to the adjusted Mauchly test (a modified version of the Gaussian likelihood ratio procedure proposed by Muirhead and Waternaux [Biometrika 67 (1980) 31–43]) or, equivalently, with respect to (an extension of ) the test for sphericity introduced by John [Biometrika 59 (1972) 169–173]. For Gaussian scores, these AREs are uniformly larger than one, irrespective of the actual radial density. Necessary and/or sufficient conditions for consistency under nonlocal, possibly nonelliptical alternatives are given. Finite sample performance is investigated via a Monte Carlo study.

A class of R-estimators based on the concepts of multivariate signed ranks and the optimal rank-based tests developed in Hallin and Paindaveine [Ann. Statist. 34 (2006) 2707–2756] is proposed for the estimation of the shape matrix of an elliptical distribution. These R-estimators are root-n consistent under any radial density g, without any moment assumptions, and semiparametrically efficient at some prespecified density f. When based on normal scores, they are uniformly more efficient than the traditional normal-theory estimator based on empirical covariance matrices (the asymptotic normality of which, moreover, requires finite moments of order four), irrespective of the actual underlying elliptical density. They rely on an original rank-based version of Le Cam’s one-step methodology which avoids the unpleasant nonparametric estimation of cross-information quantities that is generally required in the context of R-estimation. Although they are not strictly affine-equivariant, they are shown to be equivariant in a weak asymptotic sense. Simulations confirm their feasibility and excellent finite-sample performance.

This paper deals with nonparametric maximum likelihood estimation for Gaussian locally stationary processes. Our nonparametric MLE is constructed by minimizing a frequency domain likelihood over a class of functions. The asymptotic behavior of the resulting estimator is studied. The results depend on the richness of the class of functions. Both sieve estimation and global estimation are considered.

Our results apply, in particular, to estimation under shape constraints. As an example, autoregressive model fitting with a monotonic variance function is discussed in detail, including algorithmic considerations.

A key technical tool is the time-varying empirical spectral process indexed by functions. For this process, a Bernstein-type exponential inequality and a central limit theorem are derived. These results for empirical spectral processes are of independent interest.

Independent component analysis (ICA) has been widely used for blind source separation in many fields, such as brain imaging analysis, signal processing and telecommunication. Many statistical techniques based on M-estimates have been proposed for estimating the mixing matrix. Recently, several nonparametric methods have been developed, but in-depth analysis of asymptotic efficiency has not been available. We analyze ICA using semiparametric theories and propose a straightforward estimate based on the efficient score function by using B-spline approximations. The estimate is asymptotically efficient under moderate conditions and exhibits better performance than standard ICA methods in a variety of simulations.

In this paper, we introduce a family of robust estimates for the parametric and nonparametric components under a generalized partially linear model, where the data are modeled by y_i|(x_i, t_i)∼F(⋅, μ_i) with μ_i=H(η(t_i)+x_i^Tβ), for some known distribution function F and link function H. It is shown that the estimates of β are root-n consistent and asymptotically normal. Through a Monte Carlo study, the performance of these estimators is compared with that of the classical ones.

A depth-based rank sum statistic for multivariate data introduced by Liu and Singh [J. Amer. Statist. Assoc. 88 (1993) 252–260] as an extension of the Wilcoxon rank sum statistic for univariate data has been used in multivariate rank tests in quality control and in experimental studies. Those applications, however, are based on a conjectured limiting distribution, provided by Liu and Singh [J. Amer. Statist. Assoc. 88 (1993) 252–260]. The present paper proves the conjecture under general regularity conditions and, therefore, validates various applications of the rank sum statistic in the literature. The paper also shows that the corresponding rank sum tests can be more powerful than Hotelling’s T² test and some commonly used multivariate rank tests in detecting location-scale changes in multivariate distributions.

We study the rate of convergence of posterior distributions in density estimation problems for log-densities in periodic Sobolev classes characterized by a smoothness parameter p. The posterior expected density provides a nonparametric estimation procedure attaining the optimal minimax rate of convergence under Hellinger loss if the posterior distribution achieves the optimal rate over certain uniformity classes. A prior on the density class of interest is induced by a prior on the coefficients of the trigonometric series expansion of the log-density. We show that when p is known, the posterior distribution of a Gaussian prior achieves the optimal rate provided the prior variances die off sufficiently rapidly. For a mixture of normal distributions, the mixing weights on the dimension of the exponential family are assumed to be bounded below by an exponentially decreasing sequence. To avoid the use of infinite bases, we develop priors that cut off the series at a sample-size-dependent truncation point. When the degree of smoothness is unknown, a finite mixture of normal priors indexed by the smoothness parameter, which is also assigned a prior, produces the best rate. A rate-adaptive estimator is derived.

Given a random sample from a distribution with density function that depends on an unknown parameter θ, we are interested in accurately estimating the true parametric density function at a future observation from the same distribution. The asymptotic risk of Bayes predictive density estimates with Kullback–Leibler loss function D(f_θ||{f̂})=∫f_θ log(f_θ/f̂) is used to examine various ways of choosing prior distributions; the principal type of choice studied is minimax. We seek asymptotically least favorable predictive densities for which the corresponding asymptotic risk is minimax. A result resembling Stein’s paradox for estimating normal means by maximum likelihood holds for the uniform prior in the multivariate location family case: when the dimensionality of the model is at least three, the Jeffreys prior is minimax, though inadmissible. The Jeffreys prior is both admissible and minimax for one- and two-dimensional location problems.

We consider a common-components model for multivariate fractional cointegration, in which the s≥1 components have different memory parameters. The cointegrating rank may exceed 1. We decompose the true cointegrating vectors into orthogonal fractional cointegrating subspaces such that vectors from distinct subspaces yield cointegrating errors with distinct memory parameters. We estimate each cointegrating subspace separately, using appropriate sets of eigenvectors of an averaged periodogram matrix of tapered, differenced observations, based on the first m Fourier frequencies, with m fixed. The angle between the true and estimated cointegrating subspaces is o_p(1). We use the cointegrating residuals corresponding to an estimated cointegrating vector to obtain a consistent and asymptotically normal estimate of the memory parameter for the given cointegrating subspace, using a univariate Gaussian semiparametric estimator with a bandwidth that tends to ∞ more slowly than n. We use these estimates to test for fractional cointegration and to consistently identify the cointegrating subspaces.

We apply FDR thresholding to a non-Gaussian vector whose coordinates X_i, i=1, …, n, are independent exponential with individual means μ_i. The vector μ=(μ_i) is thought to be sparse, with most coordinates 1 but a small fraction significantly larger than 1; roughly, most coordinates are simply ‘noise,’ but a small fraction contain ‘signal.’ We measure risk by per-coordinate mean-squared error in recovering log(μ_i), and study minimax estimation over parameter spaces defined by constraints on the per-coordinate p-norm of log(μ_i), $\frac{1}{n}\sum_{i=1}^{n}\,\log^{p}(\mu_{i})\leq \eta^{p}$.

We show for large n and small η that FDR thresholding can be nearly minimax. The FDR control parameter 0<q<1 plays an important role: when q≤1/2, the FDR estimator is nearly minimax, while choosing a fixed q>1/2 prevents near minimaxity.

These conclusions mirror those found in the Gaussian case in Abramovich et al. [Ann. Statist. 34 (2006) 584–653]. The techniques developed here seem applicable to a wide range of other distributional assumptions, other loss measures and non-i.i.d. dependency structures.

This paper introduces a version of empirical likelihood based on the periodogram and spectral estimating equations. This formulation handles dependent data through a data transformation (i.e., a Fourier transform) and is developed in terms of the spectral distribution rather than a time domain probability distribution. The asymptotic properties of frequency domain empirical likelihood are studied for linear time processes exhibiting both short- and long-range dependence. The method results in likelihood ratios which can be used to build nonparametric, asymptotically correct confidence regions for a class of normalized (or ratio) spectral parameters, including autocorrelations. Maximum empirical likelihood estimators are possible, as well as tests of spectral moment conditions. The methodology can be applied to several inference problems such as Whittle estimation and goodness-of-fit testing.