Support vector machines (SVMs) using Gaussian kernels are one of the standard and state-of-the-art learning algorithms. In this work, we establish new oracle inequalities for such SVMs when applied to either least squares or conditional quantile regression. With the help of these oracle inequalities we then derive learning rates that are (essentially) minmax optimal under standard smoothness assumptions on the target function. We further utilize the oracle inequalities to show that these learning rates can be adaptively achieved by a simple data-dependent parameter selection method that splits the data set into a training and a validation set.

We obtain an asymptotic expansion for the null distribution function of the gradient statistic for testing composite null hypotheses in the presence of nuisance parameters. The expansion is derived using a Bayesian route based on the shrinkage argument described in [10]. Using this expansion, we propose a Bartlett-type corrected gradient statistic with chi-square distribution up to an error of order $o(n^{-1})$ under the null hypothesis. Further, we also use the expansion to modify the percentage points of the large sample reference chi-square distribution. Monte Carlo simulation experiments and various examples are presented and discussed.

Random probability vectors are of great interest especially in view of their application to statistical inference. Indeed, they can be used for identifying the de Finetti mixing measure in the representation of the law of a partially exchangeable array of random elements taking values in a separable and complete metric space. In this paper we describe the construction of a vector of Dirichlet processes based on the normalization of an exchangeable vector of completely random measures that are jointly infinitely divisible. After deducing the form of the multivariate Laplace exponent associated to the vector of the gamma completely random measures, we analyze some of their distributional properties. Our attention particularly focuses on the dependence structure and the specific partition probability function induced by the proposed vector.

We consider the problem of constructing static graphical representations of a large number of interval estimates. Because of clutter, traditional graphical summaries are visually ineffective for representing more then a few intervals. The Cartesian displays introduced in this article overcome the limitations stemming from visual clutter and can represent effectively very many intervals. The construction of a Cartesian display for symmetric intervals is first presented in the context of a multiple comparisons application. Generalizations involving the representation of asymmetric intervals are then introduced and used to summarize aspects of the posterior distributions of numerous parameter contrasts in two hierarchical Bayes models.

We consider the problem of testing significance of predictors in multivariate nonparametric quantile regression. A stochastic process is proposed, which is based on a comparison of the responses with a nonparametric quantile regression estimate under the null hypothesis. It is demonstrated that under the null hypothesis this process converges weakly to a centered Gaussian process and the asymptotic properties of the test under fixed and local alternatives are also discussed. In particular we show, that - in contrast to the nonparametric approach based on estimation of $L^{2}$-distances - the new test is able to detect local alternatives which converge to the null hypothesis with any rate $a_{n}\to 0$ such that $a_{n}\sqrt{n}\to\infty$ (here $n$ denotes the sample size). We also present a small simulation study illustrating the finite sample properties of a bootstrap version of the corresponding Kolmogorov-Smirnov test.

We consider the problem of testing a particular type of composite null hypothesis under a nonparametric multivariate regression model. For a given quadratic functional $Q$, the null hypothesis states that the regression function $f$ satisfies the constraint $Q[f]=0$, while the alternative corresponds to the functions for which $Q[f]$ is bounded away from zero. On the one hand, we provide minimax rates of testing and the exact separation constants, along with a sharp-optimal testing procedure, for diagonal and nonnegative quadratic functionals. We consider smoothness classes of ellipsoidal form and check that our conditions are fulfilled in the particular case of ellipsoids corresponding to anisotropic Sobolev classes. In this case, we present a closed form of the minimax rate and the separation constant. On the other hand, minimax rates for quadratic functionals which are neither positive nor negative makes appear two different regimes: “regular” and “irregular”. In the “regular" case, the minimax rate is equal to $n^{-1/4}$ while in the “irregular” case, the rate depends on the smoothness class and is slower than in the “regular” case. We apply this to the problem of testing the equality of Sobolev norms of two functions observed in noisy environments.

We consider the estimation of the value of a linear functional of the slope parameter in functional linear regression, where scalar responses are modeled in dependence of random functions. The theory in this paper covers in particular point-wise estimation as well as the estimation of weighted averages of the slope parameter. We propose a plug-in estimator which is based on a dimension reduction technique and additional thresholding. It is shown that this estimator is consistent under mild assumptions. We derive a lower bound for the maximal mean squared error of any estimator over a certain ellipsoid of slope parameters and a certain class of covariance operators associated with the regressor. It is shown that the proposed estimator attains this lower bound up to a constant and hence it is minimax optimal. Our results are appropriate to discuss a wide range of possible regressors, slope parameters and functionals. They are illustrated by considering the point-wise estimation of the slope parameter or its derivatives and its average value over a given interval.

We study Bayes procedures for nonparametric regression problems with Gaussian errors, giving conditions under which a Bernstein–von Mises result holds for the marginal posterior distribution of the error standard deviation. We apply our general results to show that a single Bayes procedure using a hierarchical spline-based prior on the regression function and an independent prior on the error variance, can simultaneously achieve adaptive, rate-optimal estimation of a smooth, multivariate regression function and efficient, $\sqrt{n}$-consistent estimation of the error standard deviation.

In linear mixed effects models, the covariance matrix of the response is modeled as the sum of two matrices: the product of the covariance matrix of the random effects with the associated design matrix, and the covariance matrix of the residual error. Building a linear mixed model usually involves selection of the parametrized covariance matrix structures for the random effects and the residual error. However, even if the covariance matrix of the response is not over-parametrized, some specifications of covariance structures can result in the non-identifiability of parameters. When fitting such models, software may or may not indicate a problem with model identifiability. Consequently, it is useful to have a way to check if a model is identifiable which does not rely on the software output. We derive conditions for identifiability of the covariance parameters of the response and study commonly used covariance structures. The derived conditions only rely on the covariance structures being used and properties of the design matrix associated with the random effects and are easy to check.

The present paper is about estimation and prediction in high-dimensional additive models under a sparsity assumption ($p\gg n$ paradigm). A PAC-Bayesian strategy is investigated, delivering oracle inequalities in probability. The implementation is performed through recent outcomes in high-dimensional MCMC algorithms, and the performance of our method is assessed on simulated data.

Respondent-driven sampling (RDS) is a widely used method for generating chain-referral samples from hidden populations. It is an extension of the snowball sampling method and can, given that some assumptions are met, generate unbiased population estimates. One key assumption, not likely to be met, is that the acquaintance network in which the recruitment process takes place is undirected, meaning that all recruiters should have the potential to be recruited by the person they recruit. Using a mean-field approach, we develop an estimator which is based on prior information about the average indegrees of estimated variables. When the indegree is known, such as for RDS studies over internet social networks, the estimator can greatly reduce estimate error and bias as compared with current methods; when the indegree is not known, which is most common for interview-based RDS studies, the estimator can through sensitivity analysis be used as a tool to account for uncertainties of network directedness and error in self-reported degree data. The performance of the new estimator, together with previous RDS estimators, is investigated thoroughly by simulations on networks with varying structures. We have applied the new estimator on an empirical RDS study for injecting drug users in New York City.

In this paper, motivated by an important problem in evolutionary biology, we develop two sieve type estimators for distributions that are mixtures of a finite number of discrete atoms and continuous distributions under the framework of measurement error models. While there is a large literature on deconvolution problems, only two articles have previously addressed the problem taken up in our article, and they use relatively standard Fourier deconvolution. As a result the estimators suggested in those two articles are degraded seriously by boundary effects and negativity. A major contribution of our article is correct handling of boundary effects; our method is asymptotically unbiased at the boundaries, and also is guaranteed to be nonnegative. We use roughness penalization to improve the smoothness of the resulting estimator and reduce the estimation variance. We illustrate the performance of the proposed estimators via our real driving application in evolutionary biology and two simulation studies. Furthermore, we establish asymptotic properties of the proposed estimators.

Principal curves are parameterized curves passing “through the middle” of a data cloud. These objects constitute a way of generalization of the notion of first principal component in Principal Component Analysis. Several definitions of principal curve have been proposed, one of which can be expressed as a least-square minimization problem. In the present paper, adopting this definition, we study a Gaussian model selection method for choosing the length of the principal curve, in order to avoid interpolation, and obtain a related oracle-type inequality. The proposed method is practically implemented and illustrated on cartography problems.

We establish global rates of convergence of the Maximum Likelihood Estimator (MLE) of a multivariate distribution function on ${\mathbb{R}}^{d}$ in the case of (one type of) “interval censored” data. The main finding is that the rate of convergence of the MLE in the Hellinger metric is no worse than $n^{-1/3}(\log n)^{\gamma}$ for $\gamma =(5d-4)/6$.

This paper establishes the consistency of the minimum description length (MDL) model selection procedure by [10, 11] for a class of non-stationary time series models. We consider a time series model in which the observations are viewed as coming from stationary segments. In other words, the data are assumed to come from a general time series model in which the parameters change at break-points. Each of these segments is modeled by a pre-specified family of parametric stationary time series models. [10, 11] formulated the above problem and used the minimum description length (MDL) principle to estimate the number of break-points, the location of the break-points, the order of the parametric model and the parameter values in each of the segments. The procedure performed well on a variety of examples. In this paper we show consistency of their minimal MDL model selection procedure under general regularity conditions on the likelihood function. Results about the rate of convergence of the break-point-location estimator are also given. Applications are considered for detecting changes in independent random variables, and in ARMA and GARCH processes.

The Pearson test statistic is constructed by partitioning the data into bins and computing the difference between the observed and expected counts in these bins. If the maximum likelihood estimator (MLE) of the original data is used, the statistic generally does not follow a chi-squared distribution or any explicit distribution. We propose a bootstrap-based modification of the Pearson test statistic to recover the chi-squared distribution. We compute the observed and expected counts in the partitioned bins by using the MLE obtained from a bootstrap sample. This bootstrap-sample MLE adjusts exactly the right amount of randomness to the test statistic, and recovers the chi-squared distribution. The bootstrap chi-squared test is easy to implement, as it only requires fitting exactly the same model to the bootstrap data to obtain the corresponding MLE, and then constructs the bin counts based on the original data. We examine the test size and power of the new model diagnostic procedure using simulation studies and illustrate it with a real data set.

We observe $n$ heteroscedastic stochastic processes $\{Y_{v}(t)\}_{v}$, where for any $v\in\{1,\ldots,n\}$ and $t\in [0,1]$, $Y_{v}(t)$ is the convolution product of an unknown function $f$ and a known blurring function $g_{v}$ corrupted by Gaussian noise. Under an ordinary smoothness assumption on $g_{1},\ldots,g_{n}$, our goal is to estimate the $d$-th derivatives (in weak sense) of $f$ from the observations. We propose an adaptive estimator based on wavelet block thresholding, namely the “BlockJS estimator". Taking the mean integrated squared error (MISE), our main theoretical result investigates the minimax rates over Besov smoothness spaces, and shows that our block estimator can achieve the optimal minimax rate, or is at least nearly-minimax in the least favorable situation. We also report a comprehensive suite of numerical simulations to support our theoretical findings. The practical performance of our block estimator compares very favorably to existing methods of the literature on a large set of test functions.

In the present paper we consider the varying coefficient model which represents a useful tool for exploring dynamic patterns in many applications. Existing methods typically provide asymptotic evaluation of precision of estimation procedures under the assumption that the number of observations tends to infinity. In practical applications, however, only a finite number of measurements are available. In the present paper we focus on a non-asymptotic approach to the problem. We propose a novel estimation procedure which is based on recent developments in matrix estimation. In particular, for our estimator, we obtain upper bounds for the mean squared and the pointwise estimation errors. The obtained oracle inequalities are non-asymptotic and hold for finite sample size.

The paper considers model selection in regression under the additional structural constraints on admissible models where the number of potential predictors miht be even larger than the available sample size. We develop a Bayesian formalism which is used as a natural tool for generating a wide class of model selection criteria based on penalized least squares estimation with various complexity penalties associated with a prior on a model size. The resulting criteria are adaptive to structural constraints. We establish the upper bound for the quadratic risk of the resulting MAP estimator and the corresponding lower bound for the minimax risk over a set of admissible models of a given size. We then specify the class of priors (and, therefore, the class of complexity penalties) where for the “nearly-orthogonal” design the MAP estimator is asymptotically at least nearly-minimax (up to a log-factor) simultaneously over an entire range of sparse and dense setups. Moreover, when the numbers of admissible models are “small” (e.g., ordered variable selection) or, on the opposite, for the case of complete variable selection, the proposed estimator achieves the exact minimax rates.

Important information concerning a multivariate data set, such as clusters and modal regions, is contained in the derivatives of the probability density function. Despite this importance, nonparametric estimation of higher order derivatives of the density functions have received only relatively scant attention. Kernel estimators of density functions are widely used as they exhibit excellent theoretical and practical properties, though their generalization to density derivatives has progressed more slowly due to the mathematical intractabilities encountered in the crucial problem of bandwidth (or smoothing parameter) selection. This paper presents the first fully automatic, data-based bandwidth selectors for multivariate kernel density derivative estimators. This is achieved by synthesizing recent advances in matrix analytic theory which allow mathematically and computationally tractable representations of higher order derivatives of multivariate vector valued functions. The theoretical asymptotic properties as well as the finite sample behaviour of the proposed selectors are studied. In addition, we explore in detail the applications of the new data-driven methods for two other statistical problems: clustering and bump hunting. The introduced techniques are combined with the mean shift algorithm to develop novel automatic, nonparametric clustering procedures which are shown to outperform mixture-model cluster analysis and other recent nonparametric approaches in practice. Furthermore, the advantage of the use of smoothing parameters designed for density derivative estimation for feature significance analysis for bump hunting is illustrated with a real data example.

We consider high-frequency sampled continuous-time autoregressive moving average (CARMA) models driven by finite-variance zero-mean Lévy processes. An $L^{2}$-consistent estimator for the increments of the driving Lévy process without order selection in advance is proposed if the CARMA model is invertible. In the second part we analyse the high-frequency behaviour of approximating Riemann sum processes, which represent a natural way to simulate continuous-time moving average models on a discrete grid. We compare their autocovariance structure with the one of sampled CARMA processes and show that the rule of integration plays a crucial role. Moreover, new insight into the kernel estimation procedure of Brockwell et al. [11] is given.

When the study variable is functional and storage capacities are limited or transmission costs are high, selecting with survey sampling techniques a small fraction of the observations is an interesting alternative to signal compression techniques, particularly when the goal is the estimation of simple quantities such as means or totals. We extend, in this functional framework, model-assisted estimators with linear regression models that can take account of auxiliary variables whose totals over the population are known. We first show, under weak hypotheses on the sampling design and the regularity of the trajectories, that the estimator of the mean function as well as its variance estimator are uniformly consistent. Then, under additional assumptions, we prove a functional central limit theorem and we assess rigorously a fast technique based on simulations of Gaussian processes which is employed to build asymptotic confidence bands. The accuracy of the variance function estimator is evaluated on a real dataset of sampled electricity consumption curves measured every half an hour over a period of one week.

We study the asymptotic behaviour of needlets-based approximate maximum likelihood estimators for the spectral parameters of Gaussian and isotropic spherical random fields. We prove consistency and asymptotic Gaussianity, in the high-frequency limit, thus generalizing earlier results by Durastanti et al. (2011) based upon standard Fourier analysis on the sphere. The asymptotic results are then illustrated by an extensive Monte Carlo study.

First we provide a simple set of sufficient conditions for the weak convergence of scaled affine processes with state space $\mathbb{R}_{+}\times \mathbb{R}^{d}$. We specialize our result to one-dimensional continuous state branching processes with immigration. As an application, we study the asymptotic behavior of least squares estimators of some parameters of a two-dimensional critical affine diffusion process.

Gibbs partition models are the largest class of infinite exchangeable partitions of the positive integers generalizing the product form of the probability function of the two-parameter Poisson-Dirichlet family. Here we call into question the current approach to Bayesian nonparametric estimation in species sampling problems under Gibbs priors, which incorrectly relies on treating exchangeable partition probability functions (EPPFs) as multivariate distributions on compositions of the positive integers. We show that once those multivariate distributions are correctly derived, results for corresponding sampling formulas can be obtained, generalized and sometimes fixed, working with marginals and a known result on falling factorial moments of a sum of non independent indicators. We provide an application of our findings to a recently proposed Bayesian nonparametric estimation under Gibbs priors of the predictive probability to observe a species already observed a certain number of times.

We complement the results of Fourdrinier, Mezoued and Strawderman in [5] who considered Bayesian estimation of the location parameter $\theta$ of a random vector $X$ having a unimodal spherically symmetric density $f(\|x-\theta\|^{2})$ for a spherically symmetric prior density $\pi(\|\theta\|^{2})$. In [5], expressing the Bayes estimator as $\delta_{\pi}(X)=X+\nabla M(\|X\|^{2})/m(\|X\|^{2})$, where $m$ is the marginal associated to $f(\|x-\theta\|^{2})$ and $M$ is the marginal with respect to $F(\|x-\theta\|^{2})=1/2\int_{\|x-\theta\|^{2}}^{\infty}f(t)\,dt$, it was shown that, under quadratic loss, if the sampling density $f(\|x-\theta\|^{2})$ belongs to the Berger class (i.e. there exists a positive constant $c$ such that $F(t)/f(t)\geq c$ for all $t$), conditions, dependent on the monotonicity of the ratio $F(t)/f(t)$, can be found on $\pi$ in order that $\delta_{\pi}(X)$ is minimax.

The main feature of this paper is that, in the case where $F(t)/f(t)$ is nonincreasing, if $\pi(\|\theta\|^{2})$ is a superharmonic power prior of the form $\|\theta\|^{-2k}$ with $k>0$, the membership of the sampling density to the Berger class can be droped out. Also, our techniques are different from those in [5]. First, writing $\delta_{\pi}(X)=X+g(X)$ with $g(X)\propto \nabla M(\|X\|^{2})/m(\|X\|^{2})$, we follow Brandwein and Strawderman [4] proving that, for some $b>0$, the function $h=b\,\Delta M/m$ is subharmonic and satisfies $\|g\|^{2}/2\leq -h\leq -{\rm div}g$. Also, we adapt their approach using the fact that $R^{2(k+1)}\int_{B_{\theta,R}}h(x)\,dV_{\theta,R}(x)$ is nonincreasing in $R$ for any $\theta \in{\mathbb{R}} ^{p}$, when $V_{\theta,R}$ is the uniform distribution on the ball $B_{\theta,R}$ of radius $R$ and centered at $\theta$. Examples illustrate the theory.

This paper develops two weighted measures for model selection by generalizing the Kullback-Leibler divergence measure. The concept of a model selection process that takes into account the special features of the underlying model is introduced using weighted measures. New information criteria are defined using the bias correction of an expected weighted loglikelihood estimator. Using weight functions that match the features of interest in the underlying statistical models, the new information criteria are applied to simulated studies of spline regression and copula model selection. Real data applications are also given for predicting the incidence of disease and for quantile modeling of environmental data.

The Expectation Maximization (EM) algorithm is a versatile tool for model parameter estimation in latent data models. When processing large data sets or data stream however, EM becomes intractable since it requires the whole data set to be available at each iteration of the algorithm. In this contribution, a new generic online EM algorithm for model parameter inference in general Hidden Markov Model is proposed. This new algorithm updates the parameter estimate after a block of observations is processed (online). The convergence of this new algorithm is established, and the rate of convergence is studied showing the impact of the block-size sequence. An averaging procedure is also proposed to improve the rate of convergence. Finally, practical illustrations are presented to highlight the performance of these algorithms in comparison to other online maximum likelihood procedures.

We are studying a novel class of goodness-of-fit tests for parametric count time series regression models. These test statistics are formed by considering smoothed versions of the empirical process of the Pearson residuals. Our construction yields test statistics which are consistent against Pitman’s local alternatives and they converge weakly at the usual parametric rate. To approximate the asymptotic null distribution of the test statistics, we propose a parametric bootstrap method and we study its properties. The methodology is applied to simulated and real data.

This paper describes a recursive estimation procedure for multivariate binary densities (probability distributions of vectors of Bernoulli random variables) using orthogonal expansions. For $d$ covariates, there are $2^{d}$ basis coefficients to estimate, which renders conventional approaches computationally prohibitive when $d$ is large. However, for a wide class of densities that satisfy a certain sparsity condition, our estimator runs in probabilistic polynomial time and adapts to the unknown sparsity of the underlying density in two key ways: (1) it attains near-minimax mean-squared error for moderate sample sizes, and (2) the computational complexity is lower for sparser densities. Our method also allows for flexible control of the trade-off between mean-squared error and computational complexity.

We show how the multivariate two-component mixtures with independent coordinates in each component by Hall and Zhou (2003) can be studied within the framework of conditional mixtures as recently introduced by Henry, Kitamura and Salanié (2010). Here, the conditional distribution of the random variable $Y$ given the vector of regressors $Z$ can be expressed as a two-component mixture, where only the mixture weights depend on the covariates. Under appropriate tail conditions on the characteristic functions and the distribution functions of the mixture components, which allow for flexible location-scale type mixtures, we show identification and provide asymptotically normal estimators. The main application for our results are bivariate two-component mixtures with independent coordinates, the case not previously covered by Hall and Zhou (2003). In a simulation study we investigate the finite-sample performance of the proposed methods. The main new technical ingredient is the estimation of limits of quotients of two characteristic functions in the tails from independent samples, which might be of some independent interest.

In this paper, we consider the problem of estimating nonparametrically a mean pattern intensity $\lambda$ from the observation of $n$ independent and non-homogeneous Poisson processes $N^{1},\dots,N^{n}$ on the interval $[0,1]$. This problem arises when data (counts) are collected independently from $n$ individuals according to similar Poisson processes. We show that estimating this intensity is a deconvolution problem for which the density of the random shifts plays the role of the convolution operator. In an asymptotic setting where the number $n$ of observed trajectories tends to infinity, we derive upper and lower bounds for the minimax quadratic risk over Besov balls. Non-linear thresholding in a Meyer wavelet basis is used to derive an adaptive estimator of the intensity. The proposed estimator is shown to achieve a near-minimax rate of convergence. This rate depends both on the smoothness of the intensity function and the density of the random shifts, which makes a connection between the classical deconvolution problem in nonparametric statistics and the estimation of a mean intensity from the observations of independent Poisson processes.

A heteroscedastic linear regression model is considered where responses are allowed to be missing at random. An estimator is constructed that matches the performance of the weighted least squares estimator without the knowledge of the conditional variance function. This is usually done by constructing an estimator of the variance function. Our estimator is a maximum empirical likelihood estimator based on an increasing number of estimated constraints and avoids estimating the variance function.

I consider the use of Markov random fields (MRFs) on a fine grid to represent latent spatial processes when modeling point-level and areal data, including situations with spatial misalignment. Point observations are related to the grid cell in which they reside, while areal observations are related to the (approximate) integral over the latent process within the area of interest. I review several approaches to specifying the neighborhood structure for constructing the MRF precision matrix, presenting results comparing these MRF representations analytically, in simulations, and in two examples. The results provide practical guidance for choosing a spatial process representation and highlight the importance of this choice. In particular, the results demonstrate that, and explain why, standard CAR models can behave strangely for point-level data. They show that various neighborhood weighting approaches based on higher-order neighbors that have been suggested for MRF models do not produce smooth fields, which raises doubts about their utility. Finally, they indicate that an MRF that approximates a thin plate spline compares favorably to standard CAR models and to kriging under many circumstances.

Priors constructed from scale mixtures of normal distributions have long played an important role in decision theory and shrinkage estimation. This paper demonstrates equivalence between the maximum aposteriori estimator constructed under one such prior and Zhang’s minimax concave penalization estimator. This equivalence and related multivariate generalizations stem directly from an intriguing representation of the minimax concave penalty function as the Moreau envelope of a simple convex function. Maximum aposteriori estimation under the corresponding marginal prior distribution, a generalization of the quasi-Cauchy distribution proposed by Johnstone and Silverman, leads to thresholding estimators having excellent frequentist risk properties.

The performance of nonparametric estimators is heavily dependent on a bandwidth parameter. In nonparametric Bayesian methods this parameter can be specified as a hyperparameter of the nonparametric prior. The value of this hyperparameter may be made dependent on the data. The empirical Bayes method is to set its value by maximizing the marginal likelihood of the data in the Bayesian framework. In this paper we analyze a particular version of this method, common in practice, that the hyperparameter scales the prior variance. We characterize the behavior of the random hyperparameter, and show that a nonparametric Bayes method using it gives optimal recovery over a scale of regularity classes. This scale is limited, however, by the regularity of the unscaled prior. While a prior can be scaled up to make it appropriate for arbitrarily rough truths, scaling cannot increase the nominal smoothness by much. Surprisingy the standard empirical Bayes method is even more limited in this respect than an oracle, deterministic scaling method. The same can be said for the hierarchical Bayes method.

A new time series clustering method based on comparing forecast densities for a sequence of $k>1$ consecutive horizons is proposed. The unknown $k$-dimensional forecast densities can be non-parametrically approximated by using bootstrap procedures that mimic the generating processes without parametric restrictions. However, the difficulty of constructing accurate kernel estimators of multivariate densities is well known. To circumvent the high dimensionality problem, the bootstrap prediction vectors are projected onto a lower-dimensional space using principal components analysis, and then the densities are estimated in this new space. Proper distances between pairs of estimated densities are computed and used to generate an initial dissimilarity matrix, and hence a standard hierarchical clustering is performed. The clustering procedure is examined via simulation and is applied to a real dataset involving electricity prices series.

A maximum likelihood based model selection of discrete Bayesian networks is considered. The structure learning is performed by employing a scoring function $S$, which, for a given network $G$ and $n$-sample $D_{n}$, is defined as the maximum marginal log-likelihood $l$ minus a penalization term $\lambda_{n}h$ proportional to network complexity $h(G)$, $$S(G|D_{n})=l(G|D_{n})-\lambda_{n}h(G).$$ An available case analysis is developed with the standard log-likelihood replaced by the sum of sample average node log-likelihoods. The approach utilizes partially missing data records and allows for comparison of models fitted to different samples.

In missing completely at random settings the estimation is shown to be consistent if and only if the sequence $\lambda_{n}$ converges to zero at a slower than $n^{-{1/2}}$ rate. In particular, the BIC model selection ($\lambda_{n}=0.5\log(n)/n$) applied to the node-average log-likelihood is shown to be inconsistent in general. This is in contrast to the complete data case when BIC is known to be consistent. The conclusions are confirmed by numerical experiments.

We develop a clustering framework for observations from a population with a smooth probability distribution function and derive its asymptotic properties. A clustering criterion based on a linear combination of order statistics is proposed. The asymptotic behavior of the point at which the observations are split into two clusters is examined. The results obtained can then be utilized to construct an interval estimate of the point which splits the data and develop tests for bimodality and presence of clusters.

In the present paper we consider Laplace deconvolution problem for discrete noisy data observed on an interval whose length $T_{n}$ may increase with the sample size. Although this problem arises in a variety of applications, to the best of our knowledge, it has been given very little attention by the statistical community. Our objective is to fill the gap and provide statistical analysis of Laplace deconvolution problem with noisy discrete data. The main contribution of the paper is an explicit construction of an asymptotically rate-optimal (in the minimax sense) Laplace deconvolution estimator which is adaptive to the regularity of the unknown function. We show that the original Laplace deconvolution problem can be reduced to nonparametric estimation of a regression function and its derivatives on the interval of growing length $T_{n}$. Whereas the forms of the estimators remain standard, the choices of the parameters and the minimax convergence rates, which are expressed in terms of $T_{n}^{2}/n$ in this case, are affected by the asymptotic growth of the length of the interval.

We derive an adaptive kernel estimator of the function of interest, and establish its asymptotic minimaxity over a range of Sobolev classes. We illustrate the theory by examples of construction of explicit expressions of Laplace deconvolution estimators. A simulation study shows that, in addition to providing asymptotic optimality as the number of observations tends to infinity, the proposed estimator demonstrates good performance in finite sample examples.

Multistable processes are tangent at each point to a stable process, but where the index of stability and the index of localisability varies along the path. In this work, we give two estimators of the stability and the localisability functions, and we prove the consistency of those two estimators. We illustrate these convergences with two examples, the Lévy multistable process and the Linear Multifractional Multistable Motion.

This article proposes a test to detect changes in general autocovariance structure in nonstationary time series. Our approach is founded on the locally stationary wavelet (LSW) process model for time series which has previously been used for classification and segmentation of time series. Using this framework we form a likelihood-based hypothesis test and demonstrate its performance against existing methods on various simulated examples as well as applying it to a problem arising from ocean engineering.

We consider the estimation of a regression function with random design and heteroscedastic noise in a nonparametric setting. More precisely, we address the problem of characterizing the optimal penalty when the regression function is estimated by using a penalized least-squares model selection method. In this context, we show the existence of a minimal penalty, defined to be the maximum level of penalization under which the model selection procedure totally misbehaves. The optimal penalty is shown to be twice the minimal one and to satisfy a non-asymptotic pathwise oracle inequality with leading constant almost one. Finally, the ideal penalty being unknown in general, we propose a hold-out penalization procedure and show that the latter is asymptotically optimal.

A theoretically sound bootstrap procedure is proposed for building accurate confidence intervals of parameters describing the extremal behavior of instantaneous functionals $\{f(X_{n})\}_{n\in\mathbb{N}}$ of a Harris Markov chain $X$, namely the extremal and tail indexes. Regenerative properties of the chain $X$ (or of a Nummelin extension of the latter) are here exploited in order to construct consistent estimators of these parameters, following the approach developed in [10]. Their asymptotic normality is first established and the standardization problem is also tackled. It is then proved that, based on these estimators, the regenerative block-bootstrap and its approximate version, both introduced in [7], yield asymptotically valid confidence intervals. In order to illustrate the performance of the methodology studied in this paper, simulation results are additionally displayed.

One main focus of learning theory is to find optimal rates of convergence. In classification, it is possible to obtain optimal fast rates (faster than $n^{-1/2}$) in a minimax sense. Moreover, using an aggregation procedure, the algorithms are adaptive to the parameters of the class of distributions. Here, we investigate this issue in the bipartite ranking framework. We design a ranking rule by aggregating estimators of the regression function. We use exponential weights based on the empirical ranking risk. Under several assumptions on the class of distribution, we show that this procedure is adaptive to the margin parameter and smoothness parameter and achieves the same rates as in the classification framework. Moreover, we state a minimax lower bound that establishes the optimality of the aggregation procedure in a specific case.

This paper deals with the problem of simultaneously making many ($M$) binary decisions based on one realization of a random data matrix $\mathbf{X}$. $M$ is typically large and $\mathbf{X}$ will usually have $M$ rows associated with each of the $M$ decisions to make, but for each row the data may be low dimensional. Such problems arise in many practical areas such as the biological and medical sciences, where the available dataset is from microarrays or other high-throughput technology and with the goal being to decide which among of many genes are relevant with respect to some phenotype of interest; in the engineering and reliability sciences; in astronomy; in education; and in business. A Bayesian decision-theoretic approach to this problem is implemented with the overall loss function being a cost-weighted linear combination of Type I and Type II loss functions. The class of loss functions considered allows for use of the false discovery rate (FDR), false nondiscovery rate (FNR), and missed discovery rate (MDR) in assessing the quality of decision. Through this Bayesian paradigm, the Bayes multiple decision function (BMDF) is derived and an efficient algorithm to obtain the optimal Bayes action is described. In contrast to many works in the literature where the rows of the matrix $\mathbf{X}$ are assumed to be stochastically independent, we allow a dependent data structure with the associations obtained through a class of frailty-induced Archimedean copulas. In particular, non-Gaussian dependent data structure, which is typical with failure-time data, can be entertained. The numerical implementation of the determination of the Bayes optimal action is facilitated through sequential Monte Carlo techniques. The theory developed could also be extended to the problem of multiple hypotheses testing, multiple classification and prediction, and high-dimensional variable selection. The proposed procedure is illustrated for the simple versus simple hypotheses setting and for the composite hypotheses setting through simulation studies. The procedure is also applied to a subset of a microarray data set from a colon cancer study.

We estimate convex polytopes and general convex sets in $\mathbb{R}^{d}$, $d\geq 2$ in the regression framework. We measure the risk of our estimators using a $L^{1}$-type loss function and prove upper bounds on these risks. We show, in the case of convex polytopes, that these estimators achieve the minimax rate. For convex polytopes, this minimax rate is $\frac{\ln n}{n}$, which differs from the parametric rate for non-regular families by a logarithmic factor, and we show that this extra factor is essential. Using polytopal approximations we extend our results to general convex sets, and we achieve the minimax rate up to a logarithmic factor. In addition we provide an estimator that is adaptive with respect to the number of vertices of the unknown polytope, and we prove that this estimator is optimal in all classes of convex polytopes with a given number of vertices.

False Discovery Rate (FDR) and the Bayes risk are two different statistical measures, which can be used to evaluate and compare multiple testing procedures. Recent results show that under sparsity FDR controlling procedures, like the popular Benjamini-Hochberg (BH) procedure, perform also very well in terms of the Bayes risk. In particular asymptotic Bayes optimality under sparsity (ABOS) of BH was shown previously for location and scale models based on log-concave densities. This article extends previous work to a substantially larger set of distributions of effect sizes under the alternative, where the alternative distribution of true signals does not change with the number of tests $m$, while the sample size $n$ slowly increases. ABOS of BH and the corresponding step-down procedure based on FDR levels proportional to $n^{-1/2}$ are proved. A simulation study shows that these asymptotic results are relevant already for relatively small values of $m$ and $n$. Apart from showing asymptotic optimality of BH, our results on the optimal FDR level provide a natural extension of the well known results on the significance levels of Bayesian tests.

In this paper, we propose a new classification tree, the projection pursuit classification tree (PPtree). It combines tree structured methods with projection pursuit dimension reduction. This tree is originated from the projection pursuit method for classification. In each node, one of the projection pursuit indices using class information - LDA, $L_{r}$ or PDA indices - is maximized to find the projection with the most separated group view. On this optimized data projection, the tree splitting criteria are applied to separate the groups. These steps are iterated until the last two classes are separated. The main advantages of this tree is that it effectively uses correlation between variables to find separations, and it has visual representation of the differences between groups in a 1-dimensional space that can be used to interpret results. Also in each node of the tree, the projection coefficients represent the variable importance for the group separation. This information is very helpful to select variables in classification problems.

We discuss the nonparametric Nadaraya-Watson (N-W) estimator of the drift function for ergodic stochastic processes driven by $\alpha$-stable noises and observed at discrete instants. Under geometrical mixing condition, we derive consistency and rate of convergence of the N-W estimator of the drift function. Furthermore, we obtain a central limit theorem for stable stochastic integrals. The central limit theorem has its own interest and is the crucial tool for the proofs. A simulation study illustrates the finite sample properties of the N-W estimator.

For estimating a lower bounded parametric function in the framework of Marchand and Strawderman [6], we provide through a unified approach a class of Bayesian confidence intervals with credibility $1-\alpha$ and frequentist coverage probability bounded below by $\frac{1-\alpha}{1+\alpha}$. In cases where the underlying pivotal distribution is symmetric, the findings represent extensions with respect to the specification of the credible set achieved through the choice of a spending function, and include Marchand and Strawderman’s HPD procedure result. For non-symmetric cases, the determination of a such a class of Bayesian credible sets fills a gap in the literature and includes an “equal-tails” modification of the HPD procedure. Several examples are presented demonstrating wide applicability.

Sample selection models are employed when an outcome of interest is observed for a restricted non-randomly selected sample of the population. We consider the case in which the response is binary and continuous covariates have a nonlinear relationship to the outcome. We introduce two statistical methods for the estimation of two binary regression models involving semiparametric predictors in the presence of non-random sample selection. This is achieved using a multiple-stage procedure, and a newly developed simultaneous equation estimation scheme. Both approaches are based on the penalized likelihood estimation framework. The problems of identification and inference are also discussed. The empirical properties of the proposed approaches are studied through a simulation study. The methods are then illustrated using data from the American National Election Study where the aim is to quantify public support for school integration. If non-random sample selection is neglected then the predicted probability of giving, for instance, a supportive response may be biased, an issue that can be tackled using the proposed tools.

The lasso is a popular tool for sparse linear regression, especially for problems in which the number of variables $p$ exceeds the number of observations $n$. But when $p>n$, the lasso criterion is not strictly convex, and hence it may not have a unique minimizer. An important question is: when is the lasso solution well-defined (unique)? We review results from the literature, which show that if the predictor variables are drawn from a continuous probability distribution, then there is a unique lasso solution with probability one, regardless of the sizes of $n$ and $p$. We also show that this result extends easily to $\ell_{1}$ penalized minimization problems over a wide range of loss functions.

A second important question is: how can we manage the case of non-uniqueness in lasso solutions? In light of the aforementioned result, this case really only arises when some of the predictor variables are discrete, or when some post-processing has been performed on continuous predictor measurements. Though we certainly cannot claim to provide a complete answer to such a broad question, we do present progress towards understanding some aspects of non-uniqueness. First, we extend the LARS algorithm for computing the lasso solution path to cover the non-unique case, so that this path algorithm works for any predictor matrix. Next, we derive a simple method for computing the component-wise uncertainty in lasso solutions of any given problem instance, based on linear programming. Finally, we review results from the literature on some of the unifying properties of lasso solutions, and also point out particular forms of solutions that have distinctive properties.

One major goal in clinical applications of multi-state models is the estimation of transition probabilities. The usual nonparametric estimator of the transition matrix for non-homogeneous Markov processes is the Aalen-Johansen estimator (Aalen and Johansen 1978 [1]). In this paper we propose a modification of the Aalen-Johansen estimator in the illness-death model based on presmoothing. The consistency of the proposed estimators is formally established. Simulations show that the presmoothed estimators may be much more efficient than the Aalen-Johansen estimator. A real data illustration is included.

Linear mixed effects methods for the analysis of longitudinal data provide a convenient framework for modelling within-individual correlation across time. Using spline functions allows for flexible modelling of the response as a smooth function of time. A computational connection between linear mixed effects modelling and spline smoothing has resulted in a cross-fertilization of these two fields. The connection has popularized the use of spline functions in longitudinal data analysis and the use of mixed effects software in smoothing analyses. However, care must be taken in exploiting this connection, as resulting estimates of the underlying population mean might not track the data well and associated standard errors might not reflect the true variability in the data. We discuss these shortcomings and suggest some easy-to-compute methods to eliminate them.

In this paper, a nonparametric estimator is proposed for estimating the $L_{1}$-median for multivariate conditional distribution when the covariates take values in an infinite dimensional space. The multivariate case is more appropriate to predict the components of a vector of random variables simultaneously rather than predicting each of them separately. While estimating the conditional $L_{1}$-median function using the well-known Nadarya-Waston estimator, we establish the strong consistency of this estimator as well as the asymptotic normality. We also present some simulations and provide how to built conditional confidence ellipsoids for the multivariate $L_{1}$-median regression in practice. Some numerical study in chemiometrical real data are carried out to compare the multivariate $L_{1}$-median regression with the vector of marginal median regression when the covariate $X$ is a curve as well as $X$ is a random vector.

Robust parameter estimation has been discussed as a method for reducing a bias caused by outliers. An estimating equation using a weighted score function is often used. A typical estimating equation is non-normalized, but this paper considers a normalized estimating equation, which is corrected to ensure that the mean of the weight is one. In robust parameter estimation, it is important to control the difference between the target parameter and the limit of the robust estimator, which is referred to as the latent bias in this paper. The latent bias is usually discussed in terms of influence function and breakdown point. It is illustrated by some examples that the latent bias can be close to zero for the normalized estimating equation even if the proportion of outliers is not small, but not close to zero for the non-normalized estimating equation. Furthermore, this behavior of the normalized estimating equation can be proved under mild conditions. The asymptotic normality of the robust estimator is also presented and then it is shown that the outliers are naturally ignored with an appropriate proportion of outliers from the viewpoint of asymptotic variance. The results can be extended to the regression case. The behaviors of the latent bias and mean squared error are investigated by numerical studies.

Many regularization schemes for high-dimensional regression have been put forward. Most require the choice of a tuning parameter, using model selection criteria or cross-validation. We show that a simple sign-constrained least squares estimation is a very simple and effective regularization technique for a certain class of high-dimensional regression problems. The sign constraint has to be derived via prior knowledge or an initial estimator. The success depends on conditions that are easy to check in practice. A sufficient condition for our results is that most variables with the same sign constraint are positively correlated. For a sparse optimal predictor, a non-asymptotic bound on the $\ell_{1}$-error of the regression coefficients is then proven. Without using any further regularization, the regression vector can be estimated consistently as long as $s^{2}\log(p)/n\rightarrow 0$ for $n\rightarrow\infty$, where $s$ is the sparsity of the optimal regression vector, $p$ the number of variables and $n$ sample size. The bounds are almost as tight as similar bounds for the Lasso for strongly correlated design despite the fact that the method does not have a tuning parameter and does not require cross-validation. Network tomography is shown to be an application where the necessary conditions for success of sign-constrained least squares are naturally fulfilled and empirical results confirm the effectiveness of the sign constraint for sparse recovery if predictor variables are strongly correlated.

Approximate Bayesian Computation (ABC) can be viewed as an analytic approximation of an intractable likelihood coupled with an elementary simulation step. Such a view, combined with a suitable instrumental prior distribution permits maximum-likelihood (or maximum-a-posteriori) inference to be conducted, approximately, using essentially the same techniques. An elementary approach to this problem which simply obtains a nonparametric approximation of the likelihood surface which is then maximised is developed here and the convergence of this class of algorithms is characterised theoretically. The use of non-sufficient summary statistics in this context is considered. Applying the proposed method to four problems demonstrates good performance. The proposed approach provides an alternative for approximating the maximum likelihood estimator (MLE) in complex scenarios.

New bandwidth selectors for kernel density estimation with directional data are presented in this work. These selectors are based on asymptotic and exact error expressions for the kernel density estimator combined with mixtures of von Mises distributions. The performance of the proposed selectors is investigated in a simulation study and compared with other existing rules for a large variety of directional scenarios, sample sizes and dimensions. The selector based on the exact error expression turns out to have the best behaviour of the studied selectors for almost all the situations. This selector is illustrated with real data for the circular and spherical cases.

In the present paper we consider the problem of estimating a periodic $(r+1)$-dimensional function $f$ based on observations from its noisy convolution. We construct a wavelet estimator of $f$, derive minimax lower bounds for the $L^{2}$-risk when $f$ belongs to a Besov ball of mixed smoothness and demonstrate that the wavelet estimator is adaptive and asymptotically near-optimal within a logarithmic factor, in a wide range of Besov balls. We prove in particular that choosing this type of mixed smoothness leads to rates of convergence which are free of the “curse of dimensionality” and, hence, are higher than usual convergence rates when $r$ is large.

The problem studied in the paper is motivated by seismic inversion which can be reduced to solution of noisy two-dimensional convolution equations that allow to draw inference on underground layer structures along the chosen profiles. The common practice in seismology is to recover layer structures separately for each profile and then to combine the derived estimates into a two-dimensional function. By studying the two-dimensional version of the model, we demonstrate that this strategy usually leads to estimators which are less accurate than the ones obtained as two-dimensional functional deconvolutions. Indeed, we show that unless the function $f$ is very smooth in the direction of the profiles, very spatially inhomogeneous along the other direction and the number of profiles is very limited, the functional deconvolution solution has a much better precision compared to a combination of $M$ solutions of separate convolution equations. A limited simulation study in the case of $r=1$ confirms theoretical claims of the paper.

We consider the rate of convergence of the expected loss of empirically optimal vector quantizers. Earlier results show that the mean-squared expected distortion for any fixed probability distribution supported on a bounded set and satisfying some regularity conditions decreases at the rate $\mathcal{O}(\log n/n)$. We prove that this rate is actually $\mathcal{O}(1/n)$. Although these conditions are hard to check, we show that well-clustered distributions with continuous densities supported on a bounded set are included in the scope of this result.

In probabilistic forecasting combination formulas for the aggregation of predictive distributions need to be estimated based on past experience and training data. We study combination formulas and aggregation methods for predictive cumulative distribution functions from the perspectives of calibration and dispersion, taking an original prediction space approach that applies to discrete, mixed discrete-continuous and continuous predictive distributions alike. The key idea is that aggregation methods ought to be parsimonious, yet sufficiently flexible to accommodate any type of dispersion in the component distributions. Both linear and non-linear aggregation methods are investigated, including generalized, spread-adjusted and beta-transformed linear pools. The effects and techniques are demonstrated theoretically, in simulation examples, and in case studies, where we fit combination formulas for density forecasts of S&P 500 returns and daily maximum temperature at Seattle-Tacoma Airport.

For the univariate current status and, more generally, the interval censoring model, distribution theory has been developed for the maximum likelihood estimator (MLE) and smoothed maximum likelihood estimator (SMLE) of the unknown distribution function, see, e.g., [10, 7, 4, 5, 6, 9, 13] and [11]. For the bivariate current status and interval censoring models distribution theory of this type is still absent and even the rate at which we can expect reasonable estimators to converge is unknown.

We define a purely discrete plug-in estimator of the distribution function which locally converges at rate $n^{1/3}$, and derive its (normal) limit distribution. Unlike the MLE or SMLE, this estimator is not a proper distribution function. Since the estimator is purely discrete, it demonstrates that the $n^{1/3}$ convergence rate is in principle possible for the MLE, but whether this actually holds for the MLE is still an open problem.

We compare the behavior of the plug-in estimator with the behavior of the MLE on a sieve and the SMLE in a simulation study. This indicates that the plug-in estimator and the SMLE have a smaller variance but a larger bias than the sieved MLE. The SMLE is conjectured to have a $n^{1/3}$-rate of convergence if we use bandwidths of order $n^{-1/6}$. We derive its (normal) limit distribution, using this assumption. Finally, we demonstrate the behavior of the MLE and SMLE for the bivariate interval censored data of [1], which have been discussed by many authors, see e.g., [18, 3, 2] and [15].

In a linear model, consider the class of estimators that are equivariant with respect to linear transformations of the predictor basis. Each of these estimators determines an equivariant linear prediction rule. Equivariant prediction rules may be appropriate in settings where sparsity assumptions (like those common in high-dimensional data analysis) are untenable or little is known about the relevance of the given predictor basis, insofar as it relates to the outcome. In this paper, we study the out-of-sample prediction error associated with equivariant estimators in high-dimensional linear models with Gaussian predictors and errors. We show that non-trivial equivariant prediction is impossible when the number of predictors $d$ is greater than the number of observations $n$. For $d/n\to \rho \in[0,1)$, we show that a James-Stein estimator (a scalar multiple of the ordinary least squares estimator) is asymptotically optimal for equivariant out-of-sample prediction, and derive a closed-form expression for its asymptotic predictive risk. Finally, we undertake a detailed comparative analysis involving the proposed James-Stein estimator and other well-known estimators for non-sparse settings, including the ordinary least squares estimator, ridge regression, and other James-Stein estimators for the linear model. Among other things, this comparative analysis sheds light on the role of the population-level predictor covariance matrix and reveals that other previously studied James-Stein estimators for the linear model are sub-optimal in terms of out-of-sample prediction error.

A new method is proposed for small area estimation. The principle is based upon the splitting of the sampling weights between the areas. A matrix of weights is defined. Each column of this matrix enables us to estimate the total of the variables of interest at the level of an area. This method automatically satisfies the coherence property between the local estimates and the overall estimate. Moreover, the local estimators are calibrated on auxiliary information available at the level of the small areas. This methodology also enables the use of composite estimators that are weighted means between a direct estimator and a synthetic estimator. Once the weights are computed, the estimates can be easily computed for any variable of interest. A set of simulations shows the interest of the proposed method.

In practice, it is desired to have estimates that are invariant under reparameterization. The invariance property of the estimators helps to formulate a unified solution to the underlying estimation problem. In robust Bayesian analysis, a frequent criticism is that the optimal estimators are not invariant under smooth reparameterizations. This paper considers the problem of posterior regret gamma-minimax (PRGM) estimation of the natural parameter of the exponential family of distributions under intrinsic loss functions with Kullback-Leibler distance. We show that under the class of Jeffrey’s Conjugate Prior (JCP) distributions, PRGM estimators are invariant to smooth one-to-one reparameterizations. We apply our results to several distributions and different classes of JCP, as well as the usual conjugate prior distributions. We observe that, in many cases, invariant PRGM estimators in the class of JCP distributions can be obtained by some modifications of PRGM estimators in the usual class of conjugate priors. Moreover, when the class of priors are convex or dependant on a hyper-parameter belonging to a connected set, we show that the PRGM estimator under the intrinsic loss function could be Bayes with respect to a prior distribution in the original prior class. Theoretical results are supplemented with several examples and illustrations.

The cellular tree classifier model addresses a fundamental problem in the design of classifiers for a parallel or distributed computing world: Given a data set, is it sufficient to apply a majority rule for classification, or shall one split the data into two or more parts and send each part to a potentially different computer (or cell) for further processing? At first sight, it seems impossible to define with this paradigm a consistent classifier as no cell knows the “original data size”, $n$. However, we show that this is not so by exhibiting two different consistent classifiers. The consistency is universal but is only shown for distributions with nonatomic marginals.

Diffusion tensor magnetic resonance imaging (MRI) quantifies the spatial distribution of water diffusion at each voxel on a regular grid of locations in a biological specimen by diffusion tensors– $3\times 3$ positive definite matrices. Removal of noise from DTI is an important problem due to the high scientific relevance of DTI and relatively low signal to noise ratio it provides. Leading approaches to this problem amount to estimation of weighted Karcher means of diffusion tensors within spatial neighborhoods, under various metrics imposed on the space of tensors. However, it is unclear how the behavior of these estimators varies with the magnitude of DTI sensor noise (the noise resulting from the thermal effects of MRI scanning) as well as the geometric structure of the underlying diffusion tensor neighborhoods. In this paper, we combine theoretical analysis, empirical analysis of simulated DTI data, and empirical analysis of real DTI scans to compare the noise removal performance of three kernel-based DTI smoothers that are based on Euclidean, log-Euclidean, and affine-invariant metrics. The results suggest, contrary to conventional wisdom, that imposing a simplistic Euclidean metric may in fact provide comparable or superior noise removal, especially in relatively unstructured regions and/or in the presence of moderate to high levels of sensor noise. On the contrary, log-Euclidean and affine-invariant metrics may lead to better noise removal in highly structured anatomical regions, especially when the sensor noise is of low magnitude. These findings emphasize the importance of considering the interplay of sensor noise magnitude and tensor field geometric structure when assessing diffusion tensor smoothing options. They also point to the necessity for continued development of smoothing methods that perform well across a large range of scenarios.

Many clinical trials are monitored through interim analysis. Group sequential tests are popular statistical tools for interim analysis. Sample size determination for interim analysis under group sequential setting is studied in comparing to the design without interim analysis. The effects on sample size determination were examined for both classic and fractional Brownian motion of the monitoring statistic. Selective results were obtained for two commonly used error spending functions with various conditions. The results showed that the drift parameter was generally smaller when $H>0.5$ under fractional Brownian motion and would lead to smaller sample sizes. The R code for carrying out the computation is also provided.

The Nile problem by Ronald Fisher may be interpreted as the problem of making statistical inference for a special curved exponential family when the minimal sufficient statistic is incomplete. The problem itself and its versions for general curved exponential families pose a mathematical-statistical challenge: studying the subalgebras of ancillary statistics within the $\sigma$-algebra of the (incomplete) minimal sufficient statistics and closely related questions of the structure of UMVUEs.

In this paper a new method is developed that, in particular, proves that in the classical Nile problem no statistic subject to mild natural conditions is a UMVUE. The method almost solves an old problem of the existence of UMVUEs. The method is purely statistical (vs. analytical) and works for any family possessing an ancillary statistic. It complements an analytical method that uses only the first order ancillarity (and thus works when the existence of ancillary subalgebras is an open problem) and works for curved exponential families with polynomial constraints on the canonical parameters of which the Nile problem is a special case.

Ordinary Least Squares (OLS) is recognised as being useful in the context of multiple linear regression but can also be effective under the more general framework of the single-index model. In cases where it is ineffective, transformations to the response can improve performance while still allowing for interpretation on the original scale. In this paper we introduce an influence diagnostic for OLS that can be used to assess its effectiveness in the general setting and which can also be used following response transformations. These findings are further emphasized and verified via some simulation studies.

We provide a generalization of Hotelling’s Theorem that enables inference (i) for the mean vector of a multivariate normal population and (ii) for the comparison of the mean vectors of two multivariate normal populations, when the number $p$ of components is larger than the number $n$ of sample units and the (common) covariance matrix is unknown. In particular, we extend some recent results presented in the literature by finding the (finite-$n$) $p$-asymptotic distribution of the Generalized Hotelling’s $T^{2}$ enabling the inferential analysis of large-$p$ small-$n$ normal data sets under mild assumptions.

Large-sample validity is proved for stationary bootstrapping of a bias-corrected realized volatility under market microstructure noise, which enables us to construct a bootstrap confidence interval of integrated volatility. A finite-sample simulation shows that the stationary bootstrapping confidence interval outperforms existing ones which are constructed ignoring market microstructure noise or using asymptotic normality for the bias-corrected realized volatility.

One of the most widely used data augmentation algorithms is Albert and Chib’s (1993) algorithm for Bayesian probit regression. Polson, Scott, and Windle (2013) recently introduced an analogous algorithm for Bayesian logistic regression. The main difference between the two is that Albert and Chib’s (1993) truncated normals are replaced by so-called Polya-Gamma random variables. In this note, we establish that the Markov chain underlying Polson, Scott, and Windle’s (2013) algorithm is uniformly ergodic. This theoretical result has important practical benefits. In particular, it guarantees the existence of central limit theorems that can be used to make an informed decision about how long the simulation should be run.

Let $(X,Y)\in\mathcal{X}\times\mathcal{Y}$ be a random couple with unknown distribution $P$. Let $\mathcal{G}$ be a class of measurable functions and $\ell$ a loss function. The problem of statistical learning deals with the estimation of the Bayes: \[g^{*}=\arg\min_{g\in\mathcal{G}}\mathbb{E}_{P}\ell(g,(X,Y)).\] In this paper, we study this problem when we deal with a contaminated sample $(Z_{1},Y_{1}),\dots,(Z_{n},Y_{n})$ of i.i.d. indirect observations. Each input $Z_{i}$, $i=1,\dots,n$ is distributed from a density $Af$, where $A$ is a known compact linear operator and $f$ is the density of the direct input $X$.

We derive fast rates of convergence for the excess risk of empirical risk minimizers based on regularization methods, such as deconvolution kernel density estimators or spectral cut-off. These results are comparable to the existing fast rates in Koltchinskii (2006) for the direct case. It gives some insights into the effect of indirect measurements in the presence of fast rates of convergence.

A common problem in the sciences is that a signal of interest is observed only indirectly, through smooth functionals of the signal whose values are then obscured by noise. In such inverse problems, the functionals dampen or entirely eliminate some of the signal’s interesting features. This makes it difficult or even impossible to fully reconstruct the signal, even without noise. In this paper, we develop methods for handling sequences of related inverse problems, with the problems varying either systematically or randomly over time. Such sequences often arise with automated data collection systems, like the data pipelines of large astronomical instruments such as the Large Synoptic Survey Telescope (LSST). The LSST will observe each patch of the sky many times over its lifetime under varying conditions. A possible additional complication in these problems is that the observational resolution is limited by the instrument, so that even with many repeated observations, only an approximation of the underlying signal can be reconstructed. We propose an efficient estimator for reconstructing a signal of interest given a sequence of related, resolution-limited inverse problems. We demonstrate our method’s effectiveness in some representative examples and provide theoretical support for its adoption.

This article is concerned with simultaneous tests on linear regression coefficients in high-dimensional settings. When the dimensionality is larger than the sample size, the classic $F$-test is not applicable since the sample covariance matrix is not invertible. Recently, [5] and [17] proposed testing procedures by excluding the inverse term in $F$-statistics. However, the efficiency of such $F$-statistic-based methods is adversely affected by outlying observations and heavy tailed distributions. To overcome this issue, we propose a robust score test based on rank regression. The asymptotic distributions of the proposed test statistic under the high-dimensional null and alternative hypotheses are established. Its asymptotic relative efficiency with respect to [17]’s test is closely related to that of the Wilcoxon test in comparison with the $t$-test. Simulation studies are conducted to compare the proposed procedure with other existing testing procedures and show that our procedure is generally more robust in both sizes and powers.

Consider the standard linear model $\mathbf{y}=X\boldsymbol{\beta}+\sigma\epsilon$, where the components of $\epsilon$ are iid standard normal errors. Park and Casella [14] consider a Bayesian treatment of this model with a Laplace/Inverse-Gamma prior on $(\beta,\sigma)$. They introduce a Data Augmentation approach that can be used to explore the resulting intractable posterior density, and call it the Bayesian lasso algorithm. In this paper, the Markov chain underlying the Bayesian lasso algorithm is shown to be geometrically ergodic, for arbitrary values of the sample size $n$ and the number of variables $p$. This is important, as geometric ergodicity provides theoretical justification for the use of Markov chain CLT, which can then be used to obtain asymptotic standard errors for Markov chain based estimates of posterior quantities. Kyung et al. [12] provide a proof of geometric ergodicity for the restricted case $n\geq p$, but as we explain in this paper, their proof is incorrect. Our approach is different and more direct, and enables us to establish geometric ergodicity for arbitrary $n$ and $p$.

In this paper, we propose corrections to the likelihood ratio test and John’s test for sphericity in large-dimensions. New formulas for the limiting parameters in the CLT for linear spectral statistics of sample covariance matrices with general fourth moments are first established. Using these formulas, we derive the asymptotic distribution of the two proposed test statistics under the null. These asymptotics are valid for general population, i.e. not necessarily Gaussian, provided a finite fourth-moment. Extensive Monte-Carlo experiments are conducted to assess the quality of these tests with a comparison to several existing methods from the literature. Moreover, we also obtain their asymptotic power functions under the alternative of a spiked population model as a specific alternative.

We discuss the implications of Bayes’ postulate in the setting of exchangeable Bernoulli random variables. Bayes’ postulate, here, stipulates a uniform distribution on the total number of successes in any number of trials. For an infinite sequence of exchangeable Bernoulli variables the conditions of Bayes’ postulate are equivalent to a uniform (prior) distribution on the underlying mixing variable which necessarily exists by De Finetti’s representation theorem. We show that in the presence of exchangeability, the conditions of Bayes’ postulate are implied by a considerably weaker assumption which only specifies the probability of $n$ successes in $n$ trials, for every $n$. The equivalence of the Bayes’ postulate and the weak assumption holds for both finite and infinite sequences. We also explore characterizations of the joint distribution of finitely many exchangeable Bernoulli variables in terms of probability statements similar to the weak assumption. Finally, we consider extensions of Bayes’ postulate in the framework of exchangeable multinomial trials.

We address the problem of predicting spatially dependent functional data belonging to a Hilbert space, with a Functional Data Analysis approach. Having defined new global measures of spatial variability for functional random processes, we derive a Universal Kriging predictor for functional data. Consistently with the new established theoretical results, we develop a two-step procedure for predicting georeferenced functional data: first model selection and estimation of the spatial mean (drift), then Universal Kriging prediction on the basis of the identified model. The proposed methodology is applied to daily mean temperatures curves recorded in the Maritimes Provinces of Canada.

This paper is devoted to the discrimination between a stationary long-range dependent model and a non stationary process. We develop a nonparametric test for stationarity in the framework of locally stationary long memory processes which is based on a Kolmogorov-Smirnov type distance between the time varying spectral density and its best approximation through a stationary spectral density. We show that the test statistic converges to the same limit as in the short memory case if the (possibly time varying) long memory parameter is smaller than $1/4$ and justify why the limiting distribution is different if the long memory parameter exceeds this boundary. Concerning the latter case the novel FARI($\infty$) bootstrap is introduced which provides a bootstrap-based test for stationarity which shows good empirical properties if the long memory parameter is smaller than $1/2$ which is the usual restriction in the framework of long-range dependent time series. We investigate the finite sample properties of our approach in a comprehensive simulation study and employ the new test in an analysis of two data sets.

In extreme value theory, the so-called extreme-value index is a parameter that controls the behavior of a distribution function in its right tail. Knowing this parameter is thus essential to solve many problems related to extreme events. In this paper, the estimation of the extreme-value index is considered in the presence of a random covariate, whether the conditional distribution of the variable of interest belongs to the Fréchet, Weibull or Gumbel max-domain of attraction. The pointwise weak consistency and asymptotic normality of the proposed estimator are established. We examine the finite sample performance of our estimator in a simulation study and we illustrate its behavior on a real set of fire insurance data.

This article investigates unsupervised classification techniques for categorical multivariate data. The study employs multivariate multinomial mixture modeling, which is a type of model particularly applicable to multilocus genotypic data. A model selection procedure is used to simultaneously select the number of components and the relevant variables. A non-asymptotic oracle inequality is obtained, leading to the proposal of a new penalized maximum likelihood criterion. The selected model proves to be asymptotically consistent under weak assumptions on the true probability underlying the observations. The main theoretical result obtained in this study suggests a penalty function defined to within a multiplicative parameter. In practice, the data-driven calibration of the penalty function is made possible by slope heuristics. Based on simulated data, this procedure is found to improve the performance of the selection procedure with respect to classical criteria such as $\mathbf{BIC}$ and $\mathbf{AIC}$. The new criterion provides an answer to the question “Which criterion for which sample size?” Examples of real dataset applications are also provided.

This paper discusses a parameter estimation method that employs an unusual estimator called the Dual estimator. For a linear regression model, we obtain two alternative estimators by subtracting or adding a certain vector to the vector of the Ordinary Least Squares Estimator (OLSE). One of them strictly dominates the latter. Moreover, under the normality assumption this estimator is unbiased, and consistent, and has significantly smaller variance than the OLSE. The use of a priori information is a universal way to choose a better alternative. An important property of the proposed method is the possibility of using the strict inequalities as a priori information. In particular, if the external information is that the L2-norm of the OLS estimate exceeds the same norm of a vector of true coefficients, one can choose a better alternative without additional parameters. If it is known that the parameter is restricted by a linear non-strict inequality, the method has a smaller Mean Squared Error than a Constrained Least Squares technique. Finally, a priori information on two possible parameter values can be successfully used for the experimental confirmation of one of two alternative theories, which is illustrated by a verification of the General Theory of Relativity based upon astronomical data.

The main goal in this paper is to propose a new approach to deriving oracle inequalities related to the exponential weighting method. The paper focuses on recovering an unknown vector from noisy data with the help of the family of ordered smoothers [12]. The estimators withing this family are aggregated using the exponential weighting method and the aim is to control the risk of the aggregated estimate. Based on the natural probabilistic properties of the unbiased risk estimate, we derive new oracle inequalities for the mean square risk and show that the exponential weighting permits to improve Kneip’s oracle inequality.

In real life we often have to deal with situations where the sampled observations are independent and share common parameters in their distribution but are not identically distributed. While the methods based on maximum likelihood provide canonical approaches for doing statistical inference in such contexts, it carries with it the usual baggage of lack of robustness to small deviations from the assumed conditions. In the present paper we develop a general estimation method for handling such situations based on a minimum distance approach which exploits the robustness properties of the density power divergence measure (Basu et al. 1998 [2]). We establish the asymptotic properties of the proposed estimators, and illustrate the benefits of our method in case of linear regression.

In this paper we study the problem of estimation of a distribution from data that contain small measurement errors. The only assumption on these errors is that the average absolute measurement error converges to zero for sample size tending to infinity with probability one. In particular we do not assume that the measurement errors are independent with expectation zero. Throughout the paper we assume that the distribution, which has to be estimated, has a density with respect to the Lebesgue-Borel measure.

We show that the empirical measure based on the data with measurement error leads to an uniform consistent estimate of the distribution function. Furthermore, we show that in general no estimate is consistent in the total variation sense for all distributions under the above assumptions. However, in case that the average measurement error converges to zero faster than a properly chosen sequence of bandwidths, the total variation error of the distribution estimate corresponding to a kernel density estimate converges to zero for all distributions. In case of a general additive error model we show that this result even holds if only the average measurement error converges to zero. The results are applied in the context of estimation of the density of residuals in a random design regression model, where the residual error is not independent from the predictor.

We investigate the large sample behavior of a $p$-value based procedure for estimating the threshold level at which a regression function takes off from its baseline value – a problem that frequently arises in environmental statistics, engineering and other related fields. The estimate is constructed via fitting a “stump” function to approximate $p$-values obtained from tests for deviation of the regression function from its baseline level. The smoothness of the regression function in the vicinity of the threshold determines the rate of convergence: a “cusp” of order $k$ at the threshold yields an optimal convergence rate of $n^{-1/{(2k+1)}}$, $n$ being the number of sampled covariates. We show that the asymptotic distribution of the normalized estimate of the threshold, for both i.i.d. and short range dependent errors, is the minimizer of an integrated and transformed Gaussian process. We study the finite sample behavior of confidence intervals obtained through the asymptotic approximation using simulations, consider extensions to short-range dependent data, and apply our inference procedure to two real data sets.

We investigate the frequentist posterior contraction rate of nonparametric Bayesian procedures in linear inverse problems in both the mildly and severely ill-posed cases. A theorem is proved in a general Hilbert space setting under approximation-theoretic assumptions on the prior. The result is applied to non-conjugate priors, notably sieve and wavelet series priors, as well as in the conjugate setting. In the mildly ill-posed setting minimax optimal rates are obtained, with sieve priors being rate adaptive over Sobolev classes. In the severely ill-posed setting, oversmoothing the prior yields minimax rates. Previously established results in the conjugate setting are obtained using this method. Examples of applications include deconvolution, recovering the initial condition in the heat equation and the Radon transform.

The present article contributes a goodness of fit test for the survival function under random right censoring. The test is based on a central limit theorem for the Integrated Square Error of an already existing in the literature kernel survival function estimate. Establishment of its asymptotic distribution yields the proposed test statistic for drawing decision on the null hypothesis of correctness of the assumed survival function. Numerical simulations quantify the empirical nominal level and power of the suggested test for various sample sizes and amounts of censoring and facilitate comparison with the power of the data driven Neyman goodness of fit test for censored samples.

A breakthrough is provided in the study of the existence problem for maximum likelihood estimators (MLE) in the hierarchical generalized linear model (HGLM) of Poisson-gamma type, as well as in the negative binomial regression model. Any more than the uniqueness problem associated, the existence problem of MLE for these models has not yet been studied except in the very special case of the sample. This issue is addressed here for the Poisson-gamma HGLM, and a sufficient condition is obtained to ensure the MLE existence in that case. It is also shown that this condition has the same effect in the negative binomial regression model with the index parameter considered as unknown. In the latter model, the obtained condition appears as a natural extension of the necessary and sufficient condition well known for solving the existence and uniqueness problems for the index parameter MLE in the sample case.

Posterior predictive $p$-values do not in general have uniform distributions under the null hypothesis (except in the special case of ancillary test variables) but instead tend to have distributions more concentrated near 0.5. From different perspectives, such nonuniform distributions have been portrayed as desirable (as reflecting an ability of vague prior distributions to nonetheless yield accurate posterior predictions) or undesirable (as making it more difficult to reject a false model). We explore this tension through two simple normal-distribution examples. In one example, we argue that the low power of the posterior predictive check is desirable from a statistical perspective; in the other, the posterior predictive check seems inappropriate. Our conclusion is that the relevance of the $p$-value depends on the applied context, a point which (ironically) can be seen even in these two toy examples.

A new estimation method for the two-component mixture model introduced in [29] is proposed. This model consists of a two-component mixture of linear regressions in which one component is entirely known while the proportion, the slope, the intercept and the error distribution of the other component are unknown. In spite of good performance for datasets of reasonable size, the method proposed in [29] suffers from a serious drawback when the sample size becomes large as it is based on the optimization of a contrast function whose pointwise computation requires $O(n^{2})$ operations. The range of applicability of the method derived in this work is substantially larger as it relies on a method-of-moments estimator free of tuning parameters whose computation requires $O(n)$ operations. From a theoretical perspective, the asymptotic normality of both the estimator of the Euclidean parameter vector and of the semiparametric estimator of the c.d.f. of the error is proved under weak conditions not involving zero-symmetry assumptions. In addition, an approximate confidence band for the c.d.f. of the error can be computed using a weighted bootstrap whose asymptotic validity is proved. The finite-sample performance of the resulting estimation procedure is studied under various scenarios through Monte Carlo experiments. The proposed method is illustrated on three real datasets of size $n=150$, 51 and 176,343, respectively. Two extensions of the considered model are discussed in the final section: a model with an additional scale parameter for the first component, and a model with more than one explanatory variable.

Birnbaum’s theorem, that the sufficiency and conditionality principles entail the likelihood principle, has engendered a great deal of controversy and discussion since the publication of the result in 1962. In particular, many have raised doubts as to the validity of this result. Typically these doubts are concerned with the validity of the principles of sufficiency and conditionality as expressed by Birnbaum. Technically it would seem, however, that the proof itself is sound. In this paper we use set theory to formalize the context in which the result is proved and show that in fact Birnbaum’s theorem is incorrectly stated as a key hypothesis is left out of the statement. When this hypothesis is added, we see that sufficiency is irrelevant, and that the result is dependent on a well-known flaw in conditionality that renders the result almost vacuous.

In this paper, we propose a general class of penalized profiled semiparametric estimating functions which is applicable to a wide range of statistical models, including quantile regression, survival analysis, and missing data, among others. It is noteworthy that the estimating function can be non-smooth in the parametric and/or nonparametric components. Without imposing a specific functional structure on the nonparametric component or assuming a conditional distribution of the response variable for the given covariates, we establish a unified theory which demonstrates that the resulting estimator for the parametric component possesses the oracle property. Monte Carlo studies indicate that the proposed estimator performs well. An empirical example is also presented to illustrate the usefulness of the new method.

Blanchard and Roquain (2009) presented for the first time methods of adapting the Benjamini-Hochberg (BH) method to data through an estimate of the proportion of true null hypotheses that continue to control the false discovery rate (FDR) under positive dependence in a non-asymptotic setting. However, they are often too conservative to provide a real improvement of the BH method. To obtain adaptive BH methods with proven FDR control improving the original BH method in more situations than what are seen in Blanchard and Roquain (2009), we propose alternative versions of the Blanchard-Roquain methods under some additional assumptions allowing explicit use of pairwise correlations whenever they are available. We offer numerical evidence of improved performances of the proposed alternatives in two scenarios involving test statistics satisfying the positive dependence conditions assumed for the main results.

The Propagation-Separation Approach is an iterative procedure for pointwise estimation of local constant and local polynomial functions. The estimator is defined as a weighted mean of the observations with data-driven, iteratively updated weights. Within homogeneous regions it ensures a similar behavior as non-adaptive smoothing (propagation), while avoiding smoothing among distinct regions (separation). In order to enable a proof of stability of estimates, the authors of the original study introduced an additional memory step aggregating the estimators of the successive iteration steps. Here, we study theoretical properties of the simplified algorithm, where the memory step is omitted. In particular, we introduce a new strategy for the choice of the adaptation parameter yielding propagation and stability for local constant functions with sharp discontinuities.

We propose a semiparametric two-phase regression estimator with a semiparametric generalized propensity score estimator for estimating average treatment effects in the presence of the first-phase sampling. The proposed estimator can be easily extended to any number of treatments and does not rely on a prespecified form of the response or outcome functions. The proposed estimator is shown to reduce bias found in standard estimators that ignore the first-phase sample design, and can have improved efficiency compared to the inverse propensity weighted estimators. Results from simulation studies and from an empirical study of NHANES are presented.

Model selection criteria proposed over the years have become common procedures in applied research. This article examines the true model selection rates of any model selection criteria; with true model meaning the data generating model. The rate at which model selection criteria select the true model is important because the decision of model selection criteria affects both interpretation and prediction.

This article provides a general functional form for the mean function of the true model selection rates process, for any model selection criteria. Until now, no other article has provided a general form for the mean function of true model selection rate processes. As an illustration of the general form, this article provides the mean function for the true model selection rates of two commonly used model selection criteria, Akaike’s Information Criterion (AIC) and Bayesian Information Criterion (BIC). The simulations reveal deeper insight into properties of consistency and efficiency of AIC and BIC. Furthermore, the methodology proposed here for tracking the mean function of model selection procedures, which is based on accuracy of selection, lends itself for determining sufficient sample size in linear models for reliable inference in model selection.

We propose a nonparametric Bayesian method for estimating regression functions that arise as cumulative distribution functions (cdfs) of a stochastically ordered family of distribution supported on [0,1]. The motivating example is estimation of Huff curves which are depth duration curves for heavy storm rainfall. The Bayesian methodology is compared with the linear programming based estimation method that is currently used by the National Oceanic and Atmospheric Administration (NOAA) for producing the Huff curves. The methodology is illustrated with the rainfall data from the rain gauge stations in California, US. Some limited simulation results are provided to illustrate the finite sample performance of the proposed estimator. We also establish consistency of the proposed method.

In conditional copula models, the copula parameter is deterministically linked to a covariate via the calibration function. The latter is of central interest for inference and is usually estimated nonparametrically. However, in many applications it is scientifically important to test whether the calibration function is constant or not. Moreover, a correct model of a constant relationship results in significant gains of statistical efficiency. We develop methodology for testing a parametric formulation of the calibration function against a general alternative and propose a generalized likelihood ratio-type test that enables conditional copula model diagnostics. We derive the asymptotic null distribution of the proposed test and study its finite sample performance using simulations. The method is applied to two data examples.

This paper deals simultaneously with linear structural and functional errors-in-variables models (SEIVM and FEIVM), revisiting in this context the ordinary least squares estimators (LSE) for the slope and intercept of the corresponding simple linear regression. It has been known that, subject to some model conditions, these estimators become weakly and strongly consistent in the linear SEIVM and FEIVM with the measurement errors having finite variances when the explanatory variables have an infinite variance in the SEIVM, and a similar infinite spread in the FEIVM, while otherwise, the LSE’s require an adjustment for consistency with the so-called reliability ratio. In this paper, weak and strong consistency, with and without the possible rates of convergence being determined, is proved for the LSE’s of the slope and intecept, assuming that the measurement errors are in the domain of attraction of the normal law (DAN) and thus are, for the first time, allowed to have infinite variances. Moreover, these results are obtained under the conditions that the explanatory variables are in DAN, have an infinite variance, and dominate the measurement errors in terms of variation in the SEIVM, and under appropriately matching versions of these conditions in the FEIVM. This duality extends a previously known interplay between SEIVM’s and FEIVM’s.

We consider the problem of constructing honest and adaptive confidence sets in $L_{p}$-loss (with $p\geq1$ and $p<\infty$) over sets of Sobolev-type classes, in the setting of non-parametric Gaussian regression. The objective is to adapt the diameter of the confidence sets with respect to the smoothness degree of the underlying function, while ensuring that the true function lies in the confidence interval with high probability. When $p\geq2$, we identify two main regimes, (i) one where adaptation is possible without any restrictions on the model, and (ii) one where critical regions have to be removed. We also prove by a matching lower bound that the size of the regions that we remove can not be chosen significantly smaller. These regimes are shown to depend in a qualitative way on the index $p$, and a continuous transition from $p=2$ to $p=\infty$ is exhibited.

In both classical and Bayesian approaches, statistical inference is unified and generalized by the corresponding decision theory. This is not the case for the likelihood approach to statistical inference, in spite of the manifest success of the likelihood methods in statistics. The goal of the present work is to fill this gap, by extending the likelihood approach in order to cover decision making as well. The resulting likelihood decision functions generalize the usual likelihood methods (such as ML estimators and LR tests), while maintaining some of their key properties, and thus providing a theoretical foundation for established and new likelihood methods.

In this paper we study the semi-parametric problem of the estimation of the long-memory parameter $d$ in a Gaussian long-memory model. Considering a family of priors based on FEXP models, called FEXP priors in Rousseau et al. (2012), we derive concentration rates together with a Bernstein-von Mises theorem for the posterior distribution of $d$, under Sobolev regularity conditions on the short-memory part of the spectral density. Three different variations on the FEXP priors are studied. We prove that one of them leads to the minimax (up to a $\log n$ term) posterior concentration rate for $d$, under Sobolev conditions on the short memory part of the spectral density, while the other two lead to sub-optimal posterior concentration rates in $d$. Interestingly these results are contrary to those obtained in Rousseau et al. (2012) for the global estimation of the spectral density.

In this paper we consider a class of time-changed Lévy processes that can be represented in the form $Y_{s}=X_{\mathcal{T}(s)}$, where $X$ is a Lévy process and $\mathcal{T}$ is a non-negative and non-decreasing stochastic process independent of $X$. The aim of this work is to infer on the Blumenthal-Getoor index of the process $X$ from low-frequency observations of the time-changed Lévy process $Y$. We propose a consistent estimator for this index, derive the minimax rates of convergence and show that these rates can not be improved in general. The performance of the estimator is illustrated by numerical examples.

Least squares fitting is in general not useful for high-dimensional linear models, in which the number of predictors is of the same or even larger order of magnitude than the number of samples. Theory developed in recent years has coined a paradigm according to which sparsity-promoting regularization is regarded as a necessity in such setting. Deviating from this paradigm, we show that non-negativity constraints on the regression coefficients may be similarly effective as explicit regularization if the design matrix has additional properties, which are met in several applications of non-negative least squares (NNLS). We show that for these designs, the performance of NNLS with regard to prediction and estimation is comparable to that of the lasso. We argue further that in specific cases, NNLS may have a better $\ell_{\infty}$-rate in estimation and hence also advantages with respect to support recovery when combined with thresholding. From a practical point of view, NNLS does not depend on a regularization parameter and is hence easier to use.

This erratum concerns paper EJS723: “Group symmetry and covariance regularization”, Electronic Journal of Statistics, Vol. 6 (2012) 1600–1640.

The paper contains a notational error in the statement of Schur’s lemma and the paragraph immediately succeeding it. The corrected version is below. We would like to thank Ilya Soloveychik for pointing out the error to us.

An abundant regression is one in which most of the predictors contribute information about the response, which is contrary to the common notion of a sparse regression where few of the predictors are relevant. We discuss asymptotic characteristics of methodology for prediction in abundant linear regressions as the sample size and number of predictors increase in various alignments. We show that some of the estimators can perform well for the purpose of prediction in abundant high-dimensional regressions.

Structural breaks have become an important research topic in time series analysis and in many fields of application, e.g. econometrics, hydrology, seismology, engineering and industry, chemometrics, and medicine. In most phenomena encountered in the real world modeling assuming no change of structure may obviously lead to inconsistent estimates and poor forecasts. Much research work has been carried out in the past decades and efforts mostly concentrated on the identification of points in time where structural breaks possibly occur and on the development of statistical tests of absence of breaks. Common procedures are the CUSUM based statistics and the F based statistics proposed by Bai and Perron in several papers and technical reports. The need of specification of a probability distribution for conducting the statistical tests has been often a drawback in practical applications. If the distribution is misspecified the test may turn to be unreliable. In this paper a distribution free procedure for identification of change points and testing for structural breaks is proposed based on the empirical likelihood and the empirical likelihood ratio statistics. A comparison with the F test and the CUSUM test is performed on both simulated and real world data. The size and power of the test is comparable with existing procedures and the identification procedure is generally accurate and effective.

We study the asymptotic properties of Lasso+mLS and Lasso+ Ridge under the sparse high-dimensional linear regression model: Lasso selecting predictors and then modified Least Squares (mLS) or Ridge estimating their coefficients. First, we propose a valid inference procedure for parameter estimation based on parametric residual bootstrap after Lasso+ mLS and Lasso+Ridge. Second, we derive the asymptotic unbiasedness of Lasso+mLS and Lasso+Ridge. More specifically, we show that their biases decay at an exponential rate and they can achieve the oracle convergence rate of $s/n$ (where $s$ is the number of nonzero regression coefficients and $n$ is the sample size) for mean squared error (MSE). Third, we show that Lasso+mLS and Lasso+Ridge are asymptotically normal. They have an oracle property in the sense that they can select the true predictors with probability converging to $1$ and the estimates of nonzero parameters have the same asymptotic normal distribution that they would have if the zero parameters were known in advance. In fact, our analysis is not limited to adopting Lasso in the selection stage, but is applicable to any other model selection criteria with exponentially decay rates of the probability of selecting wrong models.