Given a bounded class of functions $\mathscr{G}$ and independent random variables $X_{1},\ldots,X_{n}$, we provide an upper bound for the expectation of the supremum of the empirical process over elements of $\mathscr{G}$ having a small variance. Our bound applies when $\mathscr{G}$ is a VC-subgraph or a VC-major class and it is of smaller order than those one could get by using a universal entropy bound over the whole class $\mathscr{G}$. It also involves explicit constants and does not require the knowledge of the entropy of $\mathscr{G}$.

Several new estimation methods have been recently proposed for the linear regression model with observation errors in the design. Different assumptions on the data generating process have motivated different estimators and analysis. In particular, the literature considered (1) observation errors in the design uniformly bounded by some $\bar{\delta}$, and (2) zero-mean independent observation errors. Under the first assumption, the rates of convergence of the proposed estimators depend explicitly on $\bar{\delta}$, while the second assumption has been essentially applied when an estimator for the second moment of the observational error is available. This work proposes and studies two new estimators which, compared to other procedures for regression models with errors in the design, exploit an additional $\ell_{\infty}$-norm regularization. The first estimator is applicable when both (1) and (2) hold but does not require an estimator for the second moment of the observational error. The second estimator is applicable under (2) and requires an estimator for the second moment of the observation error. Importantly, we impose no assumption on the accuracy of this pilot estimator, in contrast to the previously known procedures. As the recent proposals, we allow the number of covariates to be much larger than the sample size. We establish the rates of convergence of the estimators and compare them with the bounds obtained for related estimators in the literature. These comparisons show interesting insights on the interplay of the assumptions and the achievable rates of convergence.

We provide a general theoretical framework to derive Bernstein-von Mises theorems for functionals of the covariance matrix and its inverse. The conditions on functionals and priors are explicit and easy to check. Results are obtained for various functionals including entries of covariance matrix, entries of precision matrix, quadratic forms, log-determinant, eigenvalues in the Bayesian Gaussian covariance/precision matrix estimation setting, as well as for Bayesian linear and quadratic discriminant analysis.

We present a novel Bayesian nonparametric regression model for covariates $X$ and continuous response variable $Y\in\mathbb{R}$. The model is parametrized in terms of marginal distributions for $Y$ and $X$ and a regression function which tunes the stochastic ordering of the conditional distributions $F(y|x)$. By adopting an approximate composite likelihood approach, we show that the resulting posterior inference can be decoupled for the separate components of the model. This procedure can scale to very large datasets and allows for the use of standard, existing, software from Bayesian nonparametric density estimation and Plackett-Luce ranking estimation to be applied. As an illustration, we show an application of our approach to a US Census dataset, with over 1,300,000 data points and more than 100 covariates.

We study confidence regions and approximate chi-squared tests for variable groups in high-dimensional linear regression. When the size of the group is small, low-dimensional projection estimators for individual coefficients can be directly used to construct efficient confidence regions and p-values for the group. However, the existing analyses of low-dimensional projection estimators do not directly carry through for chi-squared-based inference of a large group of variables without inflating the sample size by a factor of the group size. We propose to de-bias a scaled group Lasso for chi-squared-based statistical inference for potentially very large groups of variables. We prove that the proposed methods capture the benefit of group sparsity under proper conditions, for statistical inference of the noise level and variable groups, large and small. Such benefit is especially strong when the group size is large.

In large-scale modern data analysis, first-order optimization methods are usually favored to obtain sparse estimators in high dimensions. This paper performs theoretical analysis of a class of iterative thresholding based estimators defined in this way. Oracle inequalities are built to show the nearly minimax rate optimality of such estimators under a new type of regularity conditions. Moreover, the sequence of iterates is found to be able to approach the statistical truth within the best statistical accuracy geometrically fast. Our results also reveal different benefits brought by convex and nonconvex types of shrinkage.

The classical Lorenz curve is often used to depict inequality in a population of incomes, and the associated Gini coefficient is relied upon to make comparisons between different countries and other groups. The sample estimates of these moment-based concepts are sensitive to outliers and so we investigate the extent to which quantile-based versions can capture income inequality and lead to robust procedures. Distribution-free interval estimates of the associated coefficients of inequality are obtained, as well as sample sizes required to estimate them to a given accuracy. Convexity, transference and robustness of the measures are examined and illustrated.

We consider the detection problem of correlations in a $p$-dimensional Gaussian vector, when we observe $n$ independent, identically distributed random vectors, for $n$ and $p$ large. We assume that the covariance matrix varies in some ellipsoid with parameter $\alpha >1/2$ and total energy bounded by $L>0$.

We propose a test procedure based on a U-statistic of order 2 which is weighted in an optimal way. The weights are the solution of an optimization problem, they are constant on each diagonal and non-null only for the $T$ first diagonals, where $T=o(p)$. We show that this test statistic is asymptotically Gaussian distributed under the null hypothesis and also under the alternative hypothesis for matrices close to the detection boundary. We prove upper bounds for the total error probability of our test procedure, for $\alpha>1/2$ and under the assumption $T=o(p)$ which implies that $n=o(p^{2\alpha})$. We illustrate via a numerical study the behavior of our test procedure.

Moreover, we prove lower bounds for the maximal type II error and the total error probabilities. Thus we obtain the asymptotic and the sharp asymptotically minimax separation rate $\widetilde{\varphi}=(C(\alpha,L)n^{2}p)^{-\alpha/(4\alpha +1)}$, for $\alpha>3/2$ and for $\alpha >1$ together with the additional assumption $p=o(n^{4\alpha -1})$, respectively. We deduce rate asymptotic minimax results for testing the inverse of the covariance matrix.

We construct an adaptive test procedure with respect to the parameter $\alpha$ and show that it attains the rate $\widetilde{\psi}=(n^{2}p/\ln\ln(n\sqrt{p}))^{-\alpha/(4\alpha +1)}$.

This paper considers naive and wild bootstrap procedures to construct pointwise confidence intervals for a nonparametric regression function when the predictor is of functional nature and when the data are dependent. Assuming $\alpha$-mixing conditions on the sample, the asymptotic validity of both procedures is obtained. A simulation study shows promising results when finite sample sizes are used, while an application to electricity demand data illustrates its usefulness in practice.

In this paper, we consider the problem of (multiple) change-point detection in panel data. We propose the double CUSUM statistic which utilises the cross-sectional change-point structure by examining the cumulative sums of ordered CUSUMs at each point. The efficiency of the proposed change-point test is studied, which is reflected on the rate at which the cross-sectional size of a change is permitted to converge to zero while it is still detectable. Also, the consistency of the proposed change-point detection procedure based on the binary segmentation algorithm, is established in terms of both the total number and locations (in time) of the estimated change-points. Motivated by the representation properties of the Generalised Dynamic Factor Model, we propose a bootstrap procedure for test criterion selection, which accounts for both cross-sectional and within-series correlations in high-dimensional data. The empirical performance of the double CUSUM statistics, equipped with the proposed bootstrap scheme, is investigated in a comparative simulation study with the state-of-the-art. As an application, we analyse the log returns of S&P 100 component stock prices over a period of one year.

Feature screening via the marginal screening ([5]; [7]) has gained special attention for high dimensional regression problems. However, their results are confined to the generalized linear model ($\mathrm{GLM}$) with the exponential family of distributions. This inspires us to explore the suitability of applying screening procedures to more general models, for example without assuming either the explicit form of distributions or parametric forms between response and covariates. In this paper, we extend the marginal screening procedure, by means of Bregman divergence (${\mathrm{BD}}$) as the loss function, to include not only the $\mathrm{GLM}$ but also the quasi-likelihood model. A sure screening property for the resulting screening procedure is established under this very general framework, assuming only certain moment conditions and tail properties, where the dimensionality $p_{n}$ is allowed to grow with the sample size $n$ as fast as $\log(p_{n})=O(n^{a})$ for some $a\in(0,1)$. Simulation and real data studies illustrate that a two-step procedure, which combines the feature screening in the first step and a penalized-${\mathrm{BD}}$ estimation in the second step, is practically applicable to identifying the set of relevant variables and achieving good estimation of model parameters, with the computational cost much less than those without using the screening step.

The problem of the evaluation and estimation of the mean density of random closed sets in $\mathbb{R} ^{d}$ with integer Hausdorff dimension $0<n<d$, is of great interest in many different scientific and technological fields. Among the estimators of the mean density available in literature, the so-called “Minkowski content”-based estimator reveals its benefits in applications in the non-stationary cases. We introduce here a multivariate version of such estimator, and we study its asymptotical properties by means of large and moderate deviation results. In particular we prove that the estimator is strongly consistent and asymptotically Normal. Furthermore we also provide confidence regions for the mean density of the involved random closed set in $m\geq1$ distinct points $x_{1},\ldots,x_{m}\in\mathbb{R} ^{d}$.

The detection of change-points in a spatially or time-ordered data sequence is an important problem in many fields such as genetics and finance. We derive the asymptotic distribution of a statistic recently suggested for detecting change-points, thus establishing its validity. Simulation of its estimated limit distribution leads to a new and computationally efficient change-point detection algorithm, which can be used on very long signals. To finish, we briefly assess this new algorithm on one- and multi-dimensional data.

We propose a thresholding least-squares method of inference for high-dimensional regression models when the number of parameters, $p$, tends to infinity with the sample size, $n$. Extending the asymptotic behavior of the F-test in high dimensions, we establish the oracle property of the thresholding least-squares estimator when $p=o(n)$. We propose two automatic selection procedures for the thresholding parameter using Scheffé and Bonferroni methods. We show that, under additional regularity conditions, the results continue to hold even if $p=\exp(o(n))$. Lastly, we show that, if properly centered, the residual-bootstrap estimator of the distribution of thresholding least-squares estimator is consistent, while a naive bootstrap estimator is inconsistent. In an intensive simulation study, we assess the finite sample properties of the proposed methods for various sample sizes and model parameters. The analysis of a real world data set illustrates an application of the methods in practice.

The main objective of this paper is to establish bootstrap uniform functional central limit theorem for Harris recurrent Markov chains over uniformly bounded classes of functions. We show that the result can be generalized also to the unbounded case. To avoid some complicated mixing conditions, we make use of the well-known regeneration properties of Markov chains. We show that in the atomic case the proof of the bootstrap uniform central limit theorem for Markov chains for functions dominated by a function in $L^{2}$ space proposed by Radulović (2004) can be significantly simplified. Finally, we prove bootstrap uniform central limit theorems for Fréchet differentiable functionals in a Markovian setting.

In this paper we consider the problem of uncertainty related to growth through innovations. We study a stylized, although rich, growth model, in which the stochastic innovations follow a Bayesian nonparametric model, and provide the full taxonomy of the asymptotic equilibria. In most cases the variability around the average aggregate behaviour does not vanish asymptotically: this requires to accompany usual macroeconomic mean predictions with some measure of uncertainty, which is readily yielded by the adopted Bayesian nonparametric approach. Moreover, we discover that the extent of the asymptotic variability is the result of the interaction between the rate at which the economy creates new sectors and the concavity of returns in sector specific technologies.

In this paper, we further develop the approach, originating in [13], to “computation-friendly” hypothesis testing via Convex Programming. Most of the existing results on hypothesis testing aim to quantify in a closed analytic form separation between sets of distributions allowing for reliable decision in precisely stated observation models. In contrast to this descriptive (and highly instructive) traditional framework, the approach we promote here can be qualified as operational – the testing routines and their risks are yielded by an efficient computation. All we know in advance is that, under favorable circumstances, specified in [13], the risk of such test, whether high or low, is provably near-optimal under the circumstances. As a compensation for the lack of “explanatory power,” this approach is applicable to a much wider family of observation schemes and hypotheses to be tested than those where “closed form descriptive analysis” is possible.

In the present paper our primary emphasis is on computation: we make a step further in extending the principal tool developed in [13] – testing routines based on affine detectors – to a large variety of testing problems. The price of this development is the loss of blanket near-optimality of the proposed procedures (though it is still preserved in the observation schemes studied in [13], which now become particular cases of the general setting considered here).

Distances to compact sets are widely used in the field of Topological Data Analysis for inferring geometric and topological features from point clouds. In this context, the distance to a probability measure (DTM) has been introduced by Chazal et al., 2011b as a robust alternative to the distance a compact set. In practice, the DTM can be estimated by its empirical counterpart, that is the distance to the empirical measure (DTEM). In this paper we give a tight control of the deviation of the DTEM. Our analysis relies on a local analysis of empirical processes. In particular, we show that the rate of convergence of the DTEM directly depends on the regularity at zero of a particular quantile function which contains some local information about the geometry of the support. This quantile function is the relevant quantity to describe precisely how difficult is a geometric inference problem. Several numerical experiments illustrate the convergence of the DTEM and also confirm that our bounds are tight.

The Bergsma-Dassios sign covariance is a recently proposed extension of Kendall’s tau. In contrast to tau or also Spearman’s rho, the new sign covariance $\tau^{*}$ vanishes if and only if the two considered random variables are independent. Specifically, this result has been shown for continuous as well as discrete variables. We develop large-sample distribution theory for the empirical version of $\tau^{*}$. In particular, we use theory for degenerate U-statistics to derive asymptotic null distributions under independence and demonstrate in simulations that the limiting distributions give useful approximations.

Various forms of penalty functions have been developed for regularized estimation and variable selection. Screening approaches are often used to reduce the number of covariate before penalized estimation. However, in certain problems, the number of covariates remains large after screening. For example, in genome-wide association (GWA) studies, the purpose is to identify Single Nucleotide Polymorphisms (SNPs) that are associated with certain traits, and typically there are millions of SNPs and thousands of samples. Because of the strong correlation of nearby SNPs, screening can only reduce the number of SNPs from millions to tens of thousands and the variable selection problem remains very challenging. Several penalty functions have been proposed for such high dimensional data. However, it is unclear which class of penalty functions is the appropriate choice for a particular application. In this paper, we conduct a theoretical analysis to relate the ranges of tuning parameters of various penalty functions with the dimensionality of the problem and the minimum effect size. We exemplify our theoretical results in several penalty functions. The results suggest that a class of penalty functions that bridges $L_{0}$ and $L_{1}$ penalties requires less restrictive conditions on dimensionality and minimum effect sizes in order to attain the two fundamental goals of penalized estimation: to penalize all the noise to be zero and to obtain unbiased estimation of the true signals. The penalties such as SICA and Log belong to this class, but they have not been used often in applications. The simulation and real data analysis using GWAS data suggest the promising applicability of such class of penalties.

Continuous goodness-of-fit testing is a classical problem in statistics. Despite having low power for detecting deviations at the tail of a distribution, the most popular test is based on the Kolmogorov-Smirnov statistic. While similar variance-weighted statistics such as Anderson-Darling and the Higher Criticism statistic give more weight to tail deviations, as shown in various works, they still mishandle the extreme tails.

As a viable alternative, in this paper we study some of the statistical properties of the exact $M_{n}$ statistics of Berk and Jones. In particular we show that they are consistent and asymptotically optimal for detecting a wide range of rare-weak mixture models. Additionally, we present a new computationally efficient method to calculate $p$-values for any supremum-based one-sided statistic, including the one-sided $M_{n}^{+},M_{n}^{-}$ and $R_{n}^{+},R_{n}^{-}$ statistics of Berk and Jones and the Higher Criticism statistic. Finally, we show that $M_{n}$ compares favorably to related statistics in several finite-sample simulations.

We discuss in detail the asymptotic distribution of sample expectiles. First, we show uniform consistency under the assumption of a finite mean. In case of a finite second moment, we show that for expectiles other then the mean, only the additional assumption of continuity of the distribution function at the expectile implies asymptotic normality, otherwise, the limit is non-normal. For a continuous distribution function we show the uniform central limit theorem for the expectile process. If, in contrast, the distribution is heavy-tailed, and contained in the domain of attraction of a stable law with $1<\alpha<2$, then we show that the expectile is also asymptotically stable distributed. Our findings are illustrated in a simulation section.

We consider the problem of testing for multivariate independence in independent component (IC) models. Under a symmetry assumption, we develop parametric and nonparametric (signed-rank) tests. Unlike in independent component analysis (ICA), we allow for the singular cases involving more than one Gaussian independent component. The proposed rank tests are based on componentwise signed ranks, à la Puri and Sen. Unlike the Puri and Sen tests, however, our tests (i) are affine-invariant and (ii) are, for adequately chosen scores, locally and asymptotically optimal (in the Le Cam sense) at prespecified densities. Asymptotic local powers and asymptotic relative efficiencies with respect to Wilks’ LRT are derived. Finite-sample properties are investigated through a Monte-Carlo study.

In this paper, we consider the problem of testing the mean vector in the high-dimensional settings. We proposed a new robust scalar transform invariant test based on spatial sign. The proposed test statistic is asymptotically normal under elliptical distributions. Simulation studies show that our test is very robust and efficient in a wide range of distributions.

We introduce and study a subclass of joint Bernoulli distributions which has the palindromic property. For such distributions the vector of joint probabilities is unchanged when the order of the elements is reversed. We prove for binary variables that the palindromic property is equivalent to zero constraints on all odd-order interaction parameters, be it in parameterizations which are log-linear, linear or multivariate logistic. In particular, we derive the one-to-one parametric transformations for these three types of model specifications and give simple closed forms of maximum likelihood estimates. Several special cases are discussed and a case study is described.

Motivated by spectral analysis of replicated brain signal time series, we propose a functional mixed effects approach to model replicate-specific spectral densities as random curves varying about a deterministic population-mean spectrum. In contrast to existing work, we do not assume the replicate-specific spectral curves to be independent, i.e. there may exist explicit correlation between different replicates in the population. By projecting the replicate-specific curves onto an orthonormal wavelet basis, estimation and prediction is carried out under an equivalent linear mixed effects model in the wavelet coefficient domain. To cope with potentially very localized features of the spectral curves, we develop estimators and predictors based on a combination of generalized least squares estimation and nonlinear wavelet thresholding, including asymptotic confidence sets for the population-mean curve. We derive $L_{2}$-risk bounds for the nonlinear wavelet estimator of the population-mean curve–a result that reflects the influence of correlation between different curves in the replicate population– and consistency of the estimators of the inter- and intra-curve correlation structure in an appropriate sparseness class of functions. To illustrate the proposed functional mixed effects model and our estimation and prediction procedures, we present several simulated time series data examples and we analyze a motivating brain signal dataset recorded during an associative learning experiment.

This paper develops an empirical likelihood (EL) approach to testing for stochastic ordering between two univariate distributions under right censorship. The proposed test is based on a maximally selected local EL statistic. The asymptotic null distribution is expressed in terms of a Brownian bridge. The new procedure is shown via a simulation study to have superior power to the log-rank and weighted Kaplan–Meier tests under crossing hazard alternatives. The approach is illustrated using data from a randomized clinical trial involving the treatment of severe alcoholic hepatitis.

The Lasso is a popular regularization method that can simultaneously do estimation and model selection. It contains a regularization parameter, and several information criteria have been proposed for selecting its proper value. While any of them would assure consistency in model selection, we have no appropriate rule to choose between the criteria. Meanwhile, a finite correction to the AIC has been provided in a Gaussian regression setting. The finite correction is theoretically assured from the viewpoint not of the consistency but of minimizing the prediction error and does not have the above-mentioned difficulty. Our aim is to derive such a criterion for the Lasso in generalized linear models. Towards this aim, we derive a criterion from the original definition of the AIC, that is, an asymptotically unbiased estimator of the Kullback-Leibler divergence. This becomes the finite correction in the Gaussian regression setting, and so our criterion can be regarded as its generalization. Our criterion can be easily obtained and requires fewer computational tasks than does cross-validation, but simulation studies and real data analyses indicate that its performance is almost the same as or superior to that of cross-validation. Moreover, our criterion is extended for a class of other regularization methods.

In the paper row-wise periodically correlated triangular arrays are considered. The period length is assumed to grow in time. The Fourier decomposition of the mean and autocovariance functions for each row of the matrix is presented. To construct bootstrap estimators of the Fourier coefficients two block bootstrap techniques are used. These are the circular version of the Generalized Seasonal Block Bootstrap and the Circular Block Bootstrap. Consistency results for both methods are presented. Bootstrap-t equal-tailed confidence intervals for parameters of interest are constructed. Results are illustrated by an example based on simulated data.

Recently, several authors have re-examined the power of the classical $F$-test in a non-Gaussian linear regression under a “large-$p$, large-$n$” framework [e.g. 29, 27]. They highlight the loss of power as the number of regressors $p$ increases relative to sample size $n$. These papers essentially focus only on the overall test of the null hypothesis that all $p$ slope coefficients are equal to zero. Here, we consider the general case of testing $q$ linear hypotheses on the $p+1$-dimensional regression parameter vector that includes $p$ slope coefficients and an intercept parameter. In the case of Gaussian design, we describe the dependence of the local asymptotic power function on both the relative number of parameters $p/n$ and the relative number of hypotheses $q/n$ being tested, showing that the negative effect of dimensionality is less severe if the number of hypotheses is small. Using the recent work of [23] on high-dimensional sample covariance matrices we are also able to substantially generalize previous results for non-Gaussian regressors.

We consider the problem of detecting a rectangle of activation in a grid of sensors in $d$-dimensions with noisy measurements. This has applications to massive surveillance projects and anomaly detection in large datasets in which one detects anomalously high measurements over rectangular regions, or more generally, blobs. Recently, the asymptotic distribution of a multiscale scan statistic was established in [18] under the null hypothesis, using non-constant boundary crossing probabilities for locally-stationary Gaussian random fields derived in [8]. Using a similar approach, we derive the exact asymptotic level and power of four variants of the scan statistic: an oracle scan that knows the dimensions of the activation rectangle; the multiscale scan statistic just mentioned; the adaptive variant; and an $\epsilon $-net approximation to the latter, in the spirit of [3]. This approximate scan runs in time near-linear in the size of the grid and achieves the same asymptotic level and power as the adaptive scan, and has a poly-logarithmic time parallel implementation. We complement our theory with some numerical experiments, and make some practical recommendations.

We propose a regression tree procedure to estimate the conditional distribution of a variable which is not directly observed due to censoring. The model that we consider is motivated by applications in insurance, including the analysis of guarantees that involve durations, and claim reserving. We derive consistency results for our procedure, and for the selection of an optimal subtree using a pruning strategy. These theoretical results are supported by a simulation study, and two applications involving insurance datasets. The first concerns income protection insurance, while the second deals with reserving in third-party liability insurance.

Let $\mathcal{S}_{m}$ be the set of all $m\times m$ density matrices (Hermitian positively semi-definite matrices of unit trace). Consider a problem of estimation of an unknown density matrix $\rho\in\mathcal{S}_{m}$ based on outcomes of $n$ measurements of observables $X_{1},\dots,X_{n}\in\mathbb{H}_{m}$ ($\mathbb{H}_{m}$ being the space of $m\times m$ Hermitian matrices) for a quantum system identically prepared $n$ times in state $\rho.$ Outcomes $Y_{1},\dots,Y_{n}$ of such measurements could be described by a trace regression model in which $\mathbb{E}_{\rho}(Y_{j}|X_{j})={\rm tr}(\rho X_{j}),j=1,\dots,n.$ The design variables $X_{1},\dots,X_{n}$ are often sampled at random from the uniform distribution in an orthonormal basis $\{E_{1},\dots,E_{m^{2}}\}$ of $\mathbb{H}_{m}$ (such as Pauli basis). The goal is to estimate the unknown density matrix $\rho$ based on the data $(X_{1},Y_{1}),\dots,(X_{n},Y_{n}).$ Let \[\hat{Z}:=\frac{m^{2}}{n}\sum_{j=1}^{n}Y_{j}X_{j}\] and let $\check{\rho}$ be the projection of $\hat{Z}$ onto the convex set $\mathcal{S}_{m}$ of density matrices. It is shown that for estimator $\check{\rho}$ the minimax lower bounds in classes of low rank density matrices (established earlier) are attained up logarithmic factors for all Schatten $p$-norm distances, $p\in[1,\infty]$ and for Bures version of quantum Hellinger distance. Moreover, for a slightly modified version of estimator $\check{\rho}$ the same property holds also for quantum relative entropy (Kullback-Leibler) distance between density matrices.

Community detection in networks has drawn much attention in diverse fields, especially social sciences. Given its significance, there has been a large body of literature with approaches from many fields. Here we present a statistical framework that is representative, extensible, and that yields an estimator with good properties. Our proposed approach considers a stochastic blockmodel based on a logistic regression formulation with node correction terms. We follow a Bayesian approach that explicitly captures the community behavior via prior specification. We further adopt a data augmentation strategy with latent Pólya-Gamma variables to obtain posterior samples. We conduct inference based on a principled, canonically mapped centroid estimator that formally addresses label non-identifiability and captures representative community assignments. We demonstrate the proposed model and estimation on real-world as well as simulated benchmark networks and show that the proposed model and estimator are more flexible, representative, and yield smaller error rates when compared to the MAP estimator from classical degree-corrected stochastic blockmodels.

In many real-world classification problems, the labels of training examples are randomly corrupted. Most previous theoretical work on classification with label noise assumes that the two classes are separable, that the label noise is independent of the true class label, or that the noise proportions for each class are known. In this work, we give conditions that are necessary and sufficient for the true class-conditional distributions to be identifiable. These conditions are weaker than those analyzed previously, and allow for the classes to be nonseparable and the noise levels to be asymmetric and unknown. The conditions essentially state that a majority of the observed labels are correct and that the true class-conditional distributions are “mutually irreducible,” a concept we introduce that limits the similarity of the two distributions. For any label noise problem, there is a unique pair of true class-conditional distributions satisfying the proposed conditions, and we argue that this pair corresponds in a certain sense to maximal denoising of the observed distributions.

Our results are facilitated by a connection to “mixture proportion estimation,” which is the problem of estimating the maximal proportion of one distribution that is present in another. We establish a novel rate of convergence result for mixture proportion estimation, and apply this to obtain consistency of a discrimination rule based on surrogate loss minimization. Experimental results on benchmark data and a nuclear particle classification problem demonstrate the efficacy of our approach.

We propose a methodology based on a reparameterized Birnbaum-Saunders regression model with varying precision, which generalizes the existing works in the literature on the topic. This methodology includes the estimation of model parameters, hypothesis tests for the precision parameter, a residual analysis and influence diagnostic tools. Simulation studies are conducted to evaluate its performance. We apply it to two real-world case-studies to show its potential with the R software.

We present a new matrix-free residual maximum likelihood (REML) analysis for irregularly spaced spatial data, where observations usually represent average values over very small regions that are interpreted as points. The REML analysis is obtained after embedding the sampling locations in a fine scale rectangular lattice, treating unobserved sites as missing data. The spatial random fields considered here are based on fractional Laplacian differencing on the lattice and they are unique in approximating continuum intrinsic Matérn dependence. Here, using the h-likelihood method, we derive REML estimating equations that allow for singular precision matrices, estimation of covariate effects, prediction of unobserved spatial effects and REML estimation of precision parameters as a solution to an explicit gamma non-linear model. Furthermore, we devise a sophisticated computational algorithm that enables us to achieve scalable matrix-free statistical computations. In particular, these matrix-free computations include the use of (1) the two-dimensional discrete cosine transformation that arises in the spectral decomposition of the precision matrix of our spatial random fields and that allows fast matrix-free matrix-vector multiplication, (2) a matrix-free pre-conditioned Lanczos algorithm that solves non-sparse matrix equations with linear constraints, (3) a matrix-free Hutchinson’s trace estimator that stochastically approximates the trace of a matrix, (4) a robust trust region method that always finds a local maximum of the non-concave residual log-likelihood function and (5) some preliminary computations of the log REML likelihood function based on Taylor series approximation. Using computer experiments, we provide further understanding on not just the number and values but also the basins of attraction of the local and global maxima of the REML function. This understanding significantly simplifies the problem of finding global maxima. We further demonstrate through computer experiments that our matrix-free REML estimators attain both efficiency and geostatistical inference, and surpass the widely used INLA methods in computational times. We provide an extensive application on mapping ground water arsenic concentration in Bangladesh, indicating numeric consistency of results and robustness of inference to changes of lattice spacing. The paper closes with some discussions that include computations in the stationary case, conditional simulations and matrix-free MCMC computations.

In order to identify important variables that are involved in making optimal treatment decision, Lu, Zhang and Zeng (2013) proposed a penalized least squared regression framework for a fixed number of predictors, which is robust against the misspecification of the conditional mean model. Two problems arise: (i) in a world of explosively big data, effective methods are needed to handle ultra-high dimensional data set, for example, with the dimension of predictors is of the non-polynomial (NP) order of the sample size; (ii) both the propensity score and conditional mean models need to be estimated from data under NP dimensionality.

In this paper, we propose a robust procedure for estimating the optimal treatment regime under NP dimensionality. In both steps, penalized regressions are employed with the non-concave penalty function, where the conditional mean model of the response given predictors may be misspecified. The asymptotic properties, such as weak oracle properties, selection consistency and oracle distributions, of the proposed estimators are investigated. In addition, we study the limiting distribution of the estimated value function for the obtained optimal treatment regime. The empirical performance of the proposed estimation method is evaluated by simulations and an application to a depression dataset from the STAR∗D study.

We study nonparametric clustering of smooth random curves on the basis of the $L^{2}$ gradient flow associated to a pseudo-density functional and we discuss the conditions under which the clustering is well-defined both at the population and at the sample level. We provide an algorithm to idenify significant local modes of the estimated pseudo-density, which are associated to informative sample clusters, and we prove its consistency and other statistical properties. Our theory is developed under weak assumptions, which essentially reduce to the integrability of the random curves. If the underlying probability distribution is supported on a finite-dimensional subspace, we show that the proposed pseudo-density functional and the expectation of a kernel density estimator induce the same gradient flow, hence the same population clustering. Although our theory is developed for smooth curves that belong to a potentially infinite-dimensional functional space, we provide consistent procedures that can be used with real functional data (discretized and noisy curves). We illustrate these procedures by means of applications both on simulated and real datasets.

This paper focuses on the prominent sphericity test when the dimension $p$ is much lager than sample size $n$. The classical likelihood ratio test(LRT) is no longer applicable when $p\gg n$. Therefore a Quasi-LRT is proposed and its asymptotic distribution of the test statistic under the null when $p/n\rightarrow\infty,n\rightarrow\infty$ is well established in this paper. We also re-examine the well-known John’s invariant test for sphericity in this ultra-dimensional setting. An amazing result from the paper states that John’s test statistic has exactly the same limiting distribution under the ultra-dimensional setting with under other high-dimensional settings known in the literature. Therefore, John’s test has been found to possess the powerful dimension-proof property, which keeps exactly the same limiting distribution under the null with any $(n,p)$-asymptotic, i.e. $p/n\rightarrow[0,\infty]$, $n\rightarrow\infty$. Nevertheless, the asymptotic distribution of both test statistics under the alternative hypothesis with a general population covariance matrix is also derived and incorporates the null distributions as special cases. The power functions are presented and proven to converge to 1 as $p/n\rightarrow\infty,~n\rightarrow\infty,~n^{3}/p=O(1)$. All asymptotic results are derived for general population with finite fourth order moment. Numerical experiments are implemented to illustrate the finite sample performance of the results.

Empirical likelihood serves as a good nonparametric alternative to the traditional parametric likelihood. The former involves much less assumptions than the latter, but very often gets the same asymptotic inferential efficiency. While empirical likelihood has been studied quite extensively in the frequentist literature, the corresponding Bayesian literature is somewhat sparse. Bayesian methods hold promise, however, especially with the availability of historical information, which often can be used successfully for the construction of priors. In addition, Bayesian methods very often overcome the curse of dimensionality by providing suitable dimension reduction through judicious use of priors and analyzing data with the resultant posteriors. In this paper, we provide asymptotic expansion of posteriors for a very general class of priors along with the empirical likelihood and its variations, such as the exponentially tilted empirical likelihood and the Cressie–Read version of the empirical likelihood. Other than obtaining the celebrated Bernstein–von Mises theorem as a special case, our approach also aids in finding non-subjective priors based on empirical likelihood and its variations as mentioned above.

This paper concerns the robust regression model when the number of predictors and the number of observations grow in a similar rate. Theory for M-estimators in this regime has been recently developed by several authors (El Karoui et al., 2013; Bean et al., 2013; Donoho and Montanari, 2013). Motivated by the inability of M-estimators to successfully estimate the Euclidean norm of the coefficient vector, we consider a Bayesian framework for this model. We suggest a two-component mixture of normals prior for the coefficients and develop a Gibbs sampler procedure for sampling from relevant posterior distributions, while utilizing a scale mixture of normal representation for the error distribution. Unlike M-estimators, the proposed Bayes estimator is consistent in the Euclidean norm sense. Simulation results demonstrate the superiority of the Bayes estimator over traditional estimation methods.

Verification bias is a well-known problem that may occur in the evaluation of predictive ability of diagnostic tests. When a binary disease status is considered, various solutions can be found in the literature to correct inference based on usual measures of test accuracy, such as the receiver operating characteristic (ROC) curve or the area underneath. Evaluation of the predictive ability of continuous diagnostic tests in the presence of verification bias for an ordinal three-class disease status is here discussed. In particular, several verification bias-corrected estimators of the ROC surface and of the volume underneath are proposed. Consistency and asymptotic normality of the proposed estimators are established and their finite sample behavior is investigated by means of Monte Carlo simulation studies. Two illustrations are also given.

We study the model $Y_{i}=X_{i}U_{i},\;i=1,\ldots,n$ where the $U_{i}$’s are i.i.d. with $\beta(1,k)$ density, $k\ge1$, $k$ integer, the $X_{i}$’s are i.i.d., nonnegative with unknown density $f$. The sequences $(X_{i}),(U_{i}),$ are independent. We aim at estimating $f$ on ${\mathbb{R}}^{+}$ from the observations $(Y_{1},\dots,Y_{n})$. We propose projection estimators using a Laguerre basis. A data-driven procedure is described in order to select the dimension of the projection space, which performs automatically the bias variance compromise. Then, we give upper bounds on the ${\mathbb{L}}^{2}$-risk on specific Sobolev-Laguerre spaces. Lower bounds matching with the upper bounds within a logarithmic factor are proved. The method is illustrated on simulated data.

Many methods have been proposed for community detection in networks, but most of them do not take into account additional information on the nodes that is often available in practice. In this paper, we propose a new joint community detection criterion that uses both the network edge information and the node features to detect community structures. One advantage our method has over existing joint detection approaches is the flexibility of learning the impact of different features which may differ across communities. Another advantage is the flexibility of choosing the amount of influence the feature information has on communities. We show the method performs well on simulated and real networks.

We consider linear function-on-function regression models with multiple predictive curves. We first apply the wavelet transformation to the predictive curves and transform the original model to a linear model with functional response and high dimensional multivariate predictors. Based on the best finite dimensional approximation to the signal part in the response curve, we find an expansion of the vector of coefficient functions, which enjoys a good predictive property. To estimate this expansion, we propose a penalized generalized eigenvalue problem followed by a penalized least squares problem. We establish the sparse oracle inequalities for our estimates in the high-dimensional settings. The choices of tuning parameters and the number of components are provided. Simulations studies and application to real datasets demonstrate that our method has good predictive performance and is efficient in dimension reduction.

In this paper we study posterior asymptotics for conditional density estimation in the supremum $L_{1}$ norm. Compared to the expected $L_{1}$ norm, the supremum $L_{1}$ norm allows accurate prediction at any designated conditional density. We model the conditional density as a regression tree by defining a data dependent sequence of increasingly finer partitions of the predictor space and by specifying the conditional density to be the same across all predictor values in a partition set. Each conditional density is modeled independently so that the prior specifies a type of dependence between conditional densities which disappears after a certain number of observations have been observed. The rate at which the number of partition sets increases with the sample size determines when the dependence between pairs of conditional densities is set to zero and, ultimately, drives posterior convergence at the true data distribution.

We consider independent component analysis (ICA) using a Bayesian approach. The latent sources are allowed to be block-wise independent while the underlying block structure is unknown. We consider prior distributions on the block structure, the mixing matrix and the marginal density functions of latent sources using a Dirichlet mixture and random series priors. We obtain a minimax-optimal posterior contraction rate of the joint density of the latent sources. This finding reveals that Bayesian ICA adaptively achieves the optimal rate of convergence according to the unknown smoothness level of the true marginal density functions and the unknown block structure. We evaluate the empirical performance of the proposed method by simulation studies.

We propose a Bayesian nonparametric approach to modelling and predicting a class of functional time series with application to energy markets, based on fully observed, noise-free functional data. Traders in such contexts conceive profitable strategies if they can anticipate the impact of their bidding actions on the aggregate demand and supply curves, which in turn need to be predicted reliably. Here we propose a simple Bayesian nonparametric method for predicting such curves, which take the form of monotonic bounded step functions. We borrow ideas from population genetics by defining a class of interacting particle systems to model the functional trajectory, and develop an implementation strategy which uses ideas from Markov chain Monte Carlo and approximate Bayesian computation techniques and allows to circumvent the intractability of the likelihood. Our approach shows great adaptation to the degree of smoothness of the curves and the volatility of the functional series, proves to be robust to an increase of the forecast horizon and yields an uncertainty quantification for the functional forecasts. We illustrate the model and discuss its performance with simulated datasets and on real data relative to the Italian natural gas market.

Lower Urinary Tract Symptoms (LUTS) affect a significant proportion of the population and often lead to a reduced quality of life. LUTS overlap across a wide variety of diseases, which makes the diagnostic process extremely complicated. In this work we focus on the relation between LUTS and Urinary Tract Infection (UTI). The latter is detected through the number of White Blood Cells (WBC) in a sample of urine: WBC$\geq1$ indicates UTI and high levels may indicate complications. The objective of this work is to provide the clinicians with a tool for supporting the diagnostic process, deepening the available knowledge about LUTS and UTI. We analyze data recording both LUTS profile and WBC count for each patient. We propose to model the WBC using a random partition model in which we specify a prior distribution over the partition of the patients which includes the clustering information contained in the LUTS profile. Then, within each cluster, the WBC counts are assumed to be generated by a zero-inflated Poisson distribution. The results of the predictive distribution allows to identify the symptoms configuration most associated with the presence of UTI as well as with severe infections.

A simple approach for modeling multivariate extremes is to consider the vector of component-wise maxima and their max-stable distributions. The extremal dependence can be inferred by estimating the angular measure or, alternatively, the Pickands dependence function. We propose a nonparametric Bayesian model that allows, in the bivariate case, the simultaneous estimation of both functional representations through the use of polynomials in the Bernstein form. The constraints required to provide a valid extremal dependence are addressed in a straightforward manner, by placing a prior on the coefficients of the Bernstein polynomials which gives probability one to the set of valid functions. The prior is extended to the polynomial degree, making our approach nonparametric. Although the analytical expression of the posterior is unknown, inference is possible via a trans-dimensional MCMC scheme. We show the efficiency of the proposed methodology by means of a simulation study. The extremal behaviour of log-returns of daily exchange rates between the Pound Sterling vs the U.S. Dollar and the Pound Sterling vs the Japanese Yen is analysed for illustrative purposes.

In this article we propose novel Bayesian nonparametric methods using Dirichlet Process Mixture (DPM) models for detecting pairwise dependence between random variables while accounting for uncertainty in the form of the underlying distributions. A key criteria is that the procedures should scale to large data sets. In this regard we find that the formal calculation of the Bayes factor for a dependent-vs.-independent DPM joint probability measure is not feasible computationally. To address this we present Bayesian diagnostic measures for characterising evidence against a “null model” of pairwise independence. In simulation studies, as well as for a real data analysis, we show that our approach provides a useful tool for the exploratory nonparametric Bayesian analysis of large multivariate data sets.

Dirichlet-process mixture models, favored for their large support and for the relative ease of their implementation, are popular choices for Bayesian density estimation. However, despite the models’ flexibility, the performance of density estimates suffers in certain situations, in particular when the true distribution is skewed or heavy tailed. We detail a method that improves performance in a variety of settings by initially transforming the sample, choosing the transformation to facilitate estimation of the density on the new scale. The effectiveness of the method is demonstrated under a variety of simulated scenarios, and in an application to body mass index (BMI) observations from a large survey of Ohio adults.

This paper proposes a nonparametric Bayesian framework called VariScan for simultaneous clustering, variable selection, and prediction in high-throughput regression settings. Poisson-Dirichlet processes are utilized to detect lower-dimensional latent clusters of covariates. An adaptive nonlinear prediction model is constructed for the response, achieving a balance between model parsimony and flexibility. Contrary to conventional belief, cluster detection is shown to be a posteriori consistent for a general class of models as the number of covariates and subjects grows. Simulation studies and data analyses demonstrate that VariScan often outperforms several well-known statistical methods.

Genetic sequence data are well described by hidden Markov models (HMMs) in which latent states correspond to clusters of similar mutation patterns. Theory from statistical genetics suggests that these HMMs are nonhomogeneous (their transition probabilities vary along the chromosome) and have large support for self transitions. We develop a new nonparametric model of genetic sequence data, based on the hierarchical Dirichlet process, which supports these self transitions and nonhomogeneity. Our model provides a parameterization of the genetic process that is more parsimonious than other more general nonparametric models which have previously been applied to population genetics. We provide truncation-free MCMC inference for our model using a new auxiliary sampling scheme for Bayesian nonparametric HMMs. In a series of experiments on male X chromosome data from the Thousand Genomes Project and also on data simulated from a population bottleneck we show the benefits of our model over the popular finite model fastPHASE, which can itself be seen as a parametric truncation of our model. We find that the number of HMM states found by our model is correlated with the time to the most recent common ancestor in population bottlenecks. This work demonstrates the flexibility of Bayesian nonparametrics applied to large and complex genetic data.

We extend classic characterisations of posterior distributions under Dirichlet process and gamma random measures priors to a dynamic framework. We consider the problem of learning, from indirect observations, two families of time-dependent processes of interest in Bayesian nonparametrics: the first is a dependent Dirichlet process driven by a Fleming–Viot model, and the data are random samples from the process state at discrete times; the second is a collection of dependent gamma random measures driven by a Dawson–Watanabe model, and the data are collected according to a Poisson point process with intensity given by the process state at discrete times. Both driving processes are diffusions taking values in the space of discrete measures whose support varies with time, and are stationary and reversible with respect to Dirichlet and gamma priors respectively. A common methodology is developed to obtain in closed form the time-marginal posteriors given past and present data. These are shown to belong to classes of finite mixtures of Dirichlet processes and gamma random measures for the two models respectively, yielding conjugacy of these classes to the type of data we consider. We provide explicit results on the parameters of the mixture components and on the mixing weights, which are time-varying and drive the mixtures towards the respective priors in absence of further data. Explicit algorithms are provided to recursively compute the parameters of the mixtures. Our results are based on the projective properties of the signals and on certain duality properties of their projections.

Directed graphs occur throughout statistical modeling of networks, and exchangeability is a natural assumption when the ordering of vertices does not matter. There is a deep structural theory for exchangeable undirected graphs, which extends to the directed case via measurable objects known as digraphons. Using digraphons, we first show how to construct models for exchangeable directed graphs, including special cases such as tournaments, linear orderings, directed acyclic graphs, and partial orderings. We then show how to construct priors on digraphons via the infinite relational digraphon model (di-IRM), a new Bayesian nonparametric block model for exchangeable directed graphs, and demonstrate inference on synthetic data.

This paper adopts a Bayesian nonparametric mixture model where the mixing distribution belongs to the wide class of normalized homogeneous completely random measures. We propose a truncation method for the mixing distribution by discarding the weights of the unnormalized measure smaller than a threshold. We prove convergence in law of our approximation, provide some theoretical properties, and characterize its posterior distribution so that a blocked Gibbs sampler is devised.

The versatility of the approximation is illustrated by two different applications. In the first the normalized Bessel random measure, encompassing the Dirichlet process, is introduced; goodness of fit indexes show its good performances as mixing measure for density estimation. The second describes how to incorporate covariates in the support of the normalized measure, leading to a linear dependent model for regression and clustering.

The Dirichlet process mixture model (DPMM) is a ubiquitous, flexible Bayesian nonparametric statistical model. However, full probabilistic inference in this model is analytically intractable, so that computationally intensive techniques such as Gibbs sampling are required. As a result, DPMM-based methods, which have considerable potential, are restricted to applications in which computational resources and time for inference is plentiful. For example, they would not be practical for digital signal processing on embedded hardware, where computational resources are at a serious premium. Here, we develop a simplified yet statistically rigorous approximate maximum a-posteriori (MAP) inference algorithm for DPMMs. This algorithm is as simple as DP-means clustering, solves the MAP problem as well as Gibbs sampling, while requiring only a fraction of the computational effort. (For freely available code that implements the MAP-DP algorithm for Gaussian mixtures see http://www.maxlittle.net/.) Unlike related small variance asymptotics (SVA), our method is non-degenerate and so inherits the “rich get richer” property of the Dirichlet process. It also retains a non-degenerate closed-form likelihood which enables out-of-sample calculations and the use of standard tools such as cross-validation. We illustrate the benefits of our algorithm on a range of examples and contrast it to variational, SVA and sampling approaches from both a computational complexity perspective as well as in terms of clustering performance. We demonstrate the wide applicabiity of our approach by presenting an approximate MAP inference method for the infinite hidden Markov model whose performance contrasts favorably with a recently proposed hybrid SVA approach. Similarly, we show how our algorithm can applied to a semiparametric mixed-effects regression model where the random effects distribution is modelled using an infinite mixture model, as used in longitudinal progression modelling in population health science. Finally, we propose directions for future research on approximate MAP inference in Bayesian nonparametrics.

In the context of regressing a response $Y$ on a predictor $X$, we consider estimating the local modes of the distribution of $Y$ given $X=x$ when $X$ is prone to measurement error. We propose two nonparametric estimation methods, with one based on estimating the joint density of $(X,Y)$ in the presence of measurement error, and the other built upon estimating the conditional density of $Y$ given $X=x$ using error-prone data. We study the asymptotic properties of each proposed mode estimator, and provide implementation details including the mean-shift algorithm for mode seeking and bandwidth selection. Numerical studies are presented to compare the proposed methods with an existing mode estimation method developed for error-free data naively applied to error-prone data.

In this article, we consider an imputation method to handle missing response values based on semiparametric quantile regression estimation. In the proposed method, the missing response values are generated using the semiparametrically estimated conditional quantile regression function at given values of covariates. Then the imputed values are used to estimate a parameter defined as the expected value of a function involving the response and covariate variables. We derive the asymptotic distribution of our estimator constructed with the imputed data and provide a variance estimator. In simulation, we compare our semiparametric quantile regression imputation method to fully parametric and nonparametric alternatives and evaluate the variance estimator based on the asymptotic distribution. We also discuss an extension for estimating a parameter defined through an estimation equation.

A piecewise-deterministic Markov process is a stochastic process whose behavior is governed by an ordinary differential equation punctuated by random jumps occurring at random times. We focus on the nonparametric estimation problem of the jump rate for such a stochastic model observed within a long time interval under an ergodicity condition. We introduce an uncountable class (indexed by the deterministic flow) of recursive kernel estimates of the jump rate and we establish their strong pointwise consistency as well as their asymptotic normality. We propose to choose among this class the estimator with the minimal variance, which is unfortunately unknown and thus remains to be estimated. We also discuss the choice of the bandwidth parameters by cross-validation methods.

We address the task of estimating multiple trajectories from unlabeled data. This problem arises in many settings, one could think of the construction of maps of transport networks from passive observation of travellers, or the reconstruction of the behaviour of uncooperative vehicles from external observations, for example. There are two coupled problems. The first is a data association problem: how to map data points onto individual trajectories. The second is, given a solution to the data association problem, to estimate those trajectories. We construct estimators as a solution to a regularized variational problem (to which approximate solutions can be obtained via the simple, efficient and widespread $k$-means method) and show that, as the number of data points, $n$, increases, these estimators exhibit stable behaviour. More precisely, we show that they converge in an appropriate Sobolev space in probability and with rate $n^{-1/2}$.

The paper studies infinite order autoregressive models for both temporal and spatial processes. We present sufficient conditions for the existence of stationary distributions. To understand the underlying dynamics and to capture the dependence structure, we introduce functional dependence measures and relate them with Lipschitz coefficients of the data-generating mechanisms. Our stability result allows both short- and long-range dependence. With functional dependence measures, we can establish an asymptotic theory for the underlying processes.

Today’s data pose unprecedented challenges to statisticians. It may be incomplete, corrupted or exposed to some unknown source of contamination. We need new methods and theories to grapple with these challenges. Robust estimation is one of the revived fields with potential to accommodate such complexity and glean useful information from modern datasets. Following our recent work on high dimensional robust covariance matrix estimation, we establish a general decision theory for robust statistics under Huber’s $\epsilon$-contamination model. We propose a solution using Scheffé estimate to a robust two-point testing problem that leads to the construction of robust estimators adaptive to the proportion of contamination. Applying the general theory, we construct robust estimators for nonparametric density estimation, sparse linear regression and low-rank trace regression. We show that these new estimators achieve the minimax rate with optimal dependence on the contamination proportion. This testing procedure, Scheffé estimate, also enjoys an optimal rate in the exponent of the testing error, which may be of independent interest.

Hierarchically associated covariates are common in many fields, and it is often of interest to incorporate their information in statistical inference. This paper proposes a novel way to explicitly integrate the information of a given hierarchical tree of covariates in high-dimensional model selection. Specifically, a set of hierarchical scores is introduced to quantify the hierarchical positions of the terminal nodes of the given hierarchical tree, where a terminal node represents either a single covariate or a group of covariates. These scores are then used to weight the corresponding penalty terms in a model selection approach. We show that the proposed estimation approach has a hierarchical grouping property, namely, two highly correlated covariates that are close to each other in the hierarchical tree will be more likely included or excluded together in the model than those which are far away. We also prove model selection consistency of the proposed estimator both between and within groups. The theoretical results are illustrated by simulation and also a real data analysis on the Systemic Lupus Erythematosus (SLE) dataset.

In recent years there has been an increased interest in statistical analysis of data with multiple types of relations among a set of entities. Such multi-relational data can be represented as multi-layer graphs where the set of vertices represents the entities and multiple types of edges represent the different relations among them. For community detection in multi-layer graphs, we consider two random graph models, the multi-layer stochastic blockmodel (MLSBM) and a model with a restricted parameter space, the restricted multi-layer stochastic blockmodel (RMLSBM). We derive consistency results for community assignments from the maximum likelihood estimators (MLEs) in both models where MLSBM is assumed to be the true model, and either the number of nodes or the number of types of edges or both grow. We compare MLEs in the two models with other baseline approaches, such as separate modeling of layers, aggregating the layers and majority voting. In simulations RMLSBM is shown to have advantage over MLSBM when either the growth rate of the number of communities is high or the growth rate of the average degree of the component graphs in the multi-graph is low. We also derive minimax rates of error and thresholds for achieving consistency of community detection in both models, which are then used to compare the multi-layer models with a baseline model, the aggregate stochastic block model. The simulation studies and real data applications confirm the superior performance of the multi-layer approaches in comparison to the baseline procedures.

Tempered distributions have received considerable attention, both from a theoretical point of view and in several important application fields. The most popular choice is perhaps the Tweedie model, which is obtained by tempering the Positive Stable distribution. Through tempering, we suggest a very flexible four-parameter family of distributions that generalizes the Tweedie model and that could be applied to data sets of non-negative observations with complex (and difficult to accommodate) features. We derive the main theoretical properties of our proposal, through which we show its wide application potential. We also embed our proposal within the theory of Lévy processes, thus providing a strengthened probabilistic motivation for its introduction. Furthermore, we derive a series expansion for the probability density function which allows us to develop algorithms for fitting the distribution to data. We finally provide applications to challenging real-world examples taken from international trade.

Understanding efficiency in high dimensional linear models is a longstanding problem of interest. Classical work with smaller dimensional problems dating back to Huber and Bickel has illustrated the clear benefits of efficient loss functions. When the number of parameters $p$ is of the same order as the sample size $n$, $p\approx n$, an efficiency pattern different from the one of Huber was recently established. In this work, we study relative efficiency of sparsity linear models with $p\gg n$. In the interest of deriving the asymptotic mean squared error for $l_{1}$ regularized M-estimators, we propose a novel, robust and sparse approximate message passing algorithm (RAMP), that is adaptive to the error distribution. Our algorithm includes many non-quadratic and non-differentiable loss functions. We derive its asymptotic mean squared error and show its convergence, while allowing $p,n,s\to \infty$, with $n/p\in (0,1)$ and $n/s\in (1,\infty)$. We identify new patterns of relative efficiency regarding $l_{1}$ penalized $M$ estimators. We show that the classical information bound is no longer reachable, even for light–tailed error distributions. Moreover, we show new breakdown points regarding the asymptotic mean squared error. The asymptotic mean squared error of the $l_{1}$ penalized least absolute deviation estimator (P-LAD) breaks down at a critical ratio of the number of observations per number of sparse parameters in the case of light-tailed distributions; whereas, in the case of heavy-tailed distributions, the asymptotic mean squared error breaks down at a critical ratio of the optimal tuning parameter of P-LAD to the optimal tuning parameter of the $l_{1}$ penalized least square estimator.

The subject of this paper is the detection of a change in the marginal distribution of a stationary linear process. By considering the marginal distribution, the change-point model can simultaneously incorporate any change in the coefficients and/or the innovations of the linear process. Furthermore, the change point can be random and data dependent. The key is an analysis of the asymptotic behaviour of the sequential empirical process, both with and without a change point. Our results hold under very mild conditions on the existence of any moment of the innovations and a corresponding condition of summability of the coefficients.

Statistical applications often involve the calculation of intractable multidimensional integrals. The Laplace formula is widely used to approximate such integrals. However, in high-dimensional or small sample size problems, the shape of the integrand function may be far from that of the Gaussian density, and thus the standard Laplace approximation can be inaccurate. We propose an improved Laplace approximation that reduces the asymptotic error of the standard Laplace formula by one order of magnitude, thus leading to third-order accuracy. We also show, by means of practical examples of various complexity, that the proposed method is extremely accurate, even in high dimensions, improving over the standard Laplace formula. Such examples also demonstrate that the accuracy of the proposed method is comparable with that of other existing methods, which are computationally more demanding. An R implementation of the improved Laplace approximation is also provided through the R package iLaplace available on CRAN.