Precision matrix estimation in a multivariate Gaussian model is fundamental to network estimation. Although there exist both Bayesian and frequentist approaches to this, it is difficult to obtain good Bayesian and frequentist properties under the same prior–penalty dual. To bridge this gap, our contribution is a novel prior–penalty dual that closely approximates the graphical horseshoe prior and penalty, and performs well in both Bayesian and frequentist senses. A chief difficulty with the graphical horseshoe prior is a lack of closed form expression of the density function, which we overcome in this article. In terms of theory, we establish posterior convergence rate of the precision matrix that matches the convergence rate of the frequentist graphical lasso estimator, in addition to the frequentist consistency of the MAP estimator at the same rate. In addition, our results also provide theoretical justifications for previously developed approaches that have been unexplored so far, e.g. for the graphical horseshoe prior. Computationally efficient EM and MCMC algorithms are developed respectively for the penalized likelihood and fully Bayesian estimation problems. In numerical experiments, the horseshoe-based approaches echo their superior theoretical properties by comprehensively outperforming the competing methods. A protein–protein interaction network estimation in B-cell lymphoma is considered to validate the proposed methodology.

Network data have become increasingly prevalent in recent research. For instance, network models have been proposed to improve the quality of estimation and hypothesis testing when comparing the effects of different types of treatment. In this paper, we focus on efficiently estimating the average treatment effect using an adaptive randomization procedure in networks. We are working on models of causal frameworks in which the outcome of a subject’s treatment is influenced not only by their own characteristics but also by the characteristics of their neighbors. The experiment’s designer does not observe these characteristics. We also consider situations where only the existing subjects’ network is revealed when assigning treatments to current subjects. Balancing the connections in the network among different treatment groups can lessen the impact of unknown characteristics. In network data, it is often difficult to obtain theoretical properties because the numbers of nodes and connections increase simultaneously. Under mild assumptions, our proposed procedure is closely related to a time-varying inhomogeneous Markov chain. We utilize the technique of Lyapunov functions to control the moments of the imbalance measure sequence in our proposed approach. As a result, our theoretical analysis demonstrates that our procedure is capable of decreasing the variability of the observed outcome in the model. The advantages of the proposed procedures are also demonstrated by extensive simulations and experiments on real network data.

We propose a novel class of partially linear transformed mean residual life (TMRL) models to investigate linear and nonlinear covariate effects on survival outcomes of interest. A martingale-based estimating equation approach with global and kernel-weighted local estimating equations is developed to estimate the parametric and nonparametric components. Unlike the existing inverse probability of censoring weighting estimating equation approach on TMRL models, the newly proposed method avoids estimating or modeling the distribution of the censoring time, thereby enhancing model capability and computational efficiency. Furthermore, we establish the asymptotic properties for the estimators of parametric and nonparametric components and develop an efficient iterative algorithm to implement the proposed procedure. Simulation studies demonstrate the satisfactory finite sample performance of the proposed method. Finally, our model is applied to the studies of lung cancer and type 2 diabetic complications.

Let X be a p-variate random vector and $\tilde{X}$ a knockoff copy of X (in the sense of [9]). A new approach for constructing $\tilde{X}$ (henceforth, NA) has been introduced in [8]. NA has essentially three advantages: (i) To build $\tilde{X}$ is straightforward; (ii) The joint distribution of $(X,\tilde{X})$ can be written in closed form; (iii) $\tilde{X}$ is often optimal under various criteria. However, for NA to apply, $X_1,\dots ,X_p$ should be conditionally independent given some random element Z. Our first result is that any probability measure μ on ${\mathbb{R}}^p$ can be approximated by a probability measure ${\mathrm{\mu }}_0$ of the form

$${\mathrm{\mu }}_0(A_1\times \cdots \times A_p)=E\{\prod _{i=1}^{p}P(X_i\in A_i\mid Z)\}.$$

The approximation is in total variation distance when μ is absolutely continuous, and an explicit formula for ${\mathrm{\mu }}_0$ is provided. If $X\sim {\mathrm{\mu }}_0$, then $X_1,\dots ,X_p$ are conditionally independent. Hence, with a negligible error, one can assume $X\sim {\mathrm{\mu }}_0$ and build $\tilde{X}$ through NA. Our second result is a characterization of the knockoffs $\tilde{X}$ obtained via NA. It is shown that $\tilde{X}$ is of this type if and only if the pair $(X,\tilde{X})$ can be extended to an infinite sequence so as to satisfy certain invariance conditions. The basic tool for proving this fact is de Finetti’s theorem for partially exchangeable sequences. In addition to the quoted results, an explicit formula for the conditional distribution of $\tilde{X}$ given X is obtained in a few cases. In one of such cases, it is assumed $X_i\in \{0,1\}$ for all i.

We consider the problem of two-sample testing for data generated under the manifold setting, namely where potentially high-dimensional observations are made for underlying objects concentrated near a low-dimensional manifold. Existing two-sample tests typically suffer from a loss of power under high-dimensionality; under the manifold setting, these tests largely ignore the underlying geometric structure of the data, resulting in misleading representations of similarity. Instead, we avoid these issues and propose a non-parametric two-sample test for general data objects which takes into account the intrinsic geometry of the data. A data-driven metric is utilized to characterize the distance between points while respecting the manifold structure. The test statistic behaves like a distance metric between distributions and is shown to be consistent against all alternatives where the two distributions have a positive energy distance. Empirical studies and data analysis of speech recordings demonstrate the test’s superior performance for manifold data.

Objects arrive at a system at times $U_1,U_2,\dots$ and leave at times $U_1+X_1,U_2+X_2,\dots$, where we assume that the arrivals are independent and uniformly distributed on the unit interval, that the delay times are independent with distribution function G, and that arrival and delay times are independent. Let ${\Pi }_n$ be the random permutation that connects the ranks of the first n arrivals and departures. We investigate the use of ${\Pi }_n$ for estimating G. We consider empirical copulas in the nonparametric and pattern frequencies in the parametric situation.

Strong invariance principles describe the error term of a Brownian approximation to the partial sums of a stochastic process. While these strong approximation results have many applications, results for continuous-time settings have been limited. In this paper, we obtain strong invariance principles for a broad class of ergodic Markov processes. Strong invariance principles provide a unified framework for analysing commonly used estimators of the asymptotic variance in settings with a dependence structure. We demonstrate how this can be used to analyse the batch means method for simulation output of Piecewise Deterministic Monte Carlo samplers. We also derive a fluctuation result for additive functionals of ergodic diffusions using our strong approximation results.

The Lorenz regression estimates the explained Gini coefficient, a quantity with a natural application in the measurement of inequality of opportunity. Assuming a single-index model, it corresponds to the Gini coefficient of the conditional expectation of a response given some covariates and it can be estimated without having to estimate the link function. However, it is prone to overestimation when many covariates are included. In this paper, we propose a penalised bootstrap procedure which selects the relevant covariates and produces valid inference for the explained Gini coefficient. The obtained estimator achieves the Oracle property. Numerically, it is computed by the SCAD-FABS algorithm, an adaptation of the FABS algorithm to the SCAD penalty. The performance of the procedure is ensured by theoretical guarantees and assessed via Monte-Carlo simulations. Finally, a real data example is presented.

We consider a class of angular Gaussian distributions that allows different degrees of isotropy for directional random variables of arbitrary dimension. To incorporate constraints imposed on the original model parameters, we propose a new parameterization of the distribution so that all new model parameters are free of constraints. Via the new parameterization, we translate the original problem of maximum likelihood estimation subject to complex constraints to a routine optimization problem free of constraints, which in turn leads to theoretically sound and numerically stable procedures for drawing likelihood-based inference. Byproducts from the likelihood-based inference are used to develop graphical and numerical diagnostic tools for assessing goodness of fit of this distribution in a data application. Simulation study and application to data from a hydrogeology study are used to demonstrate implementation and performance of the inference procedures and diagnostics methods.

A very popular class of models for networks posits that each node is represented by a point in a continuous latent space, and that the probability of an edge between nodes is a decreasing function of the distance between them in this latent space. We study the embedding problem for these models, of recovering the latent positions from the observed graph. Assuming certain natural symmetry and smoothness properties, we establish the uniform convergence of the log-likelihood of latent positions as the number of nodes grows. A consequence is that the maximum likelihood embedding converges on the true positions in a certain information-theoretic sense. Extensions of these results, to recovering distributions in the latent space, and so distributions over arbitrarily large graphs, will be treated in the sequel.

We study a minimax risk of estimating inverse functions on a plane, while keeping an estimator is also invertible. Learning invertibility from data and exploiting an invertible estimator are used in many domains, such as statistics, econometrics, and machine learning. Although the consistency and universality of invertible estimators have been well investigated, analysis of the efficiency of these methods is still under development. In this study, we study a minimax risk for estimating invertible bi-Lipschitz functions on a square in a 2-dimensional plane. We first introduce two types of $L^2$-risks to evaluate an estimator which preserves invertibility. Then, we derive lower and upper rates for minimax values for the risks associated with inverse functions. For the derivation, we exploit a representation of invertible functions using level-sets. Specifically, to obtain the upper rate, we develop an estimator asymptotically almost everywhere invertible, whose risk attains the derived minimax lower rate up to logarithmic factors. The derived minimax rate corresponds to that of the non-invertible bi-Lipschitz function, which shows that the invertibility does not reduce the complexity of the estimation problem in terms of the rate.

The logistic regression model is a simple and classic approach to binary classification, where in sparse high-dimensional settings, one believes that only a small proportion of the predictive variables are relevant to the response variable with nonnull regression coefficients. We focus on regularized logistic regression models and the analysis is valid for a large group of regularizers, including folded-concave regularizers such as MCP and SCAD. For finite samples, the discrepancy between the estimated and true nonnull coefficients is evaluated by the false discovery and true positive rates. We show that the false discovery rate can be described using a nonlinear tradeoff function of power asymptotically using a system of equations with six parameters. The analysis is conducted in an “average-over-components” fashion for the unknown parameter and follows the conventional assumptions of the literature in the relevant field. More specifically, we assume a linear growth rate $n∕p\to \mathit{\delta }>0$ covering not only the typical high dimensional settings where $p\ge n$ but also for $n>p$. Further, we propose two applications of this tradeoff function that improve the reproducibility of variable selection: (1) a sample size calculation procedure to achieve a certain power under a prespecified level of false discovery rate using the tradeoff; (2) calibration of the false discovery rate for variable selection taking power into consideration. A similar asymptotic analysis for the model-X knockoff, which provides a selection with a controlled false discovery rate, is investigated to show how to compare two selection methods by comparing the tradeoff curves. We illustrate the tradeoff analysis and its corresponding applications using simulated and real data.

In this paper, we propose a general framework for stochastic online convex optimization that allows for achieving fast-rate stochastic regret bounds. Specifically, we demonstrate that certain algorithms, including online Newton steps and a scale-free variant of Bernstein online aggregation, achieve the best-known rates in unbounded stochastic settings. To illustrate the usefulness of our approach, we apply it to calibrating parametric probabilistic forecasters of non-stationary sub-Gaussian time series. Importantly, our fast-rate stochastic regret bounds are valid at any time, providing a flexible and robust performance metric for sequential algorithms. Our proofs rely on combining self-bounded and Poissonian inequalities for martingales and sub-Gaussian random variables, respectively, under a stochastic exp-concavity assumption.

The Mann–Whitney-type stochastic shift $\mathbb{P}(Y>X)$ has long been used as a scale-free alternative to the mean difference in measuring the distance between two populations. It has recently been recast as a causal estimand, but only in standard settings where confounders are fully captured. We study the Mann–Whitney treatment effect (MWTE) in randomized trials with non-ignorable non-compliance, where the treatment received is confounded by unknown factors. First, we define and estimate a local MWTE on the compliers via the standard principal-stratification approach with randomization status as an instrumental variable (IV). Then, we derive sensitivity bounds for the local effect estimand when key IV assumptions such as exclusion restriction and monotonicity are violated. Finally, we study the asymptotic operating characteristics of the local MWTE estimator in testing the treatment effect. Analytic bounds on the asymptotic relative efficiencies show that this IV-based test is likely superior to standard intent-to-treat tests under location-shift alternatives. The proposed methodology is applied to the famous National Job Training Partnership Act Study as an illustration.

The second-order, small-scale dependence structure of a stochastic process defined in the space-time domain is key to prediction (or kriging). While great efforts have been dedicated to developing models for cases in which the spatial domain is either a finite-dimensional Euclidean space or a sphere, counterpart developments on a generalized linear network are practically non-existent. To fill this gap, we develop a broad range of parametric, non-separable space-time covariance models on generalized linear networks. For the important subgroup of Euclidean trees, we develop models by the space embedding technique, in concert with the generalized Gneiting class of models and 1-symmetric characteristic functions, and by the convex cone and scale mixture approaches. We give examples from each class of models and investigate the geometric features of these covariance functions near the origin and at infinity. We also reveal connections between different classes of space-time covariance models on Euclidean trees. We conclude the paper by investigating the performance of maximum likelihood estimators of certain proposed models in a simulation study.

We develop a statistical inference method for an optimal transport map between distributions on real numbers with uniform confidence bands. The concept of optimal transport (OT) is used to measure distances between distributions, and OT maps are used to construct the distance. OT has been applied in many fields in recent years, and its statistical properties have attracted much interest. In particular, since the OT map is a function, a uniform norm-based statistical inference is significant for visualization and interpretation. In this study, we derive a limit distribution of a uniform norm of an estimation error for the OT map, and then develop a uniform confidence band based on it. In addition to our limit theorem, we develop a bootstrap method with kernel smoothing, then also derive its validation and guarantee on an asymptotic coverage probability of the confidence band. Our proof is based on the functional delta method and the representation of OT maps on the reals.

Recent results in nonparametric regression have shown that neural network regression estimates with many hidden layers are able to achieve good rates of convergence even in case of high-dimensional predictor variables, provided suitable assumptions on the structure of the regression function are imposed. In those recent results, the estimates were defined by minimizing the empirical $L_2$ risk over a class of neural networks. In practice, however, it is not clear how this can be done exactly. In this article, motivated by some recent approximation results for neural networks, we introduce two new regression estimates defined by neural features where most of the neural network weights are chosen via random initialization and no training, thus sparing the costly data-dependent optimization. For the first estimate, which is defined by these neural features and an extra layer whose weights are set via least squares, we derive rates of convergence results in case the regression function is smooth. We then combine this estimate with the projection pursuit, where we choose the directions randomly, and we show that for sufficiently many repetitions we get a second regression estimate which achieves the one-dimensional rate of convergence (up to some logarithmic factor) in case that the regression function satisfies the assumptions of projection pursuit. Because the neural features are obtained by random initialization but not training of the weights, the two estimators thus defined are easy to implement.

This paper proposes a distributed estimation and inferential framework for sparse multivariate regression and conditional Gaussian graphical models under the unbalanced splitting setting. This type of data splitting arises when the datasets from different sources cannot be aggregated on one single machine or when the available machines are of different powers. In this paper, the number of covariates, responses and machines grow with the sample size, while sparsity is imposed. Debiased estimators of the coefficient matrix and of the precision matrix are proposed on every single machine and theoretical guarantees are provided. Moreover, new aggregated estimators that pool information across the machines using a pseudo log-likelihood function are proposed. It is shown that they enjoy efficiency and asymptotic normality as the number of machines grows with the sample size. The performance of these estimators is investigated via a simulation study and a real data example. It is shown empirically that the performances of these estimators are close to those of the non-distributed estimators which use the entire dataset.

It is well known that while the independence of random variables implies zero correlation, the opposite is not true. Namely, uncorrelated random variables are not necessarily independent. In this note we show that the implication could be reversed if we consider the localised version of the correlation coefficient. More specifically, we show that if random variables are conditionally (locally) uncorrelated for any quantile conditioning sets, then they are independent. For simplicity, we focus on the absolutely continuous case. Also, we illustrate potential usefulness of the stated result using multiple examples.

Streaming data refers to a data collection scheme where observations arrive sequentially and perpetually over time, making it challenging to fit into computer memory for statistical analysis. The ordinary least squares estimate for linear regression is sensitive to heavy-tailed errors and outliers, which are commonly encountered in applications. In this case, the Huber loss function is a useful criterion for robust regression. In this paper, we propose robust regression estimation and variable selection for streaming datasets. Unlike the renewable estimation generalized linear regression for streaming datasets, however, the Huber loss function is only first-order differentiable, which poses challenges to renewable estimation in both computation and theoretical development. To address the challenge, we introduce a new smoothed version of the Huber first derivative, which admits a fast and scalable algorithm to perform optimization for streaming data sets and achieves the best fitting of Huber function among different versions. Theoretically, the proposed statistics are shown to have the same asymptotic properties as the standard version computed on an entire data stream with the data batches pooled into one data set, without additional condition. The proposed methods are illustrated using current data and the summary statistics of historical data. Both simulations and real data analysis are conducted to illustrate the finite sample performance of the proposed methods.

In multiple hypothesis testing, it is well known that adaptive procedures can enhance power via incorporating information about the number of true nulls present. Under independence, we establish that two adaptive false discovery rate (FDR) methods, upon augmenting sign declarations, also offer directional false discovery rate ($\text{FDR}\text{dir}$) control in the strong sense. Such $\text{FDR}\text{dir}$ controlling properties are appealing, because adaptive procedures have the greatest potential to reap substantial gain in power when the underlying parameter configurations contain little to no true nulls, which are precisely settings where the $\text{FDR}\text{dir}$ is an arguably more meaningful error rate to be controlled than the FDR.

Consider an opaque medium that contains 3D particles. All particles are convex bodies of the same shape, but they vary in size. The particles are randomly positioned and oriented within the medium and cannot be observed directly. Taking a planar section of the medium we obtain a sample of observed 2D section profile areas of the intersected particles. In this paper, the distribution of interest is the underlying 3D particle size distribution for which an identifiability result is obtained. Moreover, a nonparametric estimator is proposed for this size distribution. The estimator is proven to be consistent and its performance is assessed in a simulation study.

Linear equations in functional spaces where the solution is not continuous require regularization to estimate the unknown function of interest. In this paper we consider the estimation of an infinite dimensional parameter φ by solving a linear equation $\hat{r}=K\mathit{\phi }+U$, where the random noise U has a variance Σ. Under this set-up, we derive the optimal weighting operator which minimizes the mean integrated square error (MISE). In the finite dimensional case the minimum variance estimator is obtained by weighting the equation by ${\mathrm{\Sigma }}^{-1∕2}$. However in the infinite dimensional case that we consider, regularization introduces a bias to the estimator. We show that in the infinite dimensional case, the optimal estimator in terms of the MISE should involve Σ and the unknown smoothness of φ. We then use this result to propose a new feasible two-step estimator. We illustrate our theoretical findings and the small sample properties of the proposed optimal estimator by means of simulations.

Existing works on variable selection and estimation for high-dimensional nonparametric regression focus primarily on modelling a conditional mean function, on which a restrictive additive structure is commonly imposed. We consider a more general framework which covers different types of regression derived from a broad class of convex loss functions, without assuming additivity of the nonparametric regression function to be estimated. A novel penalised local linear regression procedure is proposed for simultaneous variable selection and estimation under this framework. It performs Bridge-penalised local linear regression and regularised bandwidth estimation in a alternating optimisation scheme. The covariate dimension may exceed any polynomial order, while the number of active variables is allowed to grow slowly with sample size. The procedure is shown to be consistent in variable selection and yield a regression function estimator endowed with an oracle property. Simulation and real data examples are presented to illustrate the performance of the proposed method in mean regression, quantile regression and logistic regression problems.

Data objects taking value in a general metric space have become increasingly common in modern data analysis. In this paper, we study two important statistical inference problems, namely, two-sample testing and change-point detection, for such non-Euclidean data under temporal dependence. Typical examples of non-Euclidean valued time series include yearly mortality distributions, time-varying networks, and covariance matrix time series. To accommodate unknown temporal dependence, we advance the self-normalization (SN) technique [22] to the inference of non-Euclidean time series, which is substantially different from the existing SN-based inference for functional time series that reside in Hilbert space [33]. Theoretically, we propose new regularity conditions that could be easier to check than those in the recent literature, and derive the limiting distributions of the proposed test statistics under both null and local alternatives. For change-point detection problem, we also derive the consistency for the change-point location estimator, and combine our proposed change-point test with wild binary segmentation to perform multiple change-point estimation. Numerical simulations demonstrate the effectiveness and robustness of our proposed tests compared with existing methods in the literature. Finally, we apply our tests to two-sample inference in mortality data and change-point detection in cryptocurrency data.

A smooth test to simultaneously compare K copulas, where $K\ge 2$, is proposed. The K observed populations can be paired. The test statistic is based on the differences between moment sequences, called copula coefficients. These coefficients characterize the copulas, even in cases where the copula densities may not exist. The procedure involves a two-step data-driven procedure. In the initial step, the most significantly different coefficients are selected for all pairs of populations. The subsequent step utilizes these coefficients to identify populations that exhibit significant differences. To illustrate the efficacy of our method, we present numerical studies that demonstrate its performance. Furthermore, we apply our methodology, implemented in the “Kcop” R package, to two real datasets.

This paper considers the problems of detecting a change point and estimating the location in the correlation matrices of a sequence of high-dimensional vectors, where the dimension is large enough to be comparable to the sample size or even much larger. A new break test is proposed based on signflip parallel analysis to detect the existence of change points. Furthermore, a two-step approach combining a signflip permutation dimension reduction step and a CUSUM statistic is proposed to estimate the change point’s location and recover the support of changes. The consistency of the estimator is constructed. Simulation examples and real data applications illustrate the superior empirical performance of the proposed methods. Especially, the proposed methods outperform existing ones for non-Gaussian data and the change point in the extreme tail of a sequence and become more accurate as the dimension p increases. Supplementary materials for this article are available online.

We study limit theorems for entropic optimal transport (EOT) maps, dual potentials, and the Sinkhorn divergence. The key technical tool we use is a first and second-order Hadamard differentiability analysis of EOT potentials with respect to the marginals, which may be of independent interest. Given the differentiability results, the functional delta method is used to obtain central limit theorems for empirical EOT potentials and maps. The second-order functional delta method is leveraged to establish the limit distribution of the empirical Sinkhorn divergence under the null. Building on the latter result, we further derive the null limit distribution of the Sinkhorn independence test statistic and characterize the correct order. Since our limit theorems follow from Hadamard differentiability of the relevant maps, as a byproduct, we also obtain bootstrap consistency and asymptotic efficiency of the empirical EOT map, potentials, and Sinkhorn divergence.

Undirected graphical models are widely used to model the conditional independence structure of vector-valued data. However, in many modern applications, for example those involving EEG and fMRI data, observations are more appropriately modeled as multivariate random functions rather than vectors. Functional graphical models have been proposed to model the conditional independence structure of such functional data. We propose a neighborhood selection approach to estimate the structure of Gaussian functional graphical models, where we first estimate the neighborhood of each node via a function-on-function regression and subsequently recover the entire graph structure by combining the estimated neighborhoods. Our approach only requires assumptions on the conditional distributions of random functions, and we estimate the conditional independence structure directly. We thus circumvent the need for a well-defined precision operator that may not exist when the functions are infinite dimensional. Additionally, the neighborhood selection approach is computationally efficient and can be easily parallelized. The statistical consistency of the proposed method in the high-dimensional setting is supported by both theory and experimental results. In addition, we study the effect of the choice of the function basis used for dimensionality reduction in an intermediate step. We give a heuristic criterion for choosing a function basis and motivate two practically useful choices, which we justify by both theory and experiments.

We prove deviation inequalities for sums of high-dimensional random matrices and operators with dependence and heavy tails. Estimation of high-dimensional matrices is a concern for numerous modern applications. However, most results are stated for independent observations. Therefore, it is critical to derive results for dependent and heavy-tailed matrices. In this paper, we derive a dimension-free upper bound on the deviation of the sums. Thus, the bound does not depend explicitly on the dimension of the matrices but rather on their effective rank. Our result generalizes several existing studies on the deviation of sums of matrices. It relies on two techniques: (i) a variational approximation of the dual of moment generating functions, and (ii) robustification through the truncation of the eigenvalues of the matrices. We reveal that our results are applicable to several problems, such as covariance matrix estimation, hidden Markov models, and overparameterized linear regression.

We propose extrinsic and intrinsic deep neural network architectures as general frameworks for deep learning on manifolds. Specifically, extrinsic deep neural networks (eDNNs) preserve geometric features on manifolds by utilizing an equivariant embedding from the manifold to its image in the Euclidean space. Moreover, intrinsic deep neural networks (iDNNs) incorporate the underlying intrinsic geometry of manifolds via exponential and log maps with respect to a Riemannian structure. Consequently, we prove that the empirical risk of the empirical risk minimizers (ERM) of eDNNs and iDNNs converge in optimal rates. Overall, The eDNNs framework is simple and easy to compute, while the iDNNs framework is accurate and fast converging. To demonstrate the utilities of our framework, various simulation studies, and real data analyses are presented with eDNNs and iDNNs.

We study the problem of merging sequential or independent e-values into one e-value or e-process. We describe a class of e-value merging functions via martingales and show that it dominates all merging methods for sequential e-values. All admissible methods for constructing e-processes can also be obtained in this way. In the case of merging independent e-values, the situation becomes much more complicated, and we provide a general class of such merging functions based on martingales applied to reordered data.

In this paper we propose and study a version of the Dyadic Classification and Regression Trees (DCART) estimator from Donoho (1997) for (fixed design) quantile regression in general dimensions. We refer to this proposed estimator as the QDCART estimator. Just like the mean regression version, we show that a) a fast dynamic programming based algorithm with computational complexity $O(NlogN)$ exists for computing the QDCART estimator and b) an oracle risk bound (trading off squared error and a complexity parameter of the true signal) holds for the QDCART estimator. This oracle risk bound then allows us to demonstrate that the QDCART estimator enjoys adaptively rate optimal estimation guarantees for piecewise constant and bounded variation function classes. In contrast to existing results for the DCART estimator which requires subgaussianity of the error distribution, for our estimation guarantees to hold we do not need any restrictive tail decay assumptions on the error distribution. For instance, our results hold even when the error distribution has no first moment such as the Cauchy distribution. Furthermore, we perform extensive numerical experiments on both simulated and real data which illustrate the usefulness of the proposed methods.

The intrinsically infinite-dimensional features of the functional observations over multidimensional domains render the standard classification methods effectively inapplicable. To address this problem, we introduce a novel multiclass functional deep neural network (mfDNN) classifier as an innovative data mining and classification tool. The architecture incorporates a sparse deep neural network with Rectified Linear Unit (ReLU) activation function, minimizing cross-entropy loss in a multiclass classification framework. This design enables the utilization of modern computational tools. The convergence rates of the misclassification risk functions are also derived for both fully observed and discretely observed multidimensional functional data. The efficacy of mfDNN is demonstrated through simulations and several benchmark datasets from different application domains.

Valid online inference is an important problem in contemporary multiple testing research, to which various solutions have been proposed recently. It is well-known that these existing methods can suffer from a significant loss of power if the null p-values are conservative. In this work, we extend the previously introduced methodology to obtain more powerful procedures for the case of super-uniformly distributed p-values. These types of p-values arise in important settings, e.g. when discrete hypothesis tests are performed. To this end, we introduce the method of super-uniformity reward (SUR) that incorporates information about the individual null cumulative distribution functions. Our approach yields several new ‘rewarded’ procedures that offer uniform power improvements over known procedures and come with mathematical guarantees for controlling online error criteria based either on the family-wise error rate (FWER) or the marginal false discovery rate (mFDR). We illustrate the benefit of super-uniform rewarding in real-data analyses and simulation studies. While discrete tests serve as our leading example, we also show the benefit of our method for online p-values weighting. Finally, we present extensions of our theory to online FDR control and stopped mFDR control.

In this paper, we introduce a functional nonlinear mixed effects modeling framework designed to quantify the random, nonlinear relationship between individual spatiotemporal functional trajectories and longitudinal responses. Our proposed framework accounts for within-individual variability through a spatiotemporal process. We detail an estimation method for determining fixed and random effect functions and spatiotemporal covariance operators and establish their asymptotic properties, including uniform consistency and weak convergence. We also develop global linear hypothesis tests and bootstrap-based simultaneous confidence bands for fixed effect functions. To assess the finite-sample performance of our method, we perform a numerical analysis using both simulated and real-world datasets. Our results demonstrate that the proposed model class is significantly more flexible and effective in detecting functional fixed effects compared to existing nonlinear mixed effects models. We apply our approach to an autism research database to investigate the impact of age and spatial dynamics on fractional anisotropy along the corpus callosum white matter fiber skeleton.

We consider Gaussian random fields on the product of spheres. We study the regularity and the Hölder continuity of such random fields via their covariance function. Moreover, we approximate the Gaussian random fields using truncations of the Karhunen-Loéve expansion and conduct simulation experiments to illustrate our approximation results. Using hourly wind speed and global space-time cloud cover datasets, we discuss modelling data in a Bayesian framework using Gaussian random fields over the product of spheres with covariance approximations through truncated series expansions.

Regression adjustment is broadly applied in randomized trials under the premise that it usually improves the precision of a treatment effect estimator. However, previous work has shown that this is not always true. To further understand this phenomenon, we develop a unified comparison of the asymptotic variance of a class of linear regression-adjusted estimators. Our analysis is based on the classical theory for linear regression with heteroscedastic errors and thus does not assume that the postulated linear model is correct. For a randomized Bernoulli trial, we provide sufficient conditions under which some regression-adjusted estimators are guaranteed to be more asymptotically efficient than others. We comment on the extension of our theory to other settings such as general treatment mechanisms and generalized linear models, and find that the variance dominance phenomenon no longer occurs

Confidence intervals are a fundamental tool for quantifying the uncertainty of parameters of interest. With the increase of data privacy awareness, developing a private version of confidence intervals has gained growing attention from both statisticians and computer scientists. Differential privacy is a state-of-the-art framework for analyzing privacy loss when releasing statistics computed from sensitive data. Recent work has been done around differentially private confidence intervals, yet to the best of our knowledge, rigorous methodologies on differentially private confidence intervals in the context of survey sampling have not been studied. In this paper, we propose three differentially private algorithms for constructing confidence intervals for proportions under stratified random sampling. We articulate two variants of differential privacy that make sense for data from stratified sampling designs, analyzing each of our algorithms within one of these two variants. We establish analytical privacy guarantees and asymptotic properties of the estimators. In addition, we conduct simulation studies to evaluate the proposed private confidence intervals, and two applications to the 1940 Census data are provided.

Multivariate Bayesian error-in-variable (EIV) linear regression is considered to account for additional additive Gaussian error in the features and response. A 3-variable deterministic scan Gibbs sampler is constructed for multivariate EIV regression models using classical and Berkson errors with independent normal and inverse-Wishart priors. These Gibbs samplers are proven to always be geometrically ergodic which ensures a central limit theorem for many time averages from the Markov chains. We demonstrate the strengths and limitations of the Gibbs sampler with simulated data for large data problems, robustness to misspecification and also analyze a real-data example in astrophysics.

Individualized treatment rules (ITRs) have been widely applied in many fields such as precision medicine and personalized marketing. Beyond the extensive studies on ITR for binary or multiple treatments, there is considerable interest in applying combination treatments. This paper introduces a novel ITR estimation method for combination treatments incorporating interaction effects among treatments. Specifically, we propose the generalized ψ-loss as a non-convex surrogate in the residual weighted learning framework, offering desirable statistical and computational properties. Statistically, the minimizer of the proposed surrogate loss is Fisher-consistent with the optimal decision rules, incorporating interaction effects at any intensity level – a significant improvement over existing methods. Computationally, the proposed method applies the difference-of-convex algorithm for efficient computation. Through simulation studies and real-world data applications, we demonstrate the superior performance of the proposed method in recommending combination treatments.

Under model misspecification, it is known that Bayesian posteriors often do not properly quantify uncertainty about true or pseudo-true parameters. Even more fundamentally, misspecification leads to a lack of reproducibility in the sense that the same model will yield contradictory posteriors on independent data sets from the true distribution. To define a criterion for reproducible uncertainty quantification under misspecification, we consider the probability that two credible sets constructed from independent data sets have nonempty overlap, and we establish a lower bound on this overlap probability that holds whenever the credible sets are valid confidence sets. We prove that credible sets from the standard posterior can strongly violate this bound, indicating that it is not internally coherent under misspecification. To improve reproducibility in an easy-to-use and widely applicable way, we propose to apply bagging to the Bayesian posterior (“BayesBag”); that is, to use the average of posterior distributions conditioned on bootstrapped datasets. We motivate BayesBag from first principles based on Jeffrey conditionalization and show that the bagged posterior typically satisfies the overlap lower bound. Further, we prove a Bernstein–Von Mises theorem for the bagged posterior, establishing its asymptotic normal distribution. We demonstrate the benefits of BayesBag via simulation experiments and an application to crime rate prediction.

In a range of genomic applications, it is of interest to quantify the evidence that the signal at site i is active given conditionally independent replicate observations summarized by the sample mean and variance $(\overline{Y},s^2)$ at each site. We study the version of the problem in which the signal distribution is sparse, and the error distribution has an unknown site-specific variance so that the null distribution of the standardized statistic is Student-t rather than Gaussian. The main contribution of this paper is a sparse-mixture approximation to the non-null density of the t-ratio. This formula demonstrates the effect of low degrees of freedom on the Bayes factor, or the conditional probability that the site is active. We illustrate some differences on a HIV dataset for gene-expression data.

This manuscript discusses the regression analysis of a semiparametric proportional mean model for panel count data. A spline-based generalized estimating estimation (GEE) approach is applied to account for the correlation among cumulative counts. To avoid the potential issue of overfitting, a penalization technique is applied to regularize the spline estimation. An easy-to-implement and computationally efficient two-stage iterative algorithm is developed to accomplish the penalized estimation. The proposed methodology does not specify the stochastic model of the underlying counting process and hence provides great flexibility for model fitting. Theoretically, the uniform convergence and the optimal rate of convergence for the functional estimator are established, and the asymptotic normality for regression parameter estimators is shown to be valid even if the working covariance matrix is misspecified. The semiparametric efficiency for regression parameter estimators can be achieved if the working matrix is correctly specified. Further, to address the issue of the underestimation of the variance-covariance matrix of regression parameter estimators for small sample sizes, which is brought up by GEE methodology, we propose a novel approach based on the modified sandwich estimator to compensate for the deficiency in variance-covariance estimation. Numerically, an extensive Monte Carlo study was conducted to evaluate the finite-sample performance of penalized spline estimators and the impact of the selection of the working matrix on the estimation, along with the robustness of the methodology to the underlying counting process. The proposed penalized approach was further applied to analyze data from a non-melanoma skin cancer chemoprevention study.

Analyzing large samples of high-dimensional data under dependence is a challenging statistical problem as long time series may have change points, most importantly in the mean and the marginal covariances, for which one needs valid tests. Inference for large covariance matrices is especially difficult due to noise accumulation, resulting in singular estimates and poor power of related tests. The singularity of the sample covariance matrix in high dimensions can be overcome by considering a linear combination with a regular, more structured target matrix. This approach is known as shrinkage, and the target matrix is typically of diagonal form. In this paper, we consider covariance shrinkage towards structured nonparametric estimators of the bandable or Toeplitz type, respectively, aiming at improved estimation accuracy and statistical power of tests even under nonstationarity. We derive feasible Gaussian approximation results for bilinear projections of the shrinkage estimators which are valid under nonstationarity and dependence. These approximations especially enable us to formulate a statistical test for structural breaks in the marginal covariance structure of high-dimensional time series without restrictions on the dimension, and which is robust against nonstationarity of nuisance parameter. We show via simulations that shrinkage helps to increase the power of the proposed tests. Moreover, we suggest a data-driven choice of the shrinkage weights, and assess its performance by means of a Monte Carlo study. The results indicate that the proposed shrinkage estimator is superior for non-Toeplitz covariance structures close to fractional Gaussian noise.

We show that two popular selective inference procedures, namely data carving (Fithian et al., 2017) and selection with a randomized response (Tian and Taylor, 2018), when combined with the polyhedral method (Lee et al., 2016), result in confidence intervals whose length is bounded. This contrasts results for confidence intervals based on the polyhedral method alone, whose expected length is typically infinite (Kivaranovic and Leeb, 2021). Moreover, we show that these two procedures always dominate corresponding sample-splitting methods in terms of interval length.

Compositional data arise in many areas of research in the natural and biomedical sciences. One prominent example is in the study of the human gut microbiome, where one can measure the relative abundance of many distinct microorganisms in a subject’s gut. Often, practitioners are interested in learning how the dependencies between microbes vary across distinct populations or experimental conditions. In statistical terms, the goal is to estimate a covariance matrix for the (latent) log-abundances of the microbes in each of the populations. However, the compositional nature of the data prevents the use of standard estimators for these covariance matrices. In this article, we propose an estimator of multiple covariance matrices which allows for information sharing across distinct populations of samples. Compared to some existing estimators, which estimate the covariance matrices of interest indirectly, our estimator is direct, ensures positive definiteness, and is the solution to a convex optimization problem. We compute our estimator using a proximal-proximal gradient descent algorithm. Asymptotic properties of our estimator reveal that it can perform well in high-dimensional settings. We show that our method provides more reliable estimates than competitors in an analysis of microbiome data from subjects with myalgic encephalomyelitis/chronic fatigue syndrome and through simulation studies.

Trend filtering is a modern approach to nonparametric regression that is more adaptive to local smoothness than splines or similar basis procedures. Existing analyses of trend filtering focus on estimating a function corrupted by homoskedastic Gaussian noise, but our work extends this technique to general exponential family distributions. This extension is motivated by the need to study massive, gridded climate data derived from polar-orbiting satellites. We present algorithms tailored to large problems, theoretical results for general exponential family likelihoods, and principled methods for tuning parameter selection without excess computation.

We consider the problem of estimation in Hidden Markov models with finite state space and nonparametric emission distributions. Efficient estimators for the transition matrix are exhibited, and a semiparametric Bernstein-von Mises result is deduced. Following from this, we propose a modular approach using the cut posterior to jointly estimate the transition matrix and the emission densities. We first derive a general theorem on contraction rates for this approach. We then show how this result may be applied to obtain a contraction rate result for the emission densities in our setting; a key intermediate step is an inversion inequality relating $L^1$ distance between the marginal densities to $L^1$ distance between the emissions. Finally, a contraction result for the smoothing probabilities is shown, which avoids the common approach of sample splitting. Simulations are provided which demonstrate both the theory and the ease of its implementation.

We consider the problem of comparing several samples of stochastic processes with respect to their second-order structure, and describing the main modes of variation in this second order structure, if present. These tasks can be seen as an Analysis of Variance (ANOVA) and a Principal Component Analysis (PCA) of covariance operators, respectively. They arise naturally in functional data analysis, where several populations are to be contrasted relative to the nature of their dispersion around their means, rather than relative to their means themselves. We contribute a novel approach based on optimal (multi)transport, where each covariance can be identified with a a centred Gaussian process of corresponding covariance. By means of constructing the optimal simultaneous coupling of these Gaussian processes, we contrast the (linear) maps that achieve it with the identity with respect to a norm-induced distance. The resulting test statistic, calibrated by permutation, is seen to distinctly outperform the state-of-the-art, and to furnish considerable power even under local alternatives. This effect is seen to be genuinely functional, and is related to the potential for perfect discrimination in infinite dimensions. In the event of a rejection of the null hypothesis stipulating equality, a geometric interpretation of the transport maps allows us to construct a (tangent space) PCA revealing the main modes of variation. As a necessary step to developing our methodology, we prove results on the existence and boundedness of optimal multitransport maps. These are of independent interest in the theory of transport of Gaussian processes. The transportation ANOVA and PCA are illustrated on a variety of simulated and real examples.

Nonparametric maximum likelihood estimators (MLEs) in inverse problems often have non-normal limit distributions, like Chernoff’s distribution. However, if one considers smooth functionals of the model, with corresponding functionals of the MLE, one gets normal limit distributions and faster rates of convergence. We demonstrate this for a model for the incubation time of a disease. The usual approach in the latter models is to use parametric distributions, like Weibull and gamma distributions, which leads to inconsistent estimators. Smoothed bootstrap methods are discussed for constructing confidence intervals.