Peter Hall made wide-ranging and far-reaching contributions to nonparametric modeling. He was one of the leading figures in the developments of nonparametric techniques with over 300 published papers in the field alone. This article gives a selective overview on the contributions of Peter Hall to nonparametric function estimation and modeling. The focuses are on density estimation, nonparametric regression, bandwidth selection, boundary corrections, inference under shape constraints, estimation of residual variances, analysis of wavelet estimators, multivariate regression and applications of nonparametric methods.

Peter Hall died in Melbourne on January 9, 2016. He was an extremely prolific researcher and contributed to many different areas of statistics. In this paper, I talk about my experience with Peter and I summarise his main contributions to deconvolution, which include measurement error problems and problems in image analysis.

Functional data analysis has become a major branch of nonparametric statistics and is a fast evolving field. Peter Hall has made fundamental contributions to this area and its theoretical underpinnings. He wrote more than 25 papers in functional data analysis between 1998 and 2016 and from 2005 on was a tenured faculty member with a 25% appointment in the Department of Statistics at the University of California, Davis. This article describes aspects of his appointment and academic life in Davis and also some of his major results in functional data analysis, along with a brief history of this area. It concludes with an outlook on new types of functional data and an emerging field of “random objects” that subsumes functional data analysis as it deals with more complex data structures.

In this article, I summarise Peter Hall’s contributions to high-dimensional data, including their geometric representations and variable selection methods based on ranking. I also discuss his work on classification problems, concluding with some personal reflections on my own interactions with him. This article complements [Ann. Statist. 44 (2016) 1821–1836; Ann. Statist. 44 (2016) 1837–1853; Ann. Statist. 44 (2016) 1854–1866 and Ann. Statist. 44 (2016) 1867–1887], which focus on other aspects of Peter’s research.

In recent years, sparse principal component analysis has emerged as an extremely popular dimension reduction technique for high-dimensional data. The theoretical challenge, in the simplest case, is to estimate the leading eigenvector of a population covariance matrix under the assumption that this eigenvector is sparse. An impressive range of estimators have been proposed; some of these are fast to compute, while others are known to achieve the minimax optimal rate over certain Gaussian or sub-Gaussian classes. In this paper, we show that, under a widely-believed assumption from computational complexity theory, there is a fundamental trade-off between statistical and computational performance in this problem. More precisely, working with new, larger classes satisfying a restricted covariance concentration condition, we show that there is an effective sample size regime in which no randomised polynomial time algorithm can achieve the minimax optimal rate. We also study the theoretical performance of a (polynomial time) variant of the well-known semidefinite relaxation estimator, revealing a subtle interplay between statistical and computational efficiency.

Two-sample $U$-statistics are widely used in a broad range of applications, including those in the fields of biostatistics and econometrics. In this paper, we establish sharp Cramér-type moderate deviation theorems for Studentized two-sample $U$-statistics in a general framework, including the two-sample $t$-statistic and Studentized Mann–Whitney test statistic as prototypical examples. In particular, a refined moderate deviation theorem with second-order accuracy is established for the two-sample $t$-statistic. These results extend the applicability of the existing statistical methodologies from the one-sample $t$-statistic to more general nonlinear statistics. Applications to two-sample large-scale multiple testing problems with false discovery rate control and the regularized bootstrap method are also discussed.

In this paper, we study parametric nonlinear regression under the Harris recurrent Markov chain framework. We first consider the nonlinear least squares estimators of the parameters in the homoskedastic case, and establish asymptotic theory for the proposed estimators. Our results show that the convergence rates for the estimators rely not only on the properties of the nonlinear regression function, but also on the number of regenerations for the Harris recurrent Markov chain. Furthermore, we discuss the estimation of the parameter vector in a conditional volatility function, and apply our results to the nonlinear regression with $I(1)$ processes and derive an asymptotic distribution theory which is comparable to that obtained by Park and Phillips [Econometrica 69 (2001) 117–161]. Some numerical studies including simulation and empirical application are provided to examine the finite sample performance of the proposed approaches and results.

In semivarying coefficient modeling of longitudinal/clustered data, of primary interest is usually the parametric component which involves unknown constant coefficients. First, we study semiparametric efficiency bound for estimation of the constant coefficients in a general setup. It can be achieved by spline regression using the true within-subject covariance matrices, which are often unavailable in reality. Thus, we propose an estimator when the covariance matrices are unknown and depend only on the index variable. First, we estimate the covariance matrices using residuals obtained from a preliminary estimation based on working independence and both spline and local linear regression. Then, using the covariance matrix estimates, we employ spline regression again to obtain our final estimator. It achieves the semiparametric efficiency bound under normality assumption and has the smallest asymptotic covariance matrix among a class of estimators even when normality is violated. Our theoretical results hold either when the number of within-subject observations diverges or when it is uniformly bounded. In addition, using the local linear estimator of the nonparametric component is superior to using the spline estimator in terms of numerical performance. The proposed method is compared with the working independence estimator and some existing method via simulations and application to a real data example.

Given $n$ samples $X_{1},X_{2},\ldots,X_{n}$ from $N(0,\Sigma)$, we are interested in estimating the $p\times p$ precision matrix $\Omega=\Sigma^{-1}$; we assume $\Omega$ is sparse in that each row has relatively few nonzeros.

We propose Partial Correlation Screening (PCS) as a new row-by-row approach. To estimate the $i$th row of $\Omega$, $1\leq i\leq p$, PCS uses a Screen step and a Clean step. In the Screen step, PCS recruits a (small) subset of indices using a stage-wise algorithm, where in each stage, the algorithm updates the set of recruited indices by adding the index $j$ that has the largest empirical partial correlation (in magnitude) with $i$, given the set of indices recruited so far. In the Clean step, PCS reinvestigates all recruited indices, removes false positives and uses the resultant set of indices to reconstruct the $i$th row.

PCS is computationally efficient and modest in memory use: to estimate a row of $\Omega$, it only needs a few rows (determined sequentially) of the empirical covariance matrix. PCS is able to execute an estimation of a large $\Omega$ (e.g., $p=10K$) in a few minutes.

Higher Criticism Thresholding (HCT) is a recent classifier that enjoys optimality, but to exploit its full potential, we need a good estimate of $\Omega$. Note that given an estimate of $\Omega$, we can always combine it with HCT to build a classifier (e.g., HCT-PCS, HCT-glasso).

We have applied HCT-PCS to two microarray data sets ($p=8K$ and $10K$) for classification, where it not only significantly outperforms HCT-glasso, but also is competitive to the Support Vector Machine (SVM) and Random Forest (RF). These suggest that PCS gives more useful estimates of $\Omega$ than the glasso; we study this carefully and have gained some interesting insight.

We show that in a broad context, PCS fully recovers the support of $\Omega$ and HCT-PCS is optimal in classification. Our theoretical study sheds interesting light on the behavior of stage-wise procedures.

Dense spatial data are commonplace nowadays, and they provide the impetus for addressing nonstationarity in a general way. This paper extends the notion of intrinsic random function by allowing the stationary component of the covariance to vary with spatial location. A nonparametric estimation procedure based on gridded data is introduced for the case where the covariance function is regularly varying at any location. An asymptotic theory is developed for the procedure on a fixed domain by letting the grid size tend to zero.

The Cauchy distribution is usually presented as a mathematical curiosity, an exception to the Law of Large Numbers, or even as an “Evil” distribution in some introductory courses. It therefore surprised us when Drton and Xiao [Bernoulli 22 (2016) 38–59] proved the following result for $m=2$ and conjectured it for $m\ge3$. Let $X=(X_{1},\ldots,X_{m})$ and $Y=(Y_{1},\ldots,Y_{m})$ be i.i.d. $\mathrm{N}(0,\Sigma)$, where $\Sigma=\{\sigma_{ij}\}\ge0$ is an $m\times m$ and arbitrary covariance matrix with $\sigma_{jj}>0$ for all $1\leq j\leq m$. Then

\[Z=\sum_{j=1}^{m}w_{j}{\frac{X_{j}}{Y_{j}}}\sim\operatorname{Cauchy}(0,1),\] as long as $\vec{w}=(w_{1},\ldots,w_{m})$ is independent of $(X,Y)$, $w_{j}\ge0,j=1,\ldots,m$, and $\sum_{j=1}^{m}w_{j}=1$. In this note, we present an elementary proof of this conjecture for any $m\geq2$ by linking $Z$ to a geometric characterization of $\operatorname{Cauchy}(0,1)$ given in Willams [Ann. Math. Stat. 40 (1969) 1083–1085]. This general result is essential to the large sample behavior of Wald tests in many applications such as factor models and contingency tables. It also leads to other unexpected results such as

\[\sum_{i=1}^{m}\sum_{j=1}^{m}\frac{w_{i}w_{j}\sigma_{ij}}{X_{i}X_{j}}\sim \mbox{Lévy}(0,1).\] This generalizes the “super Cauchy phenomenon” that the average of $m$ i.i.d. standard Lévy variables (i.e., inverse chi-squared variables with one degree of freedom) has the same distribution as that of a single standard Lévy variable multiplied by $m$ (which is obtained by taking $w_{j}=1/m$ and $\Sigma$ to be the identity matrix).

Large-scale precision matrix estimation is of fundamental importance yet challenging in many contemporary applications for recovering Gaussian graphical models. In this paper, we suggest a new approach of innovated scalable efficient estimation (ISEE) for estimating large precision matrix. Motivated by the innovated transformation, we convert the original problem into that of large covariance matrix estimation. The suggested method combines the strengths of recent advances in high-dimensional sparse modeling and large covariance matrix estimation. Compared to existing approaches, our method is scalable and can deal with much larger precision matrices with simple tuning. Under mild regularity conditions, we establish that this procedure can recover the underlying graphical structure with significant probability and provide efficient estimation of link strengths. Both computational and theoretical advantages of the procedure are evidenced through simulation and real data examples.

We study behavior of the restricted maximum likelihood (REML) estimator under a misspecified linear mixed model (LMM) that has received much attention in recent genome-wide association studies. The asymptotic analysis establishes consistency of the REML estimator of the variance of the errors in the LMM, and convergence in probability of the REML estimator of the variance of the random effects in the LMM to a certain limit, which is equal to the true variance of the random effects multiplied by the limiting proportion of the nonzero random effects present in the LMM. The asymptotic results also establish convergence rate (in probability) of the REML estimators as well as a result regarding convergence of the asymptotic conditional variance of the REML estimator. The asymptotic results are fully supported by the results of empirical studies, which include extensive simulation studies that compare the performance of the REML estimator (under the misspecified LMM) with other existing methods, and real data applications (only one example is presented) that have important genetic implications.

For a response variable $Y$, and a $d$ dimensional vector of covariates $\mathbf{X}$, the first projective direction, $\mathbf{\vartheta}$, is defined as the direction that accounts for the most variability in $Y$. The asymptotic distribution of an estimator of a trimmed version of $\mathbf{\vartheta}$ has been characterized only under the assumption of the single index model (SIM). This paper proposes the use of a flexible trimming function in the objective function, which results in the consistent estimation of $\mathbf{\vartheta}$. It also derives the asymptotic normality of the proposed estimator, and characterizes the components of the asymptotic variance which vanish when the SIM holds.

We consider nonparametric estimation of a regression curve when the data are observed with multiplicative distortion which depends on an observed confounding variable. We suggest several estimators, ranging from a relatively simple one that relies on restrictive assumptions usually made in the literature, to a sophisticated piecewise approach that involves reconstructing a smooth curve from an estimator of a constant multiple of its absolute value, and which can be applied in much more general scenarios. We show that, although our nonparametric estimators are constructed from predictors of the unobserved undistorted data, they have the same first-order asymptotic properties as the standard estimators that could be computed if the undistorted data were available. We illustrate the good numerical performance of our methods on both simulated and real datasets.

This paper considers the noisy sparse phase retrieval problem: recovering a sparse signal $\mathbf{x}\in\mathbb{R}^{p}$ from noisy quadratic measurements $y_{j}=(\mathbf{a}_{j}'\mathbf{x})^{2}+\varepsilon_{j}$, $j=1,\ldots,m$, with independent sub-exponential noise $\varepsilon_{j}$. The goals are to understand the effect of the sparsity of $\mathbf{x}$ on the estimation precision and to construct a computationally feasible estimator to achieve the optimal rates adaptively. Inspired by the Wirtinger Flow [IEEE Trans. Inform. Theory 61 (2015) 1985–2007] proposed for non-sparse and noiseless phase retrieval, a novel thresholded gradient descent algorithm is proposed and it is shown to adaptively achieve the minimax optimal rates of convergence over a wide range of sparsity levels when the $\mathbf{a}_{j}$’s are independent standard Gaussian random vectors, provided that the sample size is sufficiently large compared to the sparsity of $\mathbf{x}$.

Recently, network analysis has gained more and more attention in statistics, as well as in computer science, probability and applied mathematics. Community detection for the stochastic block model (SBM) is probably the most studied topic in network analysis. Many methodologies have been proposed. Some beautiful and significant phase transition results are obtained in various settings. In this paper, we provide a general minimax theory for community detection. It gives minimax rates of the mis-match ratio for a wide rage of settings including homogeneous and inhomogeneous SBMs, dense and sparse networks, finite and growing number of communities. The minimax rates are exponential, different from polynomial rates we often see in statistical literature. An immediate consequence of the result is to establish threshold phenomenon for strong consistency (exact recovery) as well as weak consistency (partial recovery). We obtain the upper bound by a range of penalized likelihood-type approaches. The lower bound is achieved by a novel reduction from a global mis-match ratio to a local clustering problem for one node through an exchangeability property.

Nonparametric estimation of mean and covariance functions is important in functional data analysis. We investigate the performance of local linear smoothers for both mean and covariance functions with a general weighing scheme, which includes two commonly used schemes, equal weight per observation (OBS), and equal weight per subject (SUBJ), as two special cases. We provide a comprehensive analysis of their asymptotic properties on a unified platform for all types of sampling plan, be it dense, sparse or neither. Three types of asymptotic properties are investigated in this paper: asymptotic normality, $L^{2}$ convergence and uniform convergence. The asymptotic theories are unified on two aspects: (1) the weighing scheme is very general; (2) the magnitude of the number $N_{i}$ of measurements for the $i$th subject relative to the sample size $n$ can vary freely. Based on the relative order of $N_{i}$ to $n$, functional data are partitioned into three types: non-dense, dense and ultra-dense functional data for the OBS and SUBJ schemes. These two weighing schemes are compared both theoretically and numerically. We also propose a new class of weighing schemes in terms of a mixture of the OBS and SUBJ weights, of which theoretical and numerical performances are examined and compared.