The focus of this paper is on trend estimation for a general state-space model Y_t=μ_t+ɛ_t, where the dth difference of the trend {μ_t} is assumed to be i.i.d., and the error sequence {ɛ_t} is assumed to be a mean zero stationary process. A fairly precise asymptotic expression of the mean square error is derived for the estimator obtained by penalizing the dth order differences. Optimal rate of convergence is obtained, and it is shown to be “asymptotically equivalent” to a nonparametric estimator of a fixed trend model of smoothness of order d−0.5. The results of this paper show that the optimal rate of convergence for the stochastic and nonstochastic cases are different. A criterion for selecting the penalty parameter and degree of difference d is given, along with an application to the global temperature data, which shows that a longer term history has nonlinearities that are important to take into consideration.

In regression analysis, we employ contour projection (CP) to develop a new dimension reduction theory. Accordingly, we introduce the notions of the central contour subspace and generalized contour subspace. We show that both of their structural dimensions are no larger than that of the central subspace Cook [Regression Graphics (1998b) Wiley]. Furthermore, we employ CP-sliced inverse regression, CP-sliced average variance estimation and CP-directional regression to estimate the generalized contour subspace, and we subsequently obtain their theoretical properties. Monte Carlo studies demonstrate that the three CP-based dimension reduction methods outperform their corresponding non-CP approaches when the predictors have heavy-tailed elliptical distributions. An empirical example is also presented to illustrate the usefulness of the CP method.

We propose a new sparsity-smoothness penalty for high-dimensional generalized additive models. The combination of sparsity and smoothness is crucial for mathematical theory as well as performance for finite-sample data. We present a computationally efficient algorithm, with provable numerical convergence properties, for optimizing the penalized likelihood. Furthermore, we provide oracle results which yield asymptotic optimality of our estimator for high dimensional but sparse additive models. Finally, an adaptive version of our sparsity-smoothness penalized approach yields large additional performance gains.

In this paper, we give an explanation to the failure of two likelihood ratio procedures for testing about covariance matrices from Gaussian populations when the dimension p is large compared to the sample size n. Next, using recent central limit theorems for linear spectral statistics of sample covariance matrices and of random F-matrices, we propose necessary corrections for these LR tests to cope with high-dimensional effects. The asymptotic distributions of these corrected tests under the null are given. Simulations demonstrate that the corrected LR tests yield a realized size close to nominal level for both moderate p (around 20) and high dimension, while the traditional LR tests with χ² approximation fails.

Another contribution from the paper is that for testing the equality between two covariance matrices, the proposed correction applies equally for non-Gaussian populations yielding a valid pseudo-likelihood ratio test.

Semiparametric models are often considered for analyzing longitudinal data for a good balance between flexibility and parsimony. In this paper, we study a class of marginal partially linear quantile models with possibly varying coefficients. The functional coefficients are estimated by basis function approximations. The estimation procedure is easy to implement, and it requires no specification of the error distributions. The asymptotic properties of the proposed estimators are established for the varying coefficients as well as for the constant coefficients. We develop rank score tests for hypotheses on the coefficients, including the hypotheses on the constancy of a subset of the varying coefficients. Hypothesis testing of this type is theoretically challenging, as the dimensions of the parameter spaces under both the null and the alternative hypotheses are growing with the sample size. We assess the finite sample performance of the proposed method by Monte Carlo simulation studies, and demonstrate its value by the analysis of an AIDS data set, where the modeling of quantiles provides more comprehensive information than the usual least squares approach.

This paper revisits a meta-analysis method proposed by Pearson [Biometrika 26 (1934) 425–442] and first used by David [Biometrika 26 (1934) 1–11]. It was thought to be inadmissible for over fifty years, dating back to a paper of Birnbaum [J. Amer. Statist. Assoc. 49 (1954) 559–574]. It turns out that the method Birnbaum analyzed is not the one that Pearson proposed. We show that Pearson’s proposal is admissible. Because it is admissible, it has better power than the standard test of Fisher [Statistical Methods for Research Workers (1932) Oliver and Boyd] at some alternatives, and worse power at others. Pearson’s method has the advantage when all or most of the nonzero parameters share the same sign. Pearson’s test has proved useful in a genomic setting, screening for age-related genes. This paper also presents an FFT-based method for getting hard upper and lower bounds on the CDF of a sum of nonnegative random variables.

This paper considers a class of nonparametric autoregressive models with nonstationarity. We propose a nonparametric kernel test for the conditional mean and then establish an asymptotic distribution of the proposed test. Both the setting and the results differ from earlier work on nonparametric autoregression with stationarity. In addition, we develop a new bootstrap simulation scheme for the selection of a suitable bandwidth parameter involved in the kernel test as well as the choice of a simulated critical value. The finite-sample performance of the proposed test is assessed using one simulated example and one real data example.

The bootstrap is a popular and convenient method for quantifying the authority of an empirical ordering of attributes, for example of a ranking of the performance of institutions or of the influence of genes on a response variable. In the first of these examples, the number, p, of quantities being ordered is sometimes only moderate in size; in the second it can be very large, often much greater than sample size. However, we show that in both types of problem the conventional bootstrap can produce inconsistency. Moreover, the standard n-out-of-n bootstrap estimator of the distribution of an empirical rank may not converge in the usual sense; the estimator may converge in distribution, but not in probability. Nevertheless, in many cases the bootstrap correctly identifies the support of the asymptotic distribution of ranks. In some contemporary problems, bootstrap prediction intervals for ranks are particularly long, and in this context, we also quantify the accuracy of bootstrap methods, showing that the standard bootstrap gets the order of magnitude of the interval right, but not the constant multiplier of interval length. The m-out-of-n bootstrap can improve performance and produce statistical consistency, but it requires empirical choice of m; we suggest a tuning solution to this problem. We show that in genomic examples, where it might be expected that the standard, “synchronous” bootstrap will successfully accommodate nonindependence of vector components, that approach can produce misleading results. An “independent component” bootstrap can overcome these difficulties, even in cases where components are not strictly independent.

This paper focuses on obtaining clustering information about a distribution from its i.i.d. samples. We develop theoretical results to understand and use clustering information contained in the eigenvectors of data adjacency matrices based on a radial kernel function with a sufficiently fast tail decay. In particular, we provide population analyses to gain insights into which eigenvectors should be used and when the clustering information for the distribution can be recovered from the sample. We learn that a fixed number of top eigenvectors might at the same time contain redundant clustering information and miss relevant clustering information. We use this insight to design the data spectroscopic clustering (DaSpec) algorithm that utilizes properly selected eigenvectors to determine the number of clusters automatically and to group the data accordingly. Our findings extend the intuitions underlying existing spectral techniques such as spectral clustering and Kernel Principal Components Analysis, and provide new understanding into their usability and modes of failure. Simulation studies and experiments on real-world data are conducted to show the potential of our algorithm. In particular, DaSpec is found to handle unbalanced groups and recover clusters of different shapes better than the competing methods.

Generalized likelihood ratio (GLR) test statistics are often used in the detection of spatial clustering in case-control and case-population datasets to check for a significantly large proportion of cases within some scanning window. The traditional spatial scan test statistic takes the supremum GLR value over all windows, whereas the average likelihood ratio (ALR) test statistic that we consider here takes an average of the GLR values. Numerical experiments in the literature and in this paper show that the ALR test statistic has more power compared to the spatial scan statistic. We develop in this paper accurate tail probability approximations of the ALR test statistic that allow us to by-pass computer intensive Monte Carlo procedures to estimate p-values. In models that adjust for covariates, these Monte Carlo evaluations require an initial fitting of parameters that can result in very biased p-value estimates.

This paper proposes new tests of conditional independence of two random variables given a single-index involving an unknown finite-dimensional parameter. The tests employ Rosenblatt transforms and are shown to be distribution-free while retaining computational convenience. Some results from Monte Carlo simulations are presented and discussed.

In this paper, we introduce an asymptotic test procedure to assess the stability of volatilities and cross-volatilites of linear and nonlinear multivariate time series models. The test is very flexible as it can be applied, for example, to many of the multivariate GARCH models established in the literature, and also works well in the case of high dimensionality of the underlying data. Since it is nonparametric, the procedure avoids the difficulties associated with parametric model selection, model fitting and parameter estimation. We provide the theoretical foundation for the test and demonstrate its applicability via a simulation study and an analysis of financial data. Extensions to multiple changes and the case of infinite fourth moments are also discussed.

We consider the common nonlinear regression model where the variance, as well as the mean, is a parametric function of the explanatory variables. The c-optimal design problem is investigated in the case when the parameters of both the mean and the variance function are of interest. A geometric characterization of c-optimal designs in this context is presented, which generalizes the classical result of Elfving [Ann. Math. Statist. 23 (1952) 255–262] for c-optimal designs. As in Elfving’s famous characterization, c-optimal designs can be described as representations of boundary points of a convex set. However, in the case where there appear parameters of interest in the variance, the structure of the Elfving set is different. Roughly speaking, the Elfving set corresponding to a heteroscedastic regression model is the convex hull of a set of ellipsoids induced by the underlying model and indexed by the design space. The c-optimal designs are characterized as representations of the points where the line in direction of the vector c intersects the boundary of the new Elfving set. The theory is illustrated in several examples including pharmacokinetic models with random effects.

Principal Component Analysis (PCA) is an important tool of dimension reduction especially when the dimension (or the number of variables) is very high. Asymptotic studies where the sample size is fixed, and the dimension grows [i.e., High Dimension, Low Sample Size (HDLSS)] are becoming increasingly relevant. We investigate the asymptotic behavior of the Principal Component (PC) directions. HDLSS asymptotics are used to study consistency, strong inconsistency and subspace consistency. We show that if the first few eigenvalues of a population covariance matrix are large enough compared to the others, then the corresponding estimated PC directions are consistent or converge to the appropriate subspace (subspace consistency) and most other PC directions are strongly inconsistent. Broad sets of sufficient conditions for each of these cases are specified and the main theorem gives a catalogue of possible combinations. In preparation for these results, we show that the geometric representation of HDLSS data holds under general conditions, which includes a ρ-mixing condition and a broad range of sphericity measures of the covariance matrix.

Hidden Markov models (HMMs) are probabilistic functions of finite Markov chains, or, put in other words, state space models with finite state space. In this paper, we examine subspace estimation methods for HMMs whose output lies a finite set as well. In particular, we study the geometric structure arising from the nonminimality of the linear state space representation of HMMs, and consistency of a subspace algorithm arising from a certain factorization of the singular value decomposition of the estimated linear prediction matrix. For this algorithm, we show that the estimates of the transition and emission probability matrices are consistent up to a similarity transformation, and that the m-step linear predictor computed from the estimated system matrices is consistent, i.e., converges to the true optimal linear m-step predictor.

Generalized linear models and the quasi-likelihood method extend the ordinary regression models to accommodate more general conditional distributions of the response. Nonparametric methods need no explicit parametric specification, and the resulting model is completely determined by the data themselves. However, nonparametric estimation schemes generally have a slower convergence rate such as the local polynomial smoothing estimation of nonparametric generalized linear models studied in Fan, Heckman and Wand [J. Amer. Statist. Assoc. 90 (1995) 141–150]. In this work, we propose a unified family of parametrically-guided nonparametric estimation schemes. This combines the merits of both parametric and nonparametric approaches and enables us to incorporate prior knowledge. Asymptotic results and numerical simulations demonstrate the improvement of our new estimation schemes over the original nonparametric counterpart.

One aspect of evaluating the design for an experiment is the discovery of the relationships between subspaces of the data space. Initially we establish the notation and methods for evaluating an experiment with a single randomization. Starting with two structures, or orthogonal decompositions of the data space, we describe how to combine them to form the overall decomposition for a single-randomization experiment that is “structure balanced.” The relationships between the two structures are characterized using efficiency factors. The decomposition is encapsulated in a decomposition table. Then, for experiments that involve multiple randomizations forming a chain, we take several structures that pairwise are structure balanced and combine them to establish the form of the orthogonal decomposition for the experiment. In particular, it is proven that the properties of the design for such an experiment are derived in a straightforward manner from those of the individual designs. We show how to formulate an extended decomposition table giving the sources of variation, their relationships and their degrees of freedom, so that competing designs can be evaluated.

This paper considers the efficient estimation of copula-based semiparametric strictly stationary Markov models. These models are characterized by nonparametric invariant (one-dimensional marginal) distributions and parametric bivariate copula functions where the copulas capture temporal dependence and tail dependence of the processes. The Markov processes generated via tail dependent copulas may look highly persistent and are useful for financial and economic applications. We first show that Markov processes generated via Clayton, Gumbel and Student’s t copulas and their survival copulas are all geometrically ergodic. We then propose a sieve maximum likelihood estimation (MLE) for the copula parameter, the invariant distribution and the conditional quantiles. We show that the sieve MLEs of any smooth functional is root-n consistent, asymptotically normal and efficient and that their sieve likelihood ratio statistics are asymptotically chi-square distributed. Monte Carlo studies indicate that, even for Markov models generated via tail dependent copulas and fat-tailed marginals, our sieve MLEs perform very well.

This paper studies the sparsistency and rates of convergence for estimating sparse covariance and precision matrices based on penalized likelihood with nonconvex penalty functions. Here, sparsistency refers to the property that all parameters that are zero are actually estimated as zero with probability tending to one. Depending on the case of applications, sparsity priori may occur on the covariance matrix, its inverse or its Cholesky decomposition. We study these three sparsity exploration problems under a unified framework with a general penalty function. We show that the rates of convergence for these problems under the Frobenius norm are of order (s_n log p_n/n)^1/2, where s_n is the number of nonzero elements, p_n is the size of the covariance matrix and n is the sample size. This explicitly spells out the contribution of high-dimensionality is merely of a logarithmic factor. The conditions on the rate with which the tuning parameter λ_n goes to 0 have been made explicit and compared under different penalties. As a result, for the L₁-penalty, to guarantee the sparsistency and optimal rate of convergence, the number of nonzero elements should be small: s_n'=O(p_n) at most, among O(p_n²) parameters, for estimating sparse covariance or correlation matrix, sparse precision or inverse correlation matrix or sparse Cholesky factor, where s_n' is the number of the nonzero elements on the off-diagonal entries. On the other hand, using the SCAD or hard-thresholding penalty functions, there is no such a restriction.

We investigate a method for extracting nonlinear principal components (NPCs). These NPCs maximize variation subject to smoothness and orthogonality constraints; but we allow for a general class of constraints and multivariate probability densities, including densities without compact support and even densities with algebraic tails. We provide primitive sufficient conditions for the existence of these NPCs. By exploiting the theory of continuous-time, reversible Markov diffusion processes, we give a different interpretation of these NPCs and the smoothness constraints. When the diffusion matrix is used to enforce smoothness, the NPCs maximize long-run variation relative to the overall variation subject to orthogonality constraints. Moreover, the NPCs behave as scalar autoregressions with heteroskedastic innovations; this supports semiparametric identification and estimation of a multivariate reversible diffusion process and tests of the overidentifying restrictions implied by such a process from low-frequency data. We also explore implications for stationary, possibly nonreversible diffusion processes. Finally, we suggest a sieve method to estimate the NPCs from discretely-sampled data.