This paper provides answers to questions regarding the almost sure limiting behavior of rooted, binary tree-structured rules for regression. Examples show that questions raised by Gordon and Olshen in 1984 have negative answers. For these examples of regression functions and sequences of their associated binary tree-structured approximations, for all regression functions except those in a set of the first category, almost sure consistency fails dramatically on events of full probability. One consequence is that almost sure consistency of binary tree-structured rules such as CART requires conditions beyond requiring that (1) the regression function be in ℒ¹, (2) partitions of a Euclidean feature space be into polytopes with sides parallel to coordinate axes, (3) the mesh of the partitions becomes arbitrarily fine almost surely and (4) the empirical learning sample content of each polytope be “large enough.” The material in this paper includes the solution to a problem raised by Dudley in discussions. The main results have a corollary regarding the lack of almost sure consistency of certain Bayes-risk consistent rules for classification.

We derive the maximum bias functions of the MM-estimates and the constrained M-estimates or CM-estimates of regression and compare them to the maximum bias functions of the S-estimates and the τ-estimates of regression. In these comparisons, the CM-estimates tend to exhibit the most favorable bias-robustness properties. Also, under the Gaussian model, it is shown how one can construct a CM-estimate which has a smaller maximum bias function than a given S-estimate, that is, the resulting CM-estimate dominates the S-estimate in terms of maxbias and, at the same time, is considerably more efficient.

In this paper, we systematically study the consistency of sliced average variance estimation (SAVE). The findings reveal that when the response is continuous, the asymptotic behavior of SAVE is rather different from that of sliced inverse regression (SIR). SIR can achieve $\sqrt{n}$ consistency even when each slice contains only two data points. However, SAVE cannot be $\sqrt{n}$ consistent and it even turns out to be not consistent when each slice contains a fixed number of data points that do not depend on n, where n is the sample size. These results theoretically confirm the notion that SAVE is more sensitive to the number of slices than SIR. Taking this into account, a bias correction is recommended in order to allow SAVE to be $\sqrt{n}$ consistent. In contrast, when the response is discrete and takes finite values, $\sqrt{n}$ consistency can be achieved. Therefore, an approximation through discretization, which is commonly used in practice, is studied. A simulation study is carried out for the purposes of illustration.

In functional linear regression, the slope “parameter” is a function. Therefore, in a nonparametric context, it is determined by an infinite number of unknowns. Its estimation involves solving an ill-posed problem and has points of contact with a range of methodologies, including statistical smoothing and deconvolution. The standard approach to estimating the slope function is based explicitly on functional principal components analysis and, consequently, on spectral decomposition in terms of eigenvalues and eigenfunctions. We discuss this approach in detail and show that in certain circumstances, optimal convergence rates are achieved by the PCA technique. An alternative approach based on quadratic regularisation is suggested and shown to have advantages from some points of view.

Quantile regression provides a framework for modeling statistical quantities of interest other than the conditional mean. The regression methodology is well developed for linear models, but less so for nonparametric models. We consider conditional quantiles with varying coefficients and propose a methodology for their estimation and assessment using polynomial splines. The proposed estimators are easy to compute via standard quantile regression algorithms and a stepwise knot selection algorithm. The proposed Rao-score-type test that assesses the model against a linear model is also easy to implement. We provide asymptotic results on the convergence of the estimators and the null distribution of the test statistic. Empirical results are also provided, including an application of the methodology to forced expiratory volume (FEV) data.

Given an m-dimensional compact submanifold M of Euclidean space R^s, the concept of mean location of a distribution, related to mean or expected vector, is generalized to more general R^s-valued functionals including median location, which is derived from the spatial median. The asymptotic statistical inference for general functionals of distributions on such submanifolds is elaborated. Convergence properties are studied in relation to the behavior of the underlying distributions with respect to the cutlocus. An application is given in the context of independent, but not identically distributed, samples, in particular, to a multisample setup.

The ability to remove a large amount of noise and the ability to preserve most structure are desirable properties of an image smoother. Unfortunately, they usually seem to be at odds with each other; one can only improve one property at the cost of the other. By combining M-smoothing and least-squares-trimming, the TM-smoother is introduced as a means to unify corner-preserving properties and outlier robustness. To identify edge- and corner-preserving properties, a new theory based on differential geometry is developed. Further, robustness concepts are transferred to image processing. In two examples, the TM-smoother outperforms other corner-preserving smoothers. A software package containing both the TM- and the M-smoother can be downloaded from the Internet.

Deheuvels [J. Multivariate Anal. 11 (1981) 102–113] and Genest and Rémillard [Test 13 (2004) 335–369] have shown that powerful rank tests of multivariate independence can be based on combinations of asymptotically independent Cramér–von Mises statistics derived from a Möbius decomposition of the empirical copula process. A result on the large-sample behavior of this process under contiguous sequences of alternatives is used here to give a representation of the limiting distribution of such test statistics and to compute their relative local asymptotic efficiency. Local power curves and asymptotic relative efficiencies are compared under familiar classes of copula alternatives.

We consider the asymptotic behavior of posterior distributions and Bayes estimators based on observations which are required to be neither independent nor identically distributed. We give general results on the rate of convergence of the posterior measure relative to distances derived from a testing criterion. We then specialize our results to independent, nonidentically distributed observations, Markov processes, stationary Gaussian time series and the white noise model. We apply our general results to several examples of infinite-dimensional statistical models including nonparametric regression with normal errors, binary regression, Poisson regression, an interval censoring model, Whittle estimation of the spectral density of a time series and a nonlinear autoregressive model.

This article discusses the problem of estimation of parameters in finite mixtures when the mixture components are assumed to be symmetric and to come from the same location family. We refer to these mixtures as semi-parametric because no additional assumptions other than symmetry are made regarding the parametric form of the component distributions. Because the class of symmetric distributions is so broad, identifiability of parameters is a major issue in these mixtures. We develop a notion of identifiability of finite mixture models, which we call k-identifiability, where k denotes the number of components in the mixture. We give sufficient conditions for k-identifiability of location mixtures of symmetric components when k=2 or 3. We propose a novel distance-based method for estimating the (location and mixing) parameters from a k-identifiable model and establish the strong consistency and asymptotic normality of the estimator. In the specific case of L₂-distance, we show that our estimator generalizes the Hodges–Lehmann estimator. We discuss the numerical implementation of these procedures, along with an empirical estimate of the component distribution, in the two-component case. In comparisons with maximum likelihood estimation assuming normal components, our method produces somewhat higher standard error estimates in the case where the components are truly normal, but dramatically outperforms the normal method when the components are heavy-tailed.

We derive an asymptotic theory of nonparametric estimation for a time series regression model Z_t=f(X_t)+W_t, where {X_t} and {Z_t} are observed nonstationary processes and {W_t} is an unobserved stationary process. In econometrics, this can be interpreted as a nonlinear cointegration type relationship, but we believe that our results are of wider interest. The class of nonstationary processes allowed for {X_t} is a subclass of the class of null recurrent Markov chains. This subclass contains random walk, unit root processes and nonlinear processes. We derive the asymptotics of a nonparametric estimate of f(x) under the assumption that {W_t} is a Markov chain satisfying some mixing conditions. The finite-sample properties of f̂(x) are studied by means of simulation experiments.

We consider settings where data are available on a nonparametric function and various partial derivatives. Such circumstances arise in practice, for example in the joint estimation of cost and input functions in economics. We show that when derivative data are available, local averages can be replaced in certain dimensions by nonlocal averages, thus reducing the nonparametric dimension of the problem. We derive optimal rates of convergence and conditions under which dimension reduction is achieved. Kernel estimators and their properties are analyzed, although other estimators, such as local polynomial, spline and nonparametric least squares, may also be used. Simulations and an application to the estimation of electricity distribution costs are included.

Statistical estimation and inference for marginal hazard models with varying coefficients for multivariate failure time data are important subjects in survival analysis. A local pseudo-partial likelihood procedure is proposed for estimating the unknown coefficient functions. A weighted average estimator is also proposed in an attempt to improve the efficiency of the estimator. The consistency and asymptotic normality of the proposed estimators are established and standard error formulas for the estimated coefficients are derived and empirically tested. To reduce the computational burden of the maximum local pseudo-partial likelihood estimator, a simple and useful one-step estimator is proposed. Statistical properties of the one-step estimator are established and simulation studies are conducted to compare the performance of the one-step estimator to that of the maximum local pseudo-partial likelihood estimator. The results show that the one-step estimator can save computational cost without compromising performance both asymptotically and empirically and that an optimal weighted average estimator is more efficient than the maximum local pseudo-partial likelihood estimator. A data set from the Busselton Population Health Surveys is analyzed to illustrate our proposed methodology.

This paper studies the estimation of the volatility parameter in a model where the driving process is a Brownian motion or a more general symmetric stable process that is perturbed by another Lévy process. We distinguish between a parametric case, where the law of the perturbing process is known, and a semiparametric case, where it is not. In the parametric case, we construct estimators which are asymptotically efficient. In the semiparametric case, we can obtain asymptotically efficient estimators by sampling at a sufficiently high frequency, and these estimators are efficient uniformly in the law of the perturbing process.

In many contexts such as queuing theory, spatial statistics, geostatistics and meteorology, data are observed at irregular spatial positions. One model of this situation involves considering the observation points as generated by a Poisson process. Under this assumption, we study the limit behavior of the partial sums of the marked point process {(t_i, X(t_i))}, where X(t) is a stationary random field and the points t_i are generated from an independent Poisson random measure ℕ on ℝ^d. We define the sample mean and sample variance statistics and determine their joint asymptotic behavior in a heavy-tailed setting, thus extending some finite variance results of Karr [Adv. in Appl. Probab. 18 (1986) 406–422]. New results on subsampling in the context of a marked point process are also presented, with the application of forming a confidence interval for the unknown mean under an unknown degree of heavy tails.

In the design of efficient simulation algorithms, one is often beset with a poor choice of proposal distributions. Although the performance of a given simulation kernel can clarify a posteriori how adequate this kernel is for the problem at hand, a permanent on-line modification of kernels causes concerns about the validity of the resulting algorithm. While the issue is most often intractable for MCMC algorithms, the equivalent version for importance sampling algorithms can be validated quite precisely. We derive sufficient convergence conditions for adaptive mixtures of population Monte Carlo algorithms and show that Rao–Blackwellized versions asymptotically achieve an optimum in terms of a Kullback divergence criterion, while more rudimentary versions do not benefit from repeated updating.

A sequence of null hypotheses regarding the number of negligible effects (zero effects) in orthogonal saturated designs is formulated. Two step-up simultaneous testing procedures are proposed to identify active effects (nonzero effects) under the commonly used assumption of effect sparsity. It is shown that each procedure controls the experimentwise error rate at a given α level in the strong sense.