Assume that two sequences from a finite alphabet are optimally aligned according to a scoring system that rewards similarities according to a general scoring scheme and penalizes gaps (insertions and deletions). Under the assumption that the letters in each sequence are independent and identically distributed and the two sequences are also independent, approximate $p$-values are obtained for the optimal local alignment when either (i) there are at most a fixed number of gaps, or (ii) the gap initiation cost is sufficiently large. In the latter case the approximation can be written in the same form as the well-known case of ungapped alignments.

We consider the problem of estimating an unknown function $f$ from $N$ noisy observations on a random grid. In this paper we address the following aggregation problem: given $M$ functions $f_1,\dots, f_M$, find an “aggregated ”estimator which approximates $f$ nearly as well as the best convex combination $f^*$ of $f_1,\dots, f_M$. We propose algorithms which provide approximations of $f^*$ with expected $L_2$ accuracy $O(N^{-1/4}\ln^{1/4} M$. We show that this approximation rate cannot be significantly improved. We discuss two specific applications: nonparametric prediction for a dynamic system with output nonlinearity and reconstruction in the Jones – Barron class.

We suggest a sequential, or “tracking,”algorithm for estimating a smooth fault line in a response surface. The method starts with an approximation to a point on the line, and from there the line is tracked as it meanders through the plane. The technique differs from recent approaches in that it does not require a large part of the plane to be searched for evidence of a fault line. This offers potential computational savings, and produces a method that is invariant under rotations of coordinate axes (except insofar as a rotation might affect the estimated starting point, and the relative orientation of the grid on which calculations are done). That feature is important if design points are not located on a regular grid. We investigate properties of the method under very general conditions on the design, allowing Poisson cluster processes, jiggled grid processes and deterministic, regular lattices. Uniform rates of convergence are derived in all these settings, for the case of noisy data, and shown to be within logarithmic factors of optimal pointwise convergence rates in the no-noise setting.

To deal with the curse of dimensionality in high-dimensional nonparametric problems, we consider using tensor product space ANOVA models, which extend the popular additive models and are able to capture interactions of any order. The multivariate function is given an ANOVA decomposition, that is, it is expressed as a constant plus the sum of functions of one variable (main effects), plus the sum of functions of two variables (two-factor interactions)and so on. We assume the interactions to be in tensor product spaces.We show in both regression and white noise settings, the optimal rate of convergence for the TPS-ANOVA model is within a log factor of the one-dimensional optimal rate, and that the penalized likelihood estimator in TPS-ANOVA achieves this rate of convergence. The quick optimal rate of the TPS-ANOVA model makes it very preferable in high-dimensional function estimation. Many properties of the tensor product space of Sobolev –Hilbert spaces are also given.

Given a sample $X_1,\dots, X_n$ from a distribution $F$, the problem of constructing nonparametric confidence intervals for the mean $\mu(F)$ is considered. Unlike bootstrap procedures or those based on normal approximations, we insist on any procedure being truly nonparametric in the sense that the probability that the confidence interval contains $\mu(F)$ based on a sample of size $n$ from $F$ be at least $1 - \alpha$ for all $F$ and all $n$. Bahadur and Savage proved it is impossible to find an effective (or bounded) confidence interval for $\mu(F)$ without some restrictions. Thus,we assume that $F$ is supported on a known compact set, which we take to be $[0, 1]$. In this setting, an asymptotic efficiency result is obtained that gives a lower bound on the size of any conservative interval. We then provide a construction of an interval that meets our finite sample requirement on level, yet has an asymptotic efficiency property. Thus, the price to be paid for using fully nonparametric procedures when considering the trade-off between exact inference statements and asymptotic efficiency is negligible. Much of what is accomplished for the mean generalizes to other settings as well.

We study two estimators of the mean function of a countingprocess based on “panel count data.” The setting for “panel count data” is one in which $n$ independent subjects, each with a counting process with common mean function, are observed at several possibly different times duringa study. Following a model proposed by Schick and Yu, we allow the number of observation times, and the observation times themselves, to be random variables. Our goal is to estimate the mean function of the counting process. We show that the estimator of the mean function proposed by Sun and Kalbfleisch can be viewed as a pseudo-maximum likelihood estimator when a non-homogeneous Poisson process model is assumed for the counting process. We establish consistency of both the nonparametric pseudo maximum likelihood estimator of Sun and Kalbfleisch and the full maximum likeli- hood estimator, even if the underlying counting process is not a Poisson process.We also derive the asymptotic distribution of both estimators at a fixed time $t$, and compare the resulting theoretical relative efficiency with finite sample relative efficiency by way of a limited Monte-Carlo study.

We consider a nonparametric diffusion process whose drift and diffusion coefficients are nonparametric functions of the state variable. The goal is to estimate the unknown drift coefficient.We apply a locally linear smoother with a data-driven bandwidth choice. The procedure is fully adaptive and nearly optimal up to a log log factor. The results about the quality of estimation are nonasymptotic and do not require any ergodic or mixing properties of the observed process.

We present the explicit solution of the Bayesian problem of sequential testing of two simple hypotheses about the intensity of an observed Poisson process. The method of proof consists of reducing the initial problem to a free-boundary differential-difference Stephan problem and solving the latter by use of the principles of smooth and continuous fit. A rigorous proof of the optimality of the Wald’s sequential probability ratio test in the variational formulation of the problem is obtained as a consequence of the solution of the Bayesian problem.

Suppose the $X_0,\dots, X_n$ are observations of a one-dimensional stochastic dynamic process described by autoregression equations when the autoregressive parameter is drifted with time, i.e. it is some function of time: $\theta_0,\dots, \theta_n$, with $\theta_k = \theta(k/n)$. The function $\theta(t)$ is assumed to belong a priori to a predetermined nonparametric class of functions satisfying the Lipschitz smoothness condition. At each time point $t$ those observations are accessible which have been obtained during the preceding time interval. A recursive algorithm is proposed to estimate $\theta(t)$.Under some conditions on the model,we derive the rate of convergence of the proposed estimator when the frequencyof observations $n$ tends to infinity.

The class of nonlinear stochastic regression models includes most of the linear and nonlinear models used in time series, stochastic control and stochastic approximation schemes.The consistency of least squares estimators was established first by Lai.We present another set of sufficient conditions for consistency, which avoid the use of partial derivatives and are closer in spirit to the conditions presented by Wu for non-stochastic regression models with independent errors.

This article considers the estimation of structured correlation matrices in infinitely differentiable Gaussian random field models.The problem is essentially motivated by the stochastic modeling of smooth deterministic responses in computer experiments.In particular, the log-likelihood function is determined explicitly in closed-form and the sieve maximum likelihood estimators are shown to be strongly consistent under mild conditions.

In survival analysis and in the analysis of life tables an important biometric function of interest is the life expectancy at age $x, M(x)$, defined by

$$M(x) = E[X - x|X > x],$$

where $X$ is a lifetime. $M$ is called the mean residual life function.In many applications it is reasonable to assume that $M$ is decreasing (DMRL) or increasing (IMRL); we write decreasing (increasing) for nonincreasing (non-decreasing). There is some literature on empirical estimators of $M$ and their properties. Although tests for a monotone $M$ are discussed in the literature, we are not aware of any estimators of $M$ under these order restrictions. In this paper we initiate a study of such estimation. Our projection type estimators are shown to be strongly uniformly consistent on compact intervals, and they are shown to be asymptotically “root-$n$” equivalent in probability to the (unrestricted) empirical estimator when $M$ is strictly monotone. Thus the monotonicity is obtained “free of charge”, at least in the aymptotic sense. We also consider the nonparametric maximum likelihood estimators. They do not exist for the IMRL case. They do exist for the DMRL case, but we have found the solutions to be too complex to be evaluated efficiently.

We study the nonparametric estimation of univariate monotone and unimodal densities usingthe maximum smoothed likelihood approach. The monotone estimator is the derivative of the least concave majorant of the distribution correspondingto a kernel estimator.We prove that the mapping on distributions $\Phi$ with density $\varphi$,

$$\varphi \mapsto \text{the derivative of the least concave majorant of $\Phi},$$

is a contraction in all $L^P$ norms $(1 \leq p \leq \infty)$, and some other “distances” such as the Hellinger and Kullback–Leibler distances. The contractivity implies error bounds for monotone density estimation. Almost the same error bounds hold for unimodal estimation.

We discuss in this paper asymptotics of locally optimal solutions of maximum likelihood and,more generally, $M$-estimation procedures in cases where the true value of the parameter vector lies on the boundary of the parameter set $S$.We give a counterexample showing that regularity of $S$ in the sense of Clarke is not sufficient for asymptotic equivalence of $\sqrt{n}$-consistent locally optimal $M$-estimators.We argue further that stronger properties, such as so-called near convexity or prox-regularity of $S$ are required in order to ensure that any two $\sqrt{n}$-consistent locally optimal $M$-estimators have the same asymptotics.