We prove an analogue of Blackwell's renewal theorem or the "key renewal theorem" and the existence of the limit distribution of the residual waiting time in the following setup: $X_0, X_1, \cdots$ is a Markov chain with separable metric state space and $u_0, u_1, \cdots$ is a sequence of random variables, such that the conditional distribution of $u_i$, given all $X_j$ and $u_l, l \neq i$, depends on $X_i$ and $X_{i+1}$ only. Here the $V_n \equiv \sum^{n-1}_0 u_i, n \geqq 1$, take the role of the partial sums of independent identically distributed random variables in ordinary renewal theory. E.g. the key renewal theorem in this setup states that $\lim_{t\rightarrow\infty} E\{\sum^\infty_{n=0} g(X_n, t - V_n)\mid X_0 = x\}$ exists for suitable $g(\bullet, \bullet)$, and is independent of $x$.

We establish Berry-Esseen type estimates for random variables with values in a real separable Hilbert space $H$. These estimates are then used to prove the law of the iterated logarithm for sequences of $H$-valued random variables and also to prove a functional form of the law of the iterated logarithm for $H$-valued partial sums as given by Strassen. We also prove a $\log \log$ result for $H$-valued symmetric stable random variables.

In this paper we consider the problem of optimally controlling a diffusion process on a compact interval in one-dimensional Euclidean Space. Under the assumptions that the action space is finite and the cost rate, drift and diffusion coefficients are piecewise analytic, we present a constructive proof that there exist piecewise constant $n$-discount optimal controls for all finite $n \geqq 1$ and measurable $\infty$-discount optimal controls. In addition we present a sequence of second order differential equations that characterize the coefficients of the Laurent series of the expected discounted cost of an $n$-discount optimal control.

It is supposed that in the time interval $(0, \infty)$ a stochastic process is alternately in states $A$ and $B$. Denote by $\alpha_1, \beta_1, \alpha_2, \beta_2, \cdots$ the lengths of the successive intervals spent in states $A$ and $B$ respectively. In this paper the distribution and the asymptotic distribution of the total time spent in state $A(B)$ in the interval $(0, t)$ are determined in the case where $(\alpha_1, \beta_1), (\alpha_2, \beta_2), \cdots$ are mutually independent and identically distributed vector variables.

In this paper, we study analogues of the law of the iterated logarithm for delayed sums of independent random variables. In the i.i.d. case, necessary and sufficient conditions for such analogues are obtained. We apply our results to find convergence rates for expressions of the form $P\lbrack |S_n| > b_n \rbrack$ and $P\lbrack \sup_{k\geqq n} |S_k/b_k| > \varepsilon \rbrack$ for certain upper-class sequences $(b_n)$. In this connection, certain theorems of Erdos, Baum and Katz are also generalized.

A multidimensional branching process with random environments is considered. Two results are proven about this process. The first proves that all nonzero states of the process are transient. Since the process in question is not Markov, the proof of this result is more involved than in the classical case. Our second result deals with the extinction of the process when we are in the critical case. We prove as in the classical theory that extinction occurs $\operatorname{w.p.}$1.

Let $P$ be a conservative Markov operator on $L_\infty(X, \sum, m)$. The following conditions are proved to be equivalent: (i) $P$ is ergodic and quasi-compact. (ii) $P$ is ergodic and $(I - P)L_\infty$ is closed. (iii) For every $u \in L_1$ with $\int u dm = 0$ the sequence $\{\sum^N_{n=0} uP^n\}$ is weakly sequentially compact in $L_1$. (iv) There exists an invariant probability measure $\lambda \sim m$, and for every $u \in L_1$ with $\int u dm = 0, \sup_{N\geqq 0} \|\sum^N_{n=0} uP^n\|_1 < \infty$. These conditions are used to study the quasi-compactness of the induced operators $P_f = \sum^\infty_{n=0} (PT_{1-f})^nPT_f$ in the case that $P$ is Harris-recurrent: The following conditions are equivalent: (i) $P_f$ is quasi-compact. (ii) $f$ is "special" in the (modified) sense of Neveu. (iii) There exists a $\sigma$-finite measure $\lambda$, equivalent to $m$ on $A = \{f > 0\}$, with $\int f d\lambda < \infty$, and $\int fg d\lambda = 0$ implies $\sup_{N\geqq 0} \|\sum^N_{n=0} P^n(fg)\|_\infty < \infty$. Using this characterization certain limit theorems for $P$ are obtained.

Some sets $L$ of sample paths have the desirable property that if there exists a process with given finite-dimensional distributions and with paths in $L$ (with probability 1), then every separable process with these finite-dimensional distributions has paths in $L$. A class of such sets is constructed.

Conditions for tightness and weak convergence of sequences of stochastic processes are given in terms of restrictions on the conditional probabilities of large increments and of large jumps.

The purpose of this note is to show that Neveu's uniform inequality for conditional expectations can be sharpened and extended to arbitrary conditioning sub-$\sigma$-fields. An application of this inequality yields that a sequence of conditional expectations given a $\sigma$-field $\mathscr{F}_n$ converges uniformly for all test functions to a conditional expectation given a $\sigma$-field $\mathscr{F}_\infty$ if and only if the $\sigma$-fields $\mathscr{F}_n$ converge to $\mathscr{F}_\infty$ in the usual metric.

If a $n$-dimensional function is with probability one in a convex set, the same holds true for the conditional expectation (with respect to any sub-$\sigma$-field). An extreme point of this convex set can be assumed by the conditional expectation only if it is assumed by the original function and if this function is partially measurable with respect to the conditioning sub-$\sigma$-field. These results are used to prove Jensen's inequality for conditional expectations of $n$-dimensional functions, and to give a condition for strict inequality.

It is shown that a perfect mixture of perfect probability measures need not be perfect and that a perfect mixture of nonperfect measures can be perfect. Perfect mixtures of discrete measures are shown to be perfect.

We develop a theory for distribution functions which are defined on an abstract linearly ordered space. This extends and gives additional insight into the work of J. H. B. Kemperman who was concerned with a special class of linearly ordered spaces. Attention is given to nonparametric applications. The utility of Kemperman's work on tolerance regions and of related applications to goodness of fit tests, appearing in this paper, is enhanced. In particular, it is shown that a number of standard nonparametric procedures can be extended to cover almost any kind of random sample (e.g., multivariate and most time series data) occurring in practice.

Some further results in the theory of Galton Watson processes are extended to the more general set up of a branching process with random environments. The random distribution function of the limit random variable in the supercritical case (Athreya and Karlin, Ann. Math. Statist., 40 (1969) 743-763) is investigated, and a zero-one law is established. It is shown that this random distribution function is w.p. 1. either absolutely continuous on $(0, \infty)$ with only a jump at the origin or w.p. 1. it is singular. A set of conditions is given under which the former case holds.

Consider $N$ cells into which balls are being dropped independently, in such a way that the cells are equiprobable, and each ball has probability $p_N > 0$ of staying in the cell. Let $W_N(pN, k_N)$ denote the waiting time until $k_N + 1$ cells are occupied, and let $S_N(pN, jN)$ denote the number of distinct cells occupied after $j_N$ balls have been dropped. The full characterization of the limiting distributions of these two random variables is obtained, depending upon the joint behaviour of $p_N, k_N$ and $p_N, j_N$ respectively, as $N \rightarrow \infty$. The limit distributions obtained are the negative binomial, binomial, Poisson, chi-square and normal distributions.

In a recent paper, C. Stein has given a new, direct technique for bounding the error of the normal approximation to the distribution of a sum of dependent random variables, assuming the variables form a stationary sequence with eighth moments. In the present paper we give two $L_1$ bounds on this error for an arbitrary $m$-dependent sequence with second moments.

Let $(\Omega, \mathscr{F})$ be the measurable space consisting of $\Omega$, the set of sequences $(x_1, x_2, \cdots)$ from a finite set $A$, and $\mathscr{F}$, the usual product sigma-field. Let $X_1, X_2, \cdots$ be the usual coordinate random variables defined on $\Omega$. For $n = 1,2, \cdots$, let $\mathscr{F}_n$ be the sub sigma-field of $\mathscr{F}$ generated by $X_1, X_2, \cdots, X_n$. We prove the following: if $P$ is a probability measure on $\mathscr{F}$ stationary with respect to the one-sided shift transformation on $\Omega$ and if $N$ is a positive integer, then there is a periodic measure $Q$ on $\mathscr{F}$ such that $Q = P$ over $\mathscr{F}_N$. This is a stronger result than the known fact that the periodic measures are dense in the set of stationary measures under the weak topology. We also show that if $P$ assigns positive measure to every non-empty set in $\mathscr{F}_N$, it is possible to find an ergodic measure $Q$ such that $P = Q$ over $\mathscr{F}_N$. We investigate the entropies of all such ergodic measures $Q$ which approximate $P$ in this sense, and show that there is a unique ergodic measure $Q$ of maximal entropy such that $P = Q$ over $\mathscr{F}_N$.

It is shown that the numbers of high level crossings by $p$ dependent stationary Gaussian processes have asymptotically independent Poisson distributions if the observation interval and the height of the level increase in a coordinated way. The processes may be highly correlated, even linearly dependent.

This paper contains necessary and sufficient conditions, which sometimes coincide, for the limiting distribution of a uniformly (or strongly) mixing stationary process to be the same as for the independent process with the same marginal distributions. Examples with different limits are given. Let $H$ be any distribution function and let $c_n(\xi) = \inf \{x \in R: H(x) \geqq 1 - \xi/n\}$. The limiting behavior of $H^n(c_n(\xi))$ is determined.

The divergence of the stochastic series $\sum^\infty_{n=1} S_n^+/n$ is investigated, where $S_n^+$ is the positive part of the sum of the first $n$ components of a sequence of independent, identically distributed random variables $\{X_i, i = 1,2, \cdots\}$. It is shown that if $P(X_1 = 0) \neq 1$ then either this series or the companion series $\sum^\infty_{n=1} S_n^-/n$ diverges almost surely. If $EX_1^2 < \infty$ and $EX_1 = 0$ then necessarily both of these series diverge. The method of proof also yields the almost sure divergence of $\sum^\infty_{n=1} S_n/n$. These results are extended to the series $\sum^\infty_{n=1} S_n^+/n^{1+p}$ for $0 \leqq p < \frac{1}{2}$ by a slightly different method of proof which does not, however, yield the divergence of $\sum^\infty_{n=1} S_n/n^{1+p}$.

We prove that Dudley's condition for a.s. convergence of quadratic Brownian variation on a sequence of partitions of $\lbrack 0, 1 \rbrack$ is best possible for the case in which these partitions are restricted to consist of intervals.