Analogues of classical limit laws for sums of independent random variables (central limit theorem, laws of large numbers and law of the iterated logarithm) are discussed. We stress results which go through without moment or smoothness assumptions on the underlying distributions. These include (i) estimates for the spread of the distribution of $S_n = \sum_1^nX_i$ in terms of concentration functions (Levy-Rogozin inequality), (ii) comparison of the distribution of $S_n$ on different intervals (ratio limit theorems and Spitzer's theorem for the existence of the potential kernel for recurrent random walk), (iii) study of the set of accumulation points of $S_n/\Upsilon(n)$ for suitable $\Upsilon (n) \uparrow \infty$. Only the following parallel to the law of the iterated logarithm is new: If $X_1, X_2, \cdots$ are independent random variables all with distribution $F, S_n = \sum_1^nX_1, m_n = \operatorname{med} (S_n)$, then there exists a sequence $\{\Upsilon (n)\}$ such that $\Upsilon (n) \rightarrow \infty$ and $- \infty < \lim \inf (S_n - m_n)/\Upsilon (n) < \lim \sup (S_n - m_n)/\Upsilon (n) < \infty$ w.p. 1, if and only if $F$ belongs to the domain of partial attraction of the normal law.

Upper bounds are derived for the expected value of a stopped random sum under each of four sets of assumptions concerning the summands, plus under an additional set describing a related and similar problem. Too complex to abstract, these assumptions in part, typically limit the first two moments of the summands. The bounds have an interpretation in a stock market timing problem in which the random sum represents the sequence of daily prices of a stock and the positive part of the sum reflects a potential profit a holder of an option in the stock could realize were he to exercise. In only one case are the summands required to be independent and identically distributed, and thus we obtain bounds on the expected profit that do not require the controversial random walk model for stock prices. Of course, the bounds are of interest for other reasons as well. For example, as a related result we show that if $(S_n)$ is a random walk for which the summands $(X_n)$ have a negative mean, then $E\lbrack S_T^+\rbrack < \infty$ for all stopping times $T$ if and only if $E\lbrack (X_1^+)^2\rbrack < \infty$. For the most part, techniques familiar to readers of Dubins and Savage (How to Gamble if You Must, McGraw-Hill 1965) are used.

This paper is a direct continuation of the corresponding paper [7], in which our main results were formulated in Section 3. Some of these results were proved and we prepared ourselves for the proof of the asymptotic normality of the sample sum. Our main concern in this part is to carry through the remaining proofs.

The traditional model of statistics is examined in Section 1. The model, as such, distinguishes response values only by their likelihood functions. Recognition of this restricted identification effectively precludes the need for any theory of sufficient statistics. A probability space that has an attached observer-processor-reporter (OPR) mechanism is examined in Section 2 as a means of assessing the nature of reported information; such information may or may not be observational in character depending on properties of the OPR mechanism. A variation-response model is a probability space and a class of random variables: the probability space describes the sources of variation in the system under investigation and the class of random variables describes the possible presentations of this variation in the response of the system. Section 3 examines how realized values on the probability space are distinguished or identified by the model; Section 4 considers how distributions on the probability space are identified by response variable data. In Sections 5, 6, 7 the essentials of three contemporary approaches to inference are presented and each is accompanied by criticisms that proponents of the other methods might make. The key to these criticisms lies primarily in whether hypothetical information is added to or substantiated information is omitted from the assembled information concerning the system under investigation. In certain contexts the bayesian approach makes an arbitrary but typically consistent choice of input to its analyses; it is not the input suggested by standard invariance analysis. In those cases where a variation-response model is appropriate, the use of this more embracing model presents theoretical support for the bayesian choice (Section 8); but of course with the more embracing model the bayesian premises are not needed to obtain the usual bayesian result. An example in Section 9 illustrates the theoretical simplicity of classical probabilities for certain unknowns other than realized values on a probability space.

Let $X_1, X_2, \cdots$ be a sequence of i.i.d. Poisson random variables with mean $\lambda$. It is assumed that true value of the parameter $\lambda$ lies in the set $\{0, 1, 2, \cdots\}$. From observations on the sequence it is desired to estimate the true value of the parameter with a uniformly (for all $\lambda$) small probability of error. There is no fixed sample size rule which can accomplish this. A sequential procedure based on a likelihood ratio criterion is investigated. The procedure, which depends on a parameter $\alpha > 1$, is such that (i) $P_\lambda$ (error) $< 2/(\alpha - 1)$ for all $\lambda$, and (ii) $E_\lambda$ sample size) $\sim k_\lambda \log \alpha$, as $\alpha \rightarrow \infty$, where $k_\lambda = (1 - \lambda \log (1 + 1/\lambda))^{-1}$. The procedure is asymptotically optimal as $\alpha \rightarrow \infty$.

A class of procedures is considered for the subset selection problem when the populations are from a stochastically increasing family $\{F_\lambda\}$. A theorem concerning the monotonicity of an integral associated with $\{F_\lambda\}$ which generalizes an earlier result of Lehmann is obtained. This leads to a sufficient condition for the monotonicity of the probability of a correct selection for the procedure considered. It is shown that this condition is relevant to another sufficient condition for the supremum of the expected subset size to occur when the distributions are identical. The main results are applied to the specific cases where (i) $\lambda$ is a location parameter (ii) $\lambda$ is a scale parameter and (iii) the case where the density $f_\lambda(x)$ is a convex mixture of a sequence of known density functions. The earlier known results are shown to follow from the general theory.

The weak convergence of weighted empirical cumulatives based on the ranks of independent, not necessarily identically distributed, observations to a continuous Gaussian process is proved. The results contain a shorter proof of a central limit theorem by Dupac and Hajek (1969) Ann. Math. Statist. Analogous results are proved for signed rank processes.

This paper deals with two different problems. The first one deals with asymptotic normality of simple linear rank statistics based on random number of observations $X_i$, henceforth called random rank statistics, under the alternative where each $X_i$ has a different distribution $F_i$. The second problem deals with showing that the random rank statistics as a function of the regression parameter in the simple linear regression model is asymptotically uniformly linear (hence continuous) in that parameter. Obviously the two problems are different and could be solved in separate papers but for certain lemmas which are common to the solution of both of these problems. It is suggested not to try to apply the result of Section 3 to Section 4, unless mentioned explicitly. The results of Section 2 are the results which are common, to some extent, to the solution of these two problems. Pyke and Schorack [11] proved asymptotic normality of a class of two sample random rank statistics under two sample alternatives. Our theorem 3.1 could be thought of as a generalization of the result of [11] to more than two samples situation. Our score function $\varphi$ is in smaller class than that of [11]. On the other hand our methods yield the asymptotic normality for random rank-sign statistics. This is also contained in Section 3. Section 2 proves a basic lemma about weak convergence of random weighted empirical cumulatives to a tied down continuous Gaussian process. In [5] and [7] asymptotic uniform linearity of rank statistics based on nonrandom number of observations was proved. In [5] conditions are very general on $\varphi$ and underlying distribution $F$ whereas conditions in [7] are quite stringent. But in [7] we do not need any artificial condition like (2.1) of [5] on underlying regression constants. However in both of these references, regression scores were assumed to be bounded. In Section 4 here we extend the results of [7] to random rank and random rank-sign statistics and to the case where regression scores need not be bounded. In Section 5 we show how the results of Section 4 can be used to construct a bounded length confidence interval for a regression parameter using rank sign statistics with asymptotic (as length $\rightarrow 0$) coverage probability achieved. Apart from applying Theorem 3.1 to the i.i.d. case, as is mentioned in the remark at the end of Section 5, it is hoped that Theorem 3.1 can be found applicable in some other interesting situations.

Hoeffding and Simons (Ann. Math. Statist. (1970) 341-352) show how to convert a sequence of i.i.d. indicator random variables with parameter $p$ into such a sequence with parameter $p = \frac{1}{2}$. In the present paper an extension is made to generating sequences of i.i.d. indicator random variables with parameter $p = \frac{1}{2}$ from arbitrary sequences of i.i.d. discrete random variables. Also considered is the generating of sequences of i.i.d. $r$-valued, equiprobable random variables. Some optimality criteria are established.

We consider procedures for converting input sequences of symbols generated by a stationary random process into sequences of independent, equiprobable output symbols, measuring the efficiency of such a procedure when the input sequence is finite by the expected value of the ratio of output symbols to input symbols. For a large class of processes and a large class of procedures we give an obvious information-theoretic upper bound to efficiency. We also construct procedures which attain this bound in the limit of long input sequences without making use of the process parameters, for two classes of processes. In the independent case we generalize a 1951 result of von Neumann and 1970 results of Hoeffding and Simons for independent but biased binary input, gaining a factor of 3 or 4 in efficiency. In the finite-state case we generalize a 1968 result of Samuelson for two-state binary Markov input, gaining a larger factor in efficiency.

One of two independent Bernoulli processes (arms) with unknown expectations $\rho$ and $\lambda$ is selected and observed at each of $n$ stages. The selection problem is sequential in that the process which is selected at a particular stage is a function of the results of previous selections as well as of prior information about $\rho$ and $\lambda$. The variables $\rho$ and $\lambda$ are assumed to be independent under the (prior) probability distribution. The objective is to maximize the expected number of successes from the $n$ selections. Sufficient conditions for the optimality of selecting one or the other of the arms are given and illustrated for example distributions. The stay-on-a-winner rule is proved.

The traditional model of statistics is a class of probability measures for a response variable. Under reasonable continuity this can be given as a class $C$ of probability density functions relative to an atom-free measure. With a realized value of the response variable, the model $C$ gives the possible probabilities for that realized value--it gives the likelihood function. The likelihood function can be accepted alone or in conjunction with the distribution of possible likelihood functions. In a variety of applications, the variation in a response variable can be traced to a well-defined source having a known probability distribution. The model then is not a class of probability measures but is a single probability measure and a class of random variables. Under moderate conditions this can be given as a probability density function and a class $C_2$ of transformations from the variation space to the response space. And if the distribution for variation is not completely known, the model becomes a class $C_1$ of probability density functions and a class $C_2$ of transformations from the variation space to the response space. With an observed response value, the component $C_2$ identifies a set, the set of possible values for the realized variation. If $C_2$ is a transformation group, then $C_2$ identifies a set--in a partition on the variation space. Standard probability argument using $C_1$ then gives the probability of what has been "observed," and the conditional distribution of what has not been "observed": it gives the likelihood function from the identified set, and the conditional density within the identified set. The likelihood function alone or with its distribution gives the information concerning the parameter of $C_1$; and for any assumed value of that parameter the conditional density gives the information concerning possible values for the realized variation, and accordingly gives the information concerning the parameter of $C_2$, it being what stands between the realized variation and the observed response. The probability of what is identified as having occurred--the likelihood function--is a fundamental output of a model involving density functions. The determination of this probability can however involve certain complexities as soon as the class $C_2$ of random variables is no longer effectively a group. Certainly the class $C_2$ identifies a set on the variation space. But in moderately general cases the range of alternatives can be a partition on the variation space depends on the element of $C_2$. Thus an `event' is identified but the range of possible `events' depends on the parameter of $C_2$. For two kinds of generalized model $(C_1, C_2)$ this paper explores the determination of the probability of what is identified as having occurred--it explores the determination of the likelihood function. In Section 1 the notation and results are summarized for the special model $(C_1, C_2)$ with $C_2$ a transformation group. Two generalizations are examined in Section 2: first, the class $C_2$ is a group but its application as a transformation group has an additional parameter; second, the class $C_2$ is a class of expression transformations $L$ applied to a group of transformations $G$, i.e. $C_2 = LG$. These two generalizations are not as distinct as they may at first appear but they are quite distinct in contexts. The transformed regression model is the central example. Several formulas for volume change in subspaces are recorded in Section 3 and used in Section 4 to make four determinations of likelihood for the generalized model $(C_1, C_2)$. These are applied to the transformed regression model in Section 5 and compared by means of examples in Section 6. The effects of initial variable on the likelihood functions is examined in Section 7 and two compensating routes for analysis are proposed. The class $L$ of expression transformations is examined in Section 8 and shown to be a group under mild consistency conditions. A corresponding invariant likelihood is determined in Section 9, and a transit likelihood in Section 10; the power-transformed regression model is examined in Section 11. In Section 12 the transit likelihood is shown to be the natural likelihood when the semi-direct product $LG$ is itself a group.

The Anscombe and Aumann double relation approach for defining subjective probabilities and utilities in terms of a person's preferences is generalized for the case in which the set of states of the world is unrestricted (finite or not). A monotone continuity condition enables us to prove $\sigma$-additivity; the necessity of this condition is also proved if our other assumptions hold. Although the single relation approach used by Fishburn appears to be more elegant, the present approach has the advantage of showing how the subjective probabilities arise.

For a general discussion of empirical Bayes problems and motivation of the present paper see Section 1 of the previous paper [1]. In that paper we studied the convergence to Bayes optimality and its rate properties for empirical Bayes two-action problems in certain discrete exponential families. This paper continues that investigation for the continuous case. Under appropriate conditions, Theorems 3 and 4 yield convergence rates to Bayes risk of $O(n^{-\beta})$ for $0 < \beta < 1$, for the $(n + 1)\mathrm{st}$ stage risk of the continuous case empirical Bayes procedures of Section 2. These theorems provide, for the continuous case, convergence rate results for the empirical Bayes procedures of the general type considered by Robbins [5] and Samuel [6] for two different parameterizations of a model. The rate results given here in the continuous case involve upper bounds and are weaker than the discrete case results in [1] wherein exact rates are reported. Specifically, in Section 2 we present the two cases to be considered and define the appropriate empirical Bayes procedures for each. Section 3 gives some technical lemmas and Section 4 establishes the asymptotic optimality (the asymptotic Bayes property) of the procedures introduced. The main results on rates, Theorems 3 and 4, are given in Section 5. Section 6 examines in detail two specific examples--the negative exponential and the normal distributions--and gives corollaries to Theorems 3 and 4 which state convergence rates depending on moment properties of the unknown prior distribution of the parameters. Section 7 gives an example with $\beta$ arbitrarily close to 1 in the rate $O(n^{-\beta})$. The model we consider is the following. Let $f_\lambda(x)$ be a family of Lebesgue densities indexed by a parameter $\lambda$ in an interval of the real line. As in [1], we wish to test the hypothesis $H_1: \lambda \leqq c \mathrm{vs}. H_2: \lambda > c$ with the loss function being \begin{align*}L_1(\lambda) &= 0 \quad\text{if}\quad \lambda \leqq c \\ &= b(\lambda - c) \quad\text{if}\quad \lambda > c \\ L_2(\lambda) &= b(c - \lambda) \quad\text{if}\quad \lambda \leqq c \\ &= 0 \quad\text{if}\quad \lambda > c\end{align*} where $L_i(\lambda)$ indicates the loss when action $i$ (deciding in favor of $H_i$) is taken, $i = 1, 2$ and $b$ is a positive constant. Let $\delta(x) = \mathrm{Pr}\{\text{accepting} H_1 \mid X = x\}$ be a randomized decision rule for the above two-action problem. If $G = G(\lambda)$ is a prior distribution on $\lambda$, then the risk of the (randomized) decision procedure $\delta$ under prior distribution $G$ is given as in [1] by, \begin{align*}\tag{1}r(\delta, G) &= \int\int \{L_1(\lambda)f_\lambda(x)\delta(x) + L_2(\lambda)f_\lambda(x)(1 - \delta(x))\} dx dG(\lambda) \\ &= b \int \alpha(x)\delta(x) dx + C_G\end{align*} where $C_G = \int L_2(\lambda) dG(\lambda)$ and \begin{equation*}\tag{2}\alpha(x) = \int \lambda f_\lambda(x) dG(\lambda) - cf(x)\end{equation*} with \begin{equation*}\tag{3}f(x) = \int f_\lambda(x) dG(\lambda).\end{equation*} From (1) it is clear that a Bayes rule (the minimizer of (1) given $G$) is \begin{align*}\tag{4}\delta_G(x) &= 1\quad\text{if} \alpha(x) \leqq 0 \\ &= 0\quad\text{if} \alpha(x) > 0.\end{align*} Hence, the minimal attainable risk knowing $G$ (the Bayes risk) is \begin{equation*}\tag{5}r^\ast(G) = \inf_\delta r(\delta, G) = r(\delta_G, G).\end{equation*}

The properties of both a mixture random process, specified by the multi-dimensional simple mixtures, $\hat{F}(x_1, x_2, \cdots, x_n) = a_1F_1(x_1, x_2, \cdots, x_n) + a_2F_2(x_1, x_2, \cdots, x_n), a_1 + a_2 = 1$, and a related quasi-mixture process are investigated. It is shown that if the set of random variables of the component cdf's (cumulative distribution functions) are independent, then the random variables of the resulting mixture are independent if and only if the mixture cdf $\hat{F}$ is degenerate. The quasi-mixture process, on the other hand, does have the property that factorization of the component cdf's implies factorization of the resulting mixture cdf. Specializing to the case of Gaussian cdf's, it is further shown that the GMP (Gaussian Mixture Process) never satisfies the strong mixing condition, while with reasonable assumptions on the component correlation functions the GQMP (Gaussian Quasi-Mixture Process) does satisfy the strong mixing condition. These, and other properties of the resulting mixture cdf's are of importance when mixture processes are used as models in various estimation and hypothesis testing problems. Some examples are also given for generating GMP and GQMP processes.

We prove the following central limit theorems for sums of mutually dependent random vector variables: Given that a sequence of random vector variables satisfies a certain type of decoupling condition (and two milder restrictions), we present a Lindeberg-Feller condition which we show to be both necessary and sufficient for central limit behavior. The decoupling condition and one of the two milder conditions is then applied to a Markov process with stationary transition mechanism.

For a real, stationary Gaussian process $X(t)$, it is well known that the mean number of zeros of $X(t)$ in a bounded interval is finite exactly when the covariance function $r(t)$ is twice differentiable. Cramer and Leadbetter have shown that the variance of the number of zeros of $X(t)$ in a bounded interval is finite if $(r"(t) - r"(0))/t$ is integrable around the origin. We show that this condition is also necessary. Applying this result, we then answer the question raised by several authors regarding the connection, if any, between the existence of the variance and the existence of continuously differentiable sample paths. We exhibit counterexamples in both directions.

A storage model in which the growth rate is proportional to the level of the dam plus a factor, both proportionality constant and factor depending on a finite Markov chain, is shown to be a Hunt process. Hitting distributions are shown to satisfy certain integral equations, communicating and recurrence classes are studied, and stationary measures are shown to exist when the dam is finite and the level has at least one recurrent linear growth phase, and in some other cases as well.

Let $\mathscr{M} = \{\mu_n: n \geqq 1\}$ be a sequence of probability measures defined on the measurable space $(\mathscr{R}_n, \mathscr{B}_n)$ and suppose that the measures $\{\mu_n: n \geqq 1\}$ satisfy the following condition $(\mathbf{B}): \mathbf{\forall}_\varepsilon > 0, k \geqq 1$ and $m \geqq 1$, there exists an $n \geqq m$ such that $\|\mu_k - \mu_n\| < \varepsilon$. We show that if $A \in \times^\infty_1 \mathscr{B}_n$ and if $A$ is permutation invariant then $\mu(A) = 0$ or 1. The zero-one laws of Hewitt and Savage [Trans. Amer. Math. Soc. 80 (1955) 470-501] and Horn and Schach [Ann. Math. Statist. 41 (1970) 2130-2131] follow as special cases of our result.

In Hou [2] it was shown that the class of approachable sets in a two-person game depended only on the mean vectors of the distributions involved subject to certain moment conditions. In this note we show that this result is, in fact, true whenever the mean vectors exist.

In this paper we present the concept of a weak Banach-valued quasi-martingale. The concept of a strong Banach-valued quasi-martingale was introduced by the author in an earlier paper. We prove that a weak Banach-valued quasi-martingale under certain conditions has a weak Riesz decomposition, i.e., it can be decomposed in an essentially unique way as a sum of a weak martingale and a weak quasi-potential.