Bagging is one of the most effective computationally intensive procedures to improve on unstable estimators or classifiers, useful especially for high dimensional data set problems. Here we formalize the notion of instability and derive theoretical results to analyze the variance reduction effect of bagging (or variants thereof) in mainly hard decision problems, which include estimation after testing in regression and decision trees for regression functions and classifiers. Hard decisions create instability, and bagging is shown to smooth such hard decisions, yielding smaller variance and mean squared error. With theoretical explanations, we motivate subagging based on subsampling as an alternative aggregation scheme. It is computationally cheaper but still shows approximately the same accuracy as bagging. Moreover, our theory reveals improvements in first order and in line with simulation studies.

In particular, we obtain an asymptotic limiting distribution at the cube-root rate for the split point when fitting piecewise constant functions. Denoting sample size by n, it follows that in a cylindric neighborhood of diameter $n^{-1/3}$ of the theoretically optimal split point, the variance and mean squared error reduction of subagging can be characterized analytically. Because of the slow rate, our reasoning also provides an explanation on the global scale for the whole covariate space in a decision tree with finitely many splits.

This paper introduces a class of graphical independence models that is closed under marginalization and conditioning but that contains all DAG independence models. This class of graphs, called maximal ancestral graphs, has two attractive features: there is at most one edge between each pair of vertices; every missing edge corresponds to an independence relation. These features lead to a simple parameterization of the corresponding set of distributions in the Gaussian case.

A new graphical model, called a vine, for dependent random variables is introduced. Vines generalize the Markov trees often used in modelling high-dimensional distributions. They differ from Markov trees and Bayesian belief nets in that the concept of conditional independence is weakened to allow for various forms of conditional dependence.

Vines can be used to specify multivariate distributions in a straightforward way by specifying various marginal distributions and the ways in which these marginals are to be coupled. Such distributions have applications in uncertainty analysis where the objective is to determine the sensitivity of a model output with respect to the uncertainty in unknown parameters. Expert information is frequently elicited to determine some quantitative characteristics of the distribution such as (rank) correlations. We show that it is simple to construct a minimum information vine distribution, given such expert information. Sampling from minimum information distributions with given marginals and (conditional) rank correlations specified on a vine can be performed almost as fast as independent sampling. A special case of the vine construction generalizes work of Joe and allows the construction of a multivariate normal distribution by specifying a set of partial correlations on which there are no restrictions except the obvious one that a correlation lies between $-1$ and 1.

A particular linear group symmetry model, called the dyadic symmetry model, is studied in some detail. Statistical procedures analogous to (multivariate) analysis of variance are introduced. This model may be suitable for various kinds of data collected on pairs of sampling units. Examples include (complete) diallel cross experiments in genetics and social relations analysis in psychology, for which ad hoc methods of analysis have been developed independently in those disciplines.

Our approach is based entirely on formal data structure following the principle of group symmetry, and hence its applicability is not restricted to any specific substantive areas. This paper illustrates the benefits that can be derived from the exploration of mathematical meanings in the development of statistical methods.

This paper analyzes whether standard covariance matrix tests work when dimensionality is large, and in particular larger than sample size. In the latter case, the singularity of the sample covariance matrix makes likelihood ratio tests degenerate, but other tests based on quadratic forms of sample covariance matrix eigenvalues remain well-defined. We study the consistency property and limiting distribution of these tests as dimensionality and sample size go to infinity together, with their ratio converging to a finite nonzero limit. We find that the existing test for sphericity is robust against high dimensionality, but not the test for equality of the covariance matrix to a given matrix. For the latter test, we develop a new correction to the existing test statistic that makes it robust against high dimensionality.

We propose a family of tests, based on Randles' (1989) concept of interdirections and the ranks of pseudo-Mahalanobis distances computed with respect to a multivariate M-estimator of scatter due to Tyler (1987), for the multivariate one-sample problem under elliptical symmetry. These tests, which generalize the univariate signed-rank tests, are affine-invariant. Depending on the score function considered (van der Waerden, Laplace,...), they allow for locally asymptotically maximin tests at selected densities (multivariate normal, multivariate double-exponential,...). Local powers and asymptotic relative efficiencies are derived--with respect to Hotelling's test, Randles' (1989) multivariate sign test, Peters and Randles' (1990) Wilcoxon-type test, and with respect to the Oja median tests. We, moreover, extend to the multivariate setting two famous univariate results: the traditional Chernoff-Savage (1958) property, showing that Hotelling's traditional procedure is uniformly dominated, in the Pitman sense, by the van der Waerden version of our tests, and the celebrated Hodges-Lehmann (1956) ".864 result," providing, for any fixed space dimension $k$, the lower bound for the asymptotic relative efficiency of Wilcoxon-type tests with respect to Hotelling's.

These asymptotic results are confirmed by a Monte Carlo investigation, and application to a real data set.

When the rank of the autoregression matrix is unrestricted, the maximum likelihood estimator under normality is the least squares estimator. When the rank is restricted, the maximum likelihood estimator is composed of the eigenvectors of the effect covariance matrix in the metric of the error covariance matrix corresponding to the largest eigenvalues [Anderson, T. W. (1951). Ann. Math. Statist. 22 327-351]. The asymptotic distribution of these two covariance matrices under normality is obtained and is used to derive the asymptotic distributions of the eigenvectors and eigenvalues under normality. These asymptotic distributions differ from the asymptotic distributions when the regressors are independent variables. The asymptotic distribution of the reduced rank regression is the asymptotic distribution of the least squares estimator with some restrictions; hence the covariance of the reduced rank regression is smaller than that of the least squares estimator. This result does not depend on normality.

In this paper we present Laplace approximations for two functions of matrix argument: the Type I confluent hypergeometric function and the Gauss hypergeometric function. Both of these functions play an important role in distribution theory in multivariate analysis, but from a practical point of view they have proved challenging, and they have acquired a reputation for being difficult to approximate. Appealing features of the approximations we present are: (i) they are fully explicit (and simple to evaluate in practice); and (ii) typically, they have excellent numerical accuracy. The excellent numerical accuracy is demonstrated in the calculation of noncentral moments of Wilks' $\Lambda$ and the likelihood ratio statistic for testing block independence, and in the calculation of the CDF of the noncentral distribution of Wilks' $\Lambda$ via a sequential saddlepoint approximation. Relative error properties of these approximations are also studied, and it is noted that the approximations have uniformly bounded relative errors in important cases.

A Blackwell $\epsilon$-optimal strategy in a Markov Decision Process is a strategy that is $\epsilon$-optimal for every discount factor sufficiently close to 1.

We prove the existence of Blackwell $\epsilon$-optimal strategies in finite Markov Decision Processes with partial observation.

A first general principle and nowadays state of the art for the construction of powerful multiple test procedures controlling a multiple level $\alpha$ is the so-called closure principle. In this article we introduce another powerful tool for the construction of multiple decision procedures, especially for the construction of multiple test procedures and selection procedures. This tool is based on a partition of the parameter space and will be called partitioning principle (PP). In the first part of the paper we review basic concepts of multiple hypotheses testing and discuss a slight generalization of the current theory. In the second part we present various variants of the PP for the construction of multiple test procedures, these are a general PP (GPP), a weak PP (WPP) and a strong PP (SPP). It will be shown that, depending on the underlying decision problem, a PP may lead to more powerful test procedures than a formal application of the closure principle (FCP). Moreover, the more complex SPP may be more powerful than the WPP. Based on a duality between testing and selecting PPs can also be applied for the construction of more powerful selection procedures. In the third part of the paper FCP, WPP and SPP are applied and compared in some examples.

Let $Z$ be a discrete random variable with support $\Z^+ = \{0,1,2,\dots\}$. We consider a Markov chain $Y=(Y_n)_{n=0}^\infty$ with state space $\Z^+$ and transition probabilities given by $P(Y_{n+1} = j|Y_n = i) = P(Z = i+j)/P(Z \geq i)$. We prove that convergence of $\sum_{n=1}^\infty 1/[n^3 P (Z=n)]$ is sufficient for transience of $Y$ while divergence of $\sum_{n=1}^\infty 1/[n^2 P (Z \geq n)]$ is sufficient for recurrence. Let $X$ be a $\mbox{Geometric}(p)$ random variable; that is, $P(X=x)=p(1-p)^x$ for $x \in \Z^+$. We use our results in conjunction with those of M. L. Eaton [Ann. Statist. 20 (1992) 1147-1179] and J. P. Hobert and C. P. Robert [Ann. Statist. 27 (1999) 361-373] to establish a sufficient condition for $\mathscr{P}$-admissibility of improper priors on $p$. As an illustration of this result, we prove that all prior densities of the form $p^{-1}(1-p)^{b-1}$ with $b>0$ are $\mathscr{P}$-admissible.