Markov chain Monte Carlo methods have been increasingly popular since their introduction by Gelfand and Smith. However, while the breadth and variety of Markov chain Monte Carlo applications are properly astounding, progress in the control of convergence for these algorithms has been slow, despite its relevance in practical implementations. We present here different approaches toward this goal based on functional and mixing theories, while paying particular attention to the central limit theorem and to the approximation of the limiting variance. Renewal theory in the spirit of Mykland, Tierney and Yu is presented as the most promising technique in this regard, and we illustrate its potential in several examples. In addition, we stress that many strong convergence properties can be derived from the study of simple sub-chains which are produced by Markov chain Monte Carlo algorithms, due to a duality principle obtained in Diebolt and Robert for mixture estimation. We show here the generality of this principle which applies, for instance, to most missing data models. A more empirical stopping rule for Markov chain Monte Carlo algorithms is related to the simultaneous convergence of different estimators of the quantity of interest. Besides the regular ergodic average, we propose the Rao-Blackwellized version as well as estimates based on importance sampling and trapezoidal approximations of the integrals.

This paper is a survey of the major techniques and approaches available for the numerical approximation of integrals in statistics. We classify these into five broad categories; namely, asymptotic methods, importance sampling, adaptive importance sampling, multiple quadrature and Markov chain methods. Each method is discussed, giving an outline of the basic supporting theory and particular features of the technique. Conclusions are drawn concerning the relative merits of the methods based on the discussion and their application to three examples. The following broad recommendations are made. Asymptotic methods should only be considered in contexts where the integrand has a dominant peak with approximate ellipsoidal symmetry. Importance sampling (and preferably adaptive importance sampling) based on a multivariate Student should be used instead of asymptotics methods in such a context. Multiple quadrature and, in particular, subregion adaptive integration are the algorithms of choice for lower-dimensional integrals. Due to the difficulties in assessing convergence to stationarity and the error in estimates, Markov chain methods are recommended only when there is no adequate alternative. In certain very high dimensional problems, however, Markov chain methods are the only hope. The importance of the parameterization of the integral is noted for the success of all the methods, and several useful reparameterizations are presented.

This paper reviews the literature on Bayesian experimental design. A unified view of this topic is presented, based on a decision-theoretic approach. This framework casts criteria from the Bayesian literature of design as part of a single coherent approach. The decision-theoretic structure incorporates both linear and nonlinear design problems and it suggests possible new directions to the experimental design problem, motivated by the use of new utility functions. We show that, in some special cases of linear design problems, Bayesian solutions change in a sensible way when the prior distribution and the utility function are modified to allow for the specific structure of the experiment. The decision-theoretic approach also gives a mathematical justification for selecting the appropriate optimality criterion.