A Probabilistic View on Predictive Constructions for Bayesian Learning

Abstract

Given a sequence $\mathit{X}=({\mathit{X}}_1,{\mathit{X}}_2,\dots )$ of random observations, a Bayesian forecaster aims to predict ${\mathit{X}}_{\mathit{n}+1}$ based on $({\mathit{X}}_1,\dots ,{\mathit{X}}_{\mathit{n}})$ for each $\mathit{n}\ge 0$. To this end, in principle, she only needs to select a collection $\mathit{\sigma }=({\mathit{\sigma }}_0,{\mathit{\sigma }}_1,\dots )$, called “strategy” in what follows, where ${\mathit{\sigma }}_0(·)=\mathit{P}({\mathit{X}}_1\in ·)$ is the marginal distribution of ${\mathit{X}}_1$ and ${\mathit{\sigma }}_{\mathit{n}}(·)=\mathit{P}({\mathit{X}}_{\mathit{n}+1}\in ·|{\mathit{X}}_1,\dots ,{\mathit{X}}_{\mathit{n}})$ the nth predictive distribution. Because of the Ionescu–Tulcea theorem, σ can be assigned directly, without passing through the usual prior/posterior scheme. One main advantage is that no prior probability is to be selected. In a nutshell, this is the predictive approach to Bayesian learning. A concise review of the latter is provided in this paper. We try to put such an approach in the right framework, to make clear a few misunderstandings, and to provide a unifying view. Some recent results are discussed as well. In addition, some new strategies are introduced and the corresponding distribution of the data sequence X is determined. The strategies concern generalized Pólya urns, random change points, covariates and stationary sequences.