Prequential probability: principles and properties

Abstract

Forecaster has to predict, sequentially, a string of uncertain quantities $(X_1,X_2,...)$, whose values are determined and revealed, one by one, by Nature. Various criteria may be proposed to assess Forecaster's empirical performance. The weak prequential principle requires that such a criterion should depend on Forecaster's behaviour or strategy only through the actual forecasts issued. A wide variety of appealing criteria are shown to respect this principle. We further show that many such criteria also obey the strong prequential principle, which requires that, when both Nature and Forecaster make their choices in accordance with a common joint distribution $P$ for $(X_1,X_2,...)$, certain stochastic properties, underlying and justifying the criterion and inferences based on it, hold regardless of the detailed specification of $P$. In order to understand further this compliant behaviour, we introduce the prequential framework, a game-theoretic basis for probability theory in which it is impossible to violate the prequential principles, and we describe its connections with classical probability theory. In this framework, in order to show that some criterion for assessing Forecaster's empirical performance is valid, we have to exhibit a winning strategy for a third player, Statistician, in a certain perfect-information game. We demonstrate that many performance criteria can be formulated and are valid in the framework and, therefore, satisfy both prequential principles.