Extensive scoring rules

Abstract

Scoring rules evaluate the performance of probabilistic forecasts. A scoring rule is said to be local if it assigns a score based on the observed outcome and on outcomes that are in some sense “close” to the observed outcome. All scoring rules can be derived from a concave entropy functional and the property of locality follows when the entropy is 1-homogeneous (up to an additive constant). Consequently, except for the log score, a local scoring rule has the remarkable property that it is 0-homogeneous; in other words, it assigns a score that is independent of the normalization of the quoted probability distribution. In many statistical applications, it is not plausible to treat observed outcomes as independent, e.g. time series data or multicomponent measurements. We show that local scoring rules can be easily extended to multidimensional outcome spaces. We also introduce the notion of an extensive scoring rule, i.e. a scoring rule that ensures the score of independent outcomes is a sum of independent scores. We construct local scoring rules that are extensive and show that a scoring rule is a extensive if and only if it is derived from an extensive entropy.