Proper scoring rules and Bregman divergence

Abstract

Proper scoring rules measure the quality of probabilistic forecasts. They induce dissimilarity measures of probability distributions known as Bregman divergences. We survey the literature on both entities and present their mathematical properties in a unified theoretical framework. Score and Bregman divergences are developed as a single concept. We formalize the proper affine scoring rules and present a motivating example from robust estimation. And lastly, we develop the elements of the regularity theory of entropy functions and describe under what conditions a general convex function may be identified as the entropy function of a proper scoring rule and whether this association is unique.