The Distortion-Rate Function for Nonergodic Sources

Abstract

The distortion rate function $D(R)$ is defined as an infimum of distortion with respect to a mutual information constraint. The usual coding theorems assert that, for ergodic souces, $D(R)$ is equal to $\delta(R)$, the least distortion attainable by block codes of rate $R$. If a source has ergodic components $\{\theta\}$ with weighting measure $dw(\theta)$, it has been shown by Gray and Davisson that $\delta(R)$ is the integral of the components $\delta_\theta(R)$ with respect to $dw(\theta)$. We show that $D(R)$ is the infimum of the integrals of $D_\theta(R_\theta)$ where the integral of $R_\theta$ is $R$. Our method of proof also gives a formula for the $\bar{d}$-distance in terms of ergodic components and shows that $D(R) = D'(R)$, which is defined as the infimum of distortion subject to an entropy constraint.