Total positivity in exponential families with application to binary variables

Abstract

We study exponential families of distributions that are multivariate totally positive of order 2 (${\mathrm{MTP}}_2$), show that these are convex exponential families and derive conditions for existence of the MLE. Quadratic exponential familes of ${\mathrm{MTP}}_2$ distributions contain attractive Gaussian graphical models and ferromagnetic Ising models as special examples. We show that these are defined by intersecting the space of canonical parameters with a polyhedral cone whose faces correspond to conditional independence relations. Hence ${\mathrm{MTP}}_2$ serves as an implicit regularizer for quadratic exponential families and leads to sparsity in the estimated graphical model. We prove that the maximum likelihood estimator (MLE) in an ${\mathrm{MTP}}_2$ binary exponential family exists if and only if both of the sign patterns $(1,-1)$ and $(-1,1)$ are represented in the sample for every pair of variables; in particular, this implies that the MLE may exist with $\mathit{n}=\mathit{d}$ observations, in stark contrast to unrestricted binary exponential families where $2^{\mathit{d}}$ observations are required. Finally, we provide a novel and globally convergent algorithm for computing the MLE for ${\mathrm{MTP}}_2$ Ising models similar to iterative proportional scaling and apply it to the analysis of data from two psychological disorders.