On the storage capacity of Hopfield models with correlated patterns

Abstract

We analyze the storage capacity of the Hopfield model with correlated patterns $(\xi_i^{\nu})$. We treat both the case of semantically and spatially correlated patterns (i.e., the patterns are either correlated in $\nu$ but independent in i or vice versa). We show that the standard Hopfield model of neural networks with N neurons can store $N/(\gamma \log N)$ or $\alpha N$ correlated patterns (depending on which notion of storage is used), provided that the correlation comes from a homogeneous Markov chain. This answers the open question whether the standard Hopfield model can store any increasing number of correlated patterns at all in the affirmative. While our bound on the critical value for $\alpha$ decreases with large correlations, the critical $\gamma$ behaves differently for the different types of correlations.