Abstract
We study the free energy of a most used deep architecture for restricted Boltzmann machines, where the layers are disposed in series. Assuming independent Gaussian distributed random weights, we show that the error term in the so-called replica symmetric sum rule can be optimised as a saddle point. This leads us to conjecture that in the replica symmetric approximation the free energy is given by a formula, which parallels the one achieved for two-layer case.
Acknowledgments
This paper benefited greatly from the observations of an anonymous referee, who is gratefully acknowledged.
Citation
Giuseppe Genovese. "Minimax formula for the replica symmetric free energy of deep restricted Boltzmann machines." Ann. Appl. Probab. 33 (3) 2324 - 2341, June 2023. https://doi.org/10.1214/22-AAP1868
Information