The Annals of Applied Probability

Universality in polytope phase transitions and message passing algorithms

Mohsen Bayati, Marc Lelarge, and Andrea Montanari

Full-text: Open access

Enhanced PDF (648 KB)

Abstract

We consider a class of nonlinear mappings $\mathsf{F}_{A,N}$ in $\mathbb{R}^{N}$ indexed by symmetric random matrices $A\in\mathbb{R}^{N\times N}$ with independent entries. Within spin glass theory, special cases of these mappings correspond to iterating the TAP equations and were studied by Bolthausen [Comm. Math. Phys. 325 (2014) 333–366]. Within information theory, they are known as “approximate message passing” algorithms.

We study the high-dimensional (large $N$) behavior of the iterates of $\mathsf{F}$ for polynomial functions $\mathsf{F}$, and prove that it is universal; that is, it depends only on the first two moments of the entries of $A$, under a sub-Gaussian tail condition. As an application, we prove the universality of a certain phase transition arising in polytope geometry and compressed sensing. This solves, for a broad class of random projections, a conjecture by David Donoho and Jared Tanner.

Article information

Source
Ann. Appl. Probab., Volume 25, Number 2 (2015), 753-822.

Dates
First available in Project Euclid: 19 February 2015

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1424355130

Digital Object Identifier
doi:10.1214/14-AAP1010

Mathematical Reviews number (MathSciNet)
MR3313755

Zentralblatt MATH identifier
1322.60207

Subjects
Primary: 60F05: Central limit and other weak theorems
Secondary: 68W40: Analysis of algorithms [See also 68Q25]

Keywords
Universality random matrices message passing compressed sensing polytope neighborliness

Citation

Bayati, Mohsen; Lelarge, Marc; Montanari, Andrea. Universality in polytope phase transitions and message passing algorithms. Ann. Appl. Probab. 25 (2015), no. 2, 753--822. doi:10.1214/14-AAP1010. https://projecteuclid.org/euclid.aoap/1424355130


Export citation

References

  • [1] Adamczak, R., Litvak, A. E., Pajor, A. and Tomczak-Jaegermann, N. (2011). Restricted isometry property of matrices with independent columns and neighborly polytopes by random sampling. Constr. Approx. 34 61–88.
    Mathematical Reviews (MathSciNet): MR2796091
    Digital Object Identifier: doi:10.1007/s00365-010-9117-4
  • [2] Affentranger, F. and Schneider, R. (1992). Random projections of regular simplices. Discrete Comput. Geom. 7 219–226.
    Mathematical Reviews (MathSciNet): MR1149653
  • [3] Anderson, G. W., Guionnet, A. and Zeitouni, O. (2010). An Introduction to Random Matrices. Cambridge Studies in Advanced Mathematics 118. Cambridge Univ. Press, Cambridge.
    Mathematical Reviews (MathSciNet): MR2760897
  • [4] Bai, Z. and Silverstein, J. W. (2009). Spectral Analysis of Large Dimensional Random Matrices, 2nd ed. Springer, New York.
    Mathematical Reviews (MathSciNet): MR2567175
  • [5] Bai, Z. D. and Silverstein, J. W. (1998). No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices. Ann. Probab. 26 316–345.
    Mathematical Reviews (MathSciNet): MR1617051
    Digital Object Identifier: doi:10.1214/aop/1022855421
    Project Euclid: euclid.aop/1022855421
  • [6] Bayati, M. and Montanari, A. (2012). The LASSO risk for Gaussian matrices. IEEE Trans. Inform. Theory 58 1997–2017.
    Mathematical Reviews (MathSciNet): MR2951312
    Digital Object Identifier: doi:10.1109/TIT.2011.2174612
  • [7] Bolthausen, E. (2014). An iterative construction of solutions of the TAP equations for the Sherrington–Kirkpatrick model. Comm. Math. Phys. 325 333–366.
    Mathematical Reviews (MathSciNet): MR3147441
    Digital Object Identifier: doi:10.1007/s00220-013-1862-3
  • [8] Bürgisser, P. and Cucker, F. (2010). Smoothed analysis of Moore–Penrose inversion. SIAM J. Matrix Anal. Appl. 31 2769–2783.
    Mathematical Reviews (MathSciNet): MR2740632
    Digital Object Identifier: doi:10.1137/100782954
  • [9] Donoho, D. and Tanner, J. (2009). Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 367 4273–4293.
    Mathematical Reviews (MathSciNet): MR2546388
    Digital Object Identifier: doi:10.1098/rsta.2009.0152
  • [10] Donoho, D. L. (2005). Neighborly polytopes and sparse solution of underdetermined linear equations. Technical report, Statistics Dept., Stanford Univ., Stanford, CA.
  • [11] Donoho, D. L. (2006). High-dimensional centrally symmetric polytopes with neighborliness proportional to dimension. Discrete Comput. Geom. 35 617–652.
    Mathematical Reviews (MathSciNet): MR2225676
  • [12] Donoho, D. L., Javanmard, A. and Montanari, A. (2013). Information-theoretically optimal compressed sensing via spatial coupling and approximate message passing. IEEE Trans. Inform. Theory 59 7434–7464.
    Mathematical Reviews (MathSciNet): MR3124654
    Digital Object Identifier: doi:10.1109/TIT.2013.2274513
  • [13] Donoho, D. L., Johnstone, I. and Montanari, A. (2013). Accurate prediction of phase transitions in compressed sensing via a connection to minimax denoising. IEEE Trans. Inform. Theory 59 3396–3433.
    Mathematical Reviews (MathSciNet): MR3061255
    Digital Object Identifier: doi:10.1109/TIT.2013.2239356
  • [14] Donoho, D. L., Maleki, A. and Montanari, A. (2009). Message passing algorithms for compressed sensing. Proc. Natl. Acad. Sci. USA 106 18914–18919.
  • [15] Donoho, D. L., Maleki, A. and Montanari, A. (2011). The noise-sensitivity phase transition in compressed sensing. IEEE Trans. Inform. Theory 57 6920–6941.
    Mathematical Reviews (MathSciNet): MR2882271
    Digital Object Identifier: doi:10.1109/TIT.2011.2165823
  • [16] Donoho, D. L. and Tanner, J. (2005). Neighborliness of randomly projected simplices in high dimensions. Proc. Natl. Acad. Sci. USA 102 9452–9457 (electronic).
    Mathematical Reviews (MathSciNet): MR2168716
    Digital Object Identifier: doi:10.1073/pnas.0502258102
  • [17] Donoho, D. L. and Tanner, J. (2005). Sparse nonnegative solution of underdetermined linear equations by linear programming. Proc. Natl. Acad. Sci. USA 102 9446–9451 (electronic).
    Mathematical Reviews (MathSciNet): MR2168715
    Digital Object Identifier: doi:10.1073/pnas.0502269102
  • [18] Donoho, D. L. and Tanner, J. (2009). Counting faces of randomly projected polytopes when the projection radically lowers dimension. J. Amer. Math. Soc. 22 1–53.
    Mathematical Reviews (MathSciNet): MR2449053
    Digital Object Identifier: doi:10.1090/S0894-0347-08-00600-0
  • [19] Javanmard, A. and Montanari, A. (2013). State evolution for general approximate message passing algorithms, with applications to spatial coupling. Information and Inference 2 115–144.
    Mathematical Reviews (MathSciNet): MR3311445
    Digital Object Identifier: doi:10.1093/imaiai/iat004
  • [20] Kabashima, Y., Wadayama, T. and Tanaka, T. (2009). A typical reconstruction limit for compressed sensing based on lp-norm minimization. J. Stat. Mech. L09003.
  • [21] Krzakala, F., Mézard, M., Sausset, F., Sun, Y. F. and Zdeborová, L. (2012). Statistical-physics-based reconstruction in compressed sensing. Physical Review X 2 021005.
  • [22] Lubinsky, D. S. (2007). A survey of weighted polynomial approximation with exponential weights. Surv. Approx. Theory 3 1–105.
    Mathematical Reviews (MathSciNet): MR2276420
  • [23] Maleki, A., Anitori, L., Yang, A. and Baraniuk, R. (2011). Asymptotic analysis of complex LASSO via complex approximate message passing (CAMP). Available at arXiv:1108.0477.
    arXiv: 1108.0477
  • [24] Mézard, M., Parisi, G. and Virasoro, M. A. (1987). Spin Glass Theory and Beyond. World Scientific Lecture Notes in Physics 9. World Scientific, Teaneck, NJ.
    Mathematical Reviews (MathSciNet): MR1026102
  • [25] Rangan, S. (2011). Generalized approximate message passing for estimation with random linear mixing. In IEEE Intl. Symp. on Inform. Theory (St. Perersbourg), August 2011. IEEE, Piscataway, NJ.
  • [26] Rangan, S., Fletcher, A. K. and Goyal, V. K. (2009). Asymptotic analysis of map estimation via the replica method and applications to compressed sensing. In Neural Information Processing Systems (NIPS). Vancouver.
  • [27] Schniter, P. (2010). Turbo reconstruction of structured sparse signals. In Proceedings of the Conference on Information Sciences and Systems. Princeton, NJ.
  • [28] Schniter, P. and Rangan, S. (2012). Compressive phase retrieval via generalized approximate message passing. In Communication, Control, and Computing (Allerton), 2012 50th Annual Allerton Conference on 815–822. IEEE, Piscataway, NJ.
  • [29] Tao, T. and Vu, V. (2012). Random matrices: The Universality phenomenon for Wigner ensembles. Available at arXiv:1202.0068.
    arXiv: 1202.0068
  • [30] Thouless, D. J., Anderson, P. W. and Palmer, R. G. (1977). Solution of “solvable model of a spin glass.” Philosophical Magazine 35 593–601.
  • [31] Vershik, A. M. and Sporyshev, P. V. (1992). Asymptotic behavior of the number of faces of random polyhedra and the neighborliness problem. Selecta Math. Soviet. 11 181–201. Selected translations.
    Mathematical Reviews (MathSciNet): MR1166627

The Institute of Mathematical Statistics

The Annals of Applied Probability

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more