The Annals of Statistics

On semidefinite relaxations for the block model

Arash A. Amini and Elizaveta Levina

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

The stochastic block model (SBM) is a popular tool for community detection in networks, but fitting it by maximum likelihood (MLE) involves a computationally infeasible optimization problem. We propose a new semidefinite programming (SDP) solution to the problem of fitting the SBM, derived as a relaxation of the MLE. We put ours and previously proposed SDPs in a unified framework, as relaxations of the MLE over various subclasses of the SBM, which also reveals a connection to the well-known problem of sparse PCA. Our main relaxation, which we call SDP-1, is tighter than other recently proposed SDP relaxations, and thus previously established theoretical guarantees carry over. However, we show that SDP-1 exactly recovers true communities over a wider class of SBMs than those covered by current results. In particular, the assumption of strong assortativity of the SBM, implicit in consistency conditions for previously proposed SDPs, can be relaxed to weak assortativity for our approach, thus significantly broadening the class of SBMs covered by the consistency results. We also show that strong assortativity is indeed a necessary condition for exact recovery for previously proposed SDP approaches and not an artifact of the proofs. Our analysis of SDPs is based on primal-dual witness constructions, which provides some insight into the nature of the solutions of various SDPs. In particular, we show how to combine features from SDP-1 and already available SDPs to achieve the most flexibility in terms of both assortativity and block-size constraints, as our relaxation has the tendency to produce communities of similar sizes. This tendency makes it the ideal tool for fitting network histograms, a method gaining popularity in the graphon estimation literature, as we illustrate on an example of a social networks of dolphins. We also provide empirical evidence that SDPs outperform spectral methods for fitting SBMs with a large number of blocks.

Article information

Source
Ann. Statist., Volume 46, Number 1 (2018), 149-179.

Dates
Received: January 2016
Revised: November 2016
First available in Project Euclid: 22 February 2018

Permanent link to this document
https://projecteuclid.org/euclid.aos/1519268427

Digital Object Identifier
doi:10.1214/17-AOS1545

Mathematical Reviews number (MathSciNet)
MR3766949

Zentralblatt MATH identifier
06865108

Subjects
Primary: 62G20: Asymptotic properties 90C22: Semidefinite programming 62H99: None of the above, but in this section

Keywords
Community detection network semidefinite programming stochastic block model

Citation

Amini, Arash A.; Levina, Elizaveta. On semidefinite relaxations for the block model. Ann. Statist. 46 (2018), no. 1, 149--179. doi:10.1214/17-AOS1545. https://projecteuclid.org/euclid.aos/1519268427


Export citation

References

  • [1] Abbe, E., Bandeira, A. S. and Hall, G. (2016). Exact recovery in the stochastic block model. IEEE Trans. Inform. Theory 62 471–487.
  • [2] Abbe, E. and Sandon, C. (2015). Community detection in general stochastic block models: Fundamental limits and efficient algorithms for recovery. In 2015 IEEE 56th Annual Symposium on Foundations of Computer Science—FOCS 2015 670–688. IEEE Computer Soc., Los Alamitos, CA.
  • [3] Agarwal, N., Bandeira, A. S., Koiliaris, K. and Kolla, A. (2017). Multisection in the stochastic block model using semidefinite programming. In Applied and Numerical Harmonic Analysis (H. Boche, G. Caire, R. Calderbank, et al., eds.) 125–162. Birkhäuser, Cham.
  • [4] Airoldi, E. M., Blei, D. M., Fienberg, S. E. and Xing, E. P. (2008). Mixed membership stochastic blockmodels. J. Mach. Learn. Res. 9 1981–2014.
  • [5] Airoldi, E. M., Costa, T. B. and Chan, S. H. (2013). Stochastic blockmodel approximation of a graphon: Theory and consistent estimation. In Advances in NIPS 26, 692–700.
  • [6] Aldous, D. J. (1981). Representations for partially exchangeable arrays of random variables. J. Multivariate Anal. 11 581–598.
  • [7] Ames, B. P. W. and Vavasis, S. A. (2014). Convex optimization for the planted $k$-disjoint-clique problem. Math. Program. 143 299–337.
  • [8] Amini, A. A., Chen, A., Bickel, P. J. and Levina, E. (2013). Pseudo-likelihood methods for community detection in large sparse networks. Ann. Statist. 41 2097–2122.
  • [9] Amini, A. A. and Levina, E. (2018). Supplement to “On semidefinite relaxations for the block model.” DOI:10.1214/17-AOS1545SUPP.
  • [10] Amini, A. A. and Wainwright, M. J. (2009). High-dimensional analysis of semidefinite relaxations for sparse principal components. Ann. Statist. 37 2877–2921.
  • [11] Awasthi, P., Bandeira, A. S., Charikar, M., Krishnaswamy, R., Villar, S. and Ward, R. (2015). Relax, no need to round: Integrality of clustering formulations. In ITCS’15—Proceedings of the 6th Innovations in Theoretical Computer Science 191–200. ACM, New York.
  • [12] Bandeira, A. S. (2015). Random Laplacian matrices and convex relaxations. Available at arXiv:1504.03987.
  • [13] Bandeira, A. S. (2016). A note on probably certifiably correct algorithms. C. R. Math. Acad. Sci. Paris 354 329–333.
  • [14] Bickel, P., Choi, D., Chang, X. and Zhang, H. (2013). Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels. Ann. Statist. 41 1922–1943.
  • [15] Bickel, P. J. and Chen, A. (2009). A nonparametric view of network models and Newman–Girvan and other modularities. Proc. Natl. Acad. Sci. USA 106 21068–21073.
  • [16] Burer, S. and Monteiro, R. D. C. (2003). A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Math. Program. 95 329–357.
  • [17] Cai, T. T. and Li, X. (2015). Robust and computationally feasible community detection in the presence of arbitrary outlier nodes. Ann. Statist. 43 1027–1059.
  • [18] Celisse, A., Daudin, J.-J. and Pierre, L. (2012). Consistency of maximum-likelihood and variational estimators in the stochastic block model. Electron. J. Stat. 6 1847–1899.
  • [19] Chaudhuri, K., Chung, F. and Tsiatas, A. (2012). Spectral clustering of graphs with general degrees in the extended planted partition model. In JMLR Workshop and Conference Proceedings 23, 35.1–35.23.
  • [20] Chen, Y., Sanghavi, S. and Xu, H. (2014). Improved graph clustering. IEEE Trans. Inform. Theory 60 6440–6455.
  • [21] Chen, Y. and Xu, J. (2016). Statistical-computational tradeoffs in planted problems and submatrix localization with a growing number of clusters and submatrices. J. Mach. Learn. Res. 17 882–938.
  • [22] d’Aspremont, A., El Ghaoui, L., Jordan, M. I. and Lanckriet, G. R. G. (2007). A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49 434–448.
  • [23] Decelle, A., Krzakala, F., Moore, C. and Zdeborová, L. (2011). Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Phys. Rev. E 84 066106.
  • [24] Decelle, A., Krzakala, F., Moore, C. and Zdeborová, L. (2012). Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Phys. Rev. E 84 066106.
  • [25] Feige, U. and Ofek, E. (2005). Spectral techniques applied to sparse random graphs. Random Structures Algorithms 27 251–275.
  • [26] Gao, C., Lu, Y. and Zhou, H. H. (2015). Rate-optimal graphon estimation. Ann. Statist. 43 2624–2652.
  • [27] Guédon, O. and Vershynin, R. (2016). Community detection in sparse networks via Grothendieck’s inequality. Probab. Theory Related Fields 165 1025–1049.
  • [28] Hajek, B., Wu, Y. and Xu, J. (2016). Achieving exact cluster recovery threshold via semidefinite programming. IEEE Trans. Inform. Theory 62 2788–2797.
  • [29] Hajek, B., Wu, Y. and Xu, J. (2016). Achieving exact cluster recovery threshold via semidefinite programming: Extensions. IEEE Trans. Inform. Theory 62 5918–5937.
  • [30] Holland, P. W., Laskey, K. B. and Leinhardt, S. (1983). Stochastic blockmodels: First steps. Soc. Netw. 5 109–137.
  • [31] Javanmard, A., Montanari, A. and Ricci-Tersenghi, F. (2016). Phase transitions in semidefinite relaxations. Proc. Natl. Acad. Sci. USA 113 E2218–E2223.
  • [32] Joseph, A. and Yu, B. (2016). Impact of regularization on spectral clustering. Ann. Statist. 44 1765–1791.
  • [33] Klopp, O., Tsybakov, A. B., Verzelen, N. et al. (2017). Oracle inequalities for network models and sparse graphon estimation. Ann. Statist. 45 316–354.
  • [34] Le, C. M., Levina, E. and Vershynin, R. (2015). Sparse random graphs: Regularization and concentration of the Laplacian. Preprint. Available at arXiv:1502.03049.
  • [35] Lei, J. and Rinaldo, A. (2015). Consistency of spectral clustering in stochastic block models. Ann. Statist. 43 215–237.
  • [36] Lusseau, D. and Newman, M. E. J. (2004). Identifying the role that animals play in their social networks. Proc. R. Soc. Lond., B Biol. Sci. 271 S477–S481.
  • [37] Massoulié, L. (2014). Community detection thresholds and the weak Ramanujan property. In STOC’14—Proceedings of the 2014 ACM Symposium on Theory of Computing 694–703. ACM, New York.
  • [38] Mathieu, C. and Schudy, W. (2010). Correlation clustering with noisy input. In Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms 712–728. SIAM, Philadelphia, PA.
  • [39] Moitra, A., Perry, W. and Wein, A. S. (2016). How robust are reconstruction thresholds for community detection? In STOC’16—Proceedings of the 48th Annual ACM SIGACT Symposium on Theory of Computing 828–841. ACM, New York.
  • [40] Montanari, A. (2016). A Grothendieck-type inequality for local maxima. Preprint. Available at arXiv:1603.04064.
  • [41] Montanari, A. and Sen, S. (2016). Semidefinite programs on sparse random graphs and their application to community detection. In Proceedings of the Forty-Eighth Annual ACM Symposium on Theory of Computing 814–827. ACM, New York.
  • [42] Mossel, E., Neeman, J. and Sly, A. (2012). Stochastic block models and reconstruction. Available at arXiv:1202.1499.
  • [43] Mossel, E., Neeman, J. and Sly, A. (2015). Consistency thresholds for the planted bisection model. In Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing. STOC ’15 69–75. ACM, New York.
  • [44] Mossel, E., Neeman, J. and Sly, A. (2017). A proof of the block model threshold conjecture. Combinatorica. DOI:10.1007/s00493-016-3238-8.
  • [45] Nowicki, K. and Snijders, T. A. B. (2001). Estimation and prediction for stochastic blockstructures. J. Amer. Statist. Assoc. 96 1077–1087.
  • [46] Olhede, S. C. and Wolfe, P. J. (2014). Network histograms and universality of blockmodel approximation. Proc. Natl. Acad. Sci. USA 111 14722–14727.
  • [47] Peng, J. and Wei, Y. (2007). Approximating $K$-means-type clustering via semidefinite programming. SIAM J. Optim. 18 186–205.
  • [48] Perry, A. and Wein, A. S. (2017). A semidefinite program for unbalanced multisection in the stochastic block model. In Sampling Theory and Applications (SampTA), 2017 International Conference on 64–67. IEEE, New York.
  • [49] Qin, T. and Rohe, K. (2013). Regularized spectral clustering under the degree-corrected stochastic blockmodel. In Advances in Neural Information Processing Systems 3120–3128.
  • [50] Rohe, K., Chatterjee, S. and Yu, B. (2011). Spectral clustering and the high-dimensional stochastic blockmodel. Ann. Statist. 39 1878–1915.
  • [51] Snijders, T. A. B. and Nowicki, K. (1997). Estimation and prediction for stochastic blockmodels for graphs with latent block structure. J. Classification 14 75–100.
  • [52] Tomozei, D.-C. and Massoulié, L. (2014). Distributed user profiling via spectral methods. Stoch. Syst. 4 1–43.
  • [53] Vu, V. Q., Cho, J., Lei, J. and Rohe, K. (2013). Fantope projection and selection: A near-optimal convex relaxation of sparse PCA. In Advances in Neural Information Processing Systems 2670–2678.
  • [54] Wainwright, M. J. (2009). Sharp thresholds for high-dimensional and noisy sparsity recovery using $\ell_{1}$-constrained quadratic programming (Lasso). IEEE Trans. Inform. Theory 55 2183–2202.
  • [55] Wolfe, P. J. and Olhede, S. C. (2013). Nonparametric graphon estimation. Preprint. Available at arXiv:1309.5936.
  • [56] Xing, E. P. and Jordan, M. I. (2003). On semidefinite relaxations for normalized $k$-cut and connections to spectral clustering. Technical report, Univ. California, Berkeley.
  • [57] Yan, B. and Sarkar, P. (2016). Convex relaxation for community detection with covariates. Preprint. Available at arXiv:1607.02675.
  • [58] Zhang, Y., Levina, E. and Zhu, J. (2015). Estimating network edge probabilities by neighborhood smoothing. Available at arXiv:1509.08588.
  • [59] Zhao, Y., Levina, E. and Zhu, J. (2012). Consistency of community detection in networks under degree-corrected stochastic block models. Ann. Statist. 40 2266–2292.

Supplemental materials

  • Supplement to “On semidefinite relaxations for the block model”. This supplement contains proofs of some of the results.