December 2023 Adjusted chi-square test for degree-corrected block models
Linfan Zhang, Arash A. Amini
Author Affiliations +
Ann. Statist. 51(6): 2366-2385 (December 2023). DOI: 10.1214/23-AOS2329

Abstract

We propose a goodness-of-fit test for degree-corrected stochastic block models (DCSBM). The test is based on an adjusted chi-square statistic for measuring equality of means among groups of n multinomial distributions with d1,,dn observations. In the context of network models, the number of multinomials, n, grows much faster than the number of observations, di, corresponding to the degree of node i, hence the setting deviates from classical asymptotics. We show that a simple adjustment allows the statistic to converge in distribution, under null, as long as the harmonic mean of {di} grows to infinity. When applied sequentially, the test can also be used to determine the number of communities. The test operates on a compressed version of the adjacency matrix, conditional on the degrees, and as a result is highly scalable to large sparse networks. We incorporate a novel idea of compressing the rows based on a (K+1)-community assignment when testing for K communities. This approach increases the power in sequential applications without sacrificing computational efficiency, and we prove its consistency in recovering the number of communities. Since the test statistic does not rely on a specific alternative, its utility goes beyond sequential testing and can be used to simultaneously test against a wide range of alternatives outside the DCSBM family. In particular, we prove that the test is consistent against a general family of latent-variable network models with community structure. We show the effectiveness of the approach by extensive numerical experiments with simulated and real data. In particular, applying the test to the Facebook-100 data set, a collection of one hundred social networks, we find that a DCSBM with a small number of communities (say <25) is far from a good fit in almost all cases.

Funding Statement

This work was supported by NSF Grant DMS-1945667.

Acknowledgments

We thank Mason Porter who provided access to the Facebook-100 data set.

Citation

Download Citation

Linfan Zhang. Arash A. Amini. "Adjusted chi-square test for degree-corrected block models." Ann. Statist. 51 (6) 2366 - 2385, December 2023. https://doi.org/10.1214/23-AOS2329

Information

Received: 1 January 2021; Revised: 1 September 2023; Published: December 2023
First available in Project Euclid: 20 December 2023

MathSciNet: MR4682701
zbMATH: 07783619
Digital Object Identifier: 10.1214/23-AOS2329

Subjects:
Primary: 62E17 , 62G20 , 62H99

Keywords: adjusted chi-square statistic , Community detection , degree-corrected stochastic block model , Goodness-of-fit test , nonasymptotic

Rights: Copyright © 2023 Institute of Mathematical Statistics

JOURNAL ARTICLE
20 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

Vol.51 • No. 6 • December 2023
Back to Top