May 2024 Exact detection thresholds and minimax optimality of Chatterjee’s correlation coefficient
Arnab Auddy, Nabarun Deb, Sagnik Nandy
Author Affiliations +
Bernoulli 30(2): 1640-1668 (May 2024). DOI: 10.3150/23-BEJ1648

Abstract

Recently, Chatterjee (2021) introduced a new rank-based correlation coefficient which can be used to measure the strength of dependence between two random variables. This coefficient has already attracted much attention as it converges to the Dette-Siburg-Stoimenov measure (see Dette et al. (2013)), which equals 0 if and only if the variables are independent and 1 if and only if one variable is a function of the other. Further, Chatterjee’s coefficient is computable in (near) linear time, which makes it appropriate for large-scale applications. In this paper, we expand the theoretical understanding of Chatterjee’s coefficient in two directions: (a) First we consider the problem of testing for independence using Chatterjee’s correlation. We obtain its asymptotic distribution under any changing sequence of alternatives converging to the null hypothesis (of independence). We further obtain a general result that gives exact detection thresholds and limiting power for Chatterjee’s test of independence under natural nonparametric alternatives converging to the null. As applications of this general result, we prove a n14 detection boundary for this test and compute explicitly the limiting local power on the detection boundary for popularly studied alternatives in the literature. (b) We then construct a test for non-trivial levels of dependence using Chatterjee’s coefficient. In contrast to testing for independence, we prove that, in this case, Chatterjee’s coefficient indeed yields a minimax optimal procedure with a n12 detection boundary. Our proof techniques rely on Stein’s method of exchangeable pairs, a non-asymptotic projection result, and information theoretic lower bounds.

Acknowledgements

The authors thank Bhaswar B. Bhattacharya, Sourav Chatterjee, Promit Ghosal, Fang Han, Zhen Huang, and Bodhisattva Sen for their careful reading of the paper and many constructive suggestions. We also thank the Editor, the Associate Editor, and the referees for their helpful comments that greatly improved the quality of the paper.

Authors are in alphabetical order. All authors have equal contribution.

Citation

Download Citation

Arnab Auddy. Nabarun Deb. Sagnik Nandy. "Exact detection thresholds and minimax optimality of Chatterjee’s correlation coefficient." Bernoulli 30 (2) 1640 - 1668, May 2024. https://doi.org/10.3150/23-BEJ1648

Information

Received: 1 December 2022; Published: May 2024
First available in Project Euclid: 31 January 2024

MathSciNet: MR4699567
Digital Object Identifier: 10.3150/23-BEJ1648

Keywords: Independence testing , Kantorovic-Wasserstein distance , Le Cam’s two-point method , local power , Stein’s method for locally dependent structures

JOURNAL ARTICLE
29 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

Vol.30 • No. 2 • May 2024
Back to Top