The Annals of Statistics
- Ann. Statist.
- Volume 46, Number 5 (2018), 2020-2044.
Change-point detection in multinomial data with a large number of categories
We consider a sequence of multinomial data for which the probabilities associated with the categories are subject to abrupt changes of unknown magnitudes at unknown locations. When the number of categories is comparable to or even larger than the number of subjects allocated to these categories, conventional methods such as the classical Pearson’s chi-squared test and the deviance test may not work well. Motivated by high-dimensional homogeneity tests, we propose a novel change-point detection procedure that allows the number of categories to tend to infinity. The null distribution of our test statistic is asymptotically normal and the test performs well with finite samples. The number of change-points is determined by minimizing a penalized objective function based on segmentation, and the locations of the change-points are estimated by minimizing the objective function with the dynamic programming algorithm. Under some mild conditions, the consistency of the estimators of multiple change-points is established. Simulation studies show that the proposed method performs satisfactorily for identifying change-points in terms of power and estimation accuracy, and it is illustrated with an analysis of a real data set.
Ann. Statist., Volume 46, Number 5 (2018), 2020-2044.
Received: December 2016
Revised: July 2017
First available in Project Euclid: 17 August 2018
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Wang, Guanghui; Zou, Changliang; Yin, Guosheng. Change-point detection in multinomial data with a large number of categories. Ann. Statist. 46 (2018), no. 5, 2020--2044. doi:10.1214/17-AOS1610. https://projecteuclid.org/euclid.aos/1534492827
- Supplement to “Change-point detection in multinomial data with a large number of categories”. The Supplementary Material contains all theoretical proofs of Theorems 1–5, Proposition 1 and Corollary 1 and additional simulation results.