Open Access
February 2024 Maximum likelihood for high-noise group orbit estimation and single-particle cryo-EM
Zhou Fan, Roy R. Lederman, Yi Sun, Tianhao Wang, Sheng Xu
Author Affiliations +
Ann. Statist. 52(1): 52-77 (February 2024). DOI: 10.1214/23-AOS2292


Motivated by applications to single-particle cryo-electron microscopy (cryo-EM), we study several problems of function estimation in a high noise regime, where samples are observed after random rotation and possible linear projection of the function domain. We describe a stratification of the Fisher information eigenvalues according to transcendence degrees of graded pieces of the algebra of group invariants, and we relate critical points of the log-likelihood landscape to a sequence of moment optimization problems, extending previous results for a discrete rotation group without projections.

We then compute the transcendence degrees and forms of these optimization problems for several examples of function estimation under SO(2) and SO(3) rotations, including a simplified model of cryo-EM as introduced by Bandeira, Blum-Smith, Kileel, Niles-Weed, Perry and Wein. We affirmatively resolve conjectures that third-order moments are sufficient to locally identify a generic signal up to its rotational orbit in these examples.

For low-dimensional approximations of the electric potential maps of two small protein molecules, we empirically verify that the noise scalings of the Fisher information eigenvalues conform with our theoretical predictions over a range of SNR, in a model of SO(3) rotations without projections.

Funding Statement

ZF was supported in part by NSF Grants DMS-1916198 and DMS-2142476. RRL was supported in part by NIH/NIGMS 1R01GM136780-01. YS was supported in part by NSF Grants DMS-1701654, DMS-2039183 and DMS-2054838.


We would like to thank Fred Sigworth for helpful discussions about cryo-EM, and for suggesting to us the hemoglobin example. We would also like to thank two anonymous reviewers for their detailed feedback, which has helped us significantly improve the exposition of our manuscript.


Download Citation

Zhou Fan. Roy R. Lederman. Yi Sun. Tianhao Wang. Sheng Xu. "Maximum likelihood for high-noise group orbit estimation and single-particle cryo-EM." Ann. Statist. 52 (1) 52 - 77, February 2024.


Received: 1 July 2021; Revised: 1 May 2023; Published: February 2024
First available in Project Euclid: 7 March 2024

MathSciNet: MR4718407
Digital Object Identifier: 10.1214/23-AOS2292

Primary: 62G08

Keywords: group invariance , Mixture models , nonparametric function estimation

Rights: Copyright © 2024 Institute of Mathematical Statistics

Vol.52 • No. 1 • February 2024
Back to Top