Open Access
October 2020 Controlled sequential Monte Carlo
Jeremy Heng, Adrian N. Bishop, George Deligiannidis, Arnaud Doucet
Ann. Statist. 48(5): 2904-2929 (October 2020). DOI: 10.1214/19-AOS1914


Sequential Monte Carlo methods, also known as particle methods, are a popular set of techniques for approximating high-dimensional probability distributions and their normalizing constants. These methods have found numerous applications in statistics and related fields; for example, for inference in nonlinear non-Gaussian state space models, and in complex static models. Like many Monte Carlo sampling schemes, they rely on proposal distributions which crucially impact their performance. We introduce here a class of controlled sequential Monte Carlo algorithms, where the proposal distributions are determined by approximating the solution to an associated optimal control problem using an iterative scheme. This method builds upon a number of existing algorithms in econometrics, physics and statistics for inference in state space models, and generalizes these methods so as to accommodate complex static models. We provide a theoretical analysis concerning the fluctuation and stability of this methodology that also provides insight into the properties of related algorithms. We demonstrate significant gains over state-of-the-art methods at a fixed computational complexity on a variety of applications.


Download Citation

Jeremy Heng. Adrian N. Bishop. George Deligiannidis. Arnaud Doucet. "Controlled sequential Monte Carlo." Ann. Statist. 48 (5) 2904 - 2929, October 2020.


Received: 1 February 2019; Revised: 1 July 2019; Published: October 2020
First available in Project Euclid: 19 September 2020

MathSciNet: MR4152628
Digital Object Identifier: 10.1214/19-AOS1914

Primary: 62M05
Secondary: 62F12 , 62M10

Keywords: annealed importance sampling , approximate dynamic programming , Normalizing constants , optimal control , reinforcement learning , state space models

Rights: Copyright © 2020 Institute of Mathematical Statistics

Vol.48 • No. 5 • October 2020
Back to Top