• Bernoulli
  • Volume 24, Number 3 (2018), 1636-1652.

Optimal scaling of the independence sampler: Theory and practice

Clement Lee and Peter Neal

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


The independence sampler is one of the most commonly used MCMC algorithms usually as a component of a Metropolis-within-Gibbs algorithm. The common focus for the independence sampler is on the choice of proposal distribution to obtain an as high as possible acceptance rate. In this paper, we have a somewhat different focus concentrating on the use of the independence sampler for updating augmented data in a Bayesian framework where a natural proposal distribution for the independence sampler exists. Thus, we concentrate on the proportion of the augmented data to update to optimise the independence sampler. Generic guidelines for optimising the independence sampler are obtained for independent and identically distributed product densities mirroring findings for the random walk Metropolis algorithm. The generic guidelines are shown to be informative beyond the narrow confines of idealised product densities in two epidemic examples.

Article information

Bernoulli, Volume 24, Number 3 (2018), 1636-1652.

Received: November 2015
Revised: July 2016
First available in Project Euclid: 2 February 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

augmented data birth–death-mutation model Markov jump process MCMC SIR epidemic model


Lee, Clement; Neal, Peter. Optimal scaling of the independence sampler: Theory and practice. Bernoulli 24 (2018), no. 3, 1636--1652. doi:10.3150/16-BEJ908.

Export citation


  • [1] Bailey, N.T.J. (1975). The Mathematical Theory of Infectious Diseases and Its Applications, 2nd ed. London: Griffin.
  • [2] Beskos, A., Pillai, N., Roberts, G., Sanz-Serna, J.-M. and Stuart, A. (2013). Optimal tuning of the hybrid Monte Carlo algorithm. Bernoulli 19 1501–1534.
  • [3] Dellaportas, P. and Roberts, G.O. (2003). An introduction to MCMC. In Spatial Statistics and Computational Methods (Aalborg, 2001) (J. Møller, ed.). Lecture Notes in Statist. 173 1–41. New York: Springer.
  • [4] Del Moral, P., Doucet, A. and Jasra, A. (2012). An adaptive sequential Monte Carlo method for approximate Bayesian computation. Stat. Comput. 22 1009–1020.
  • [5] Ethier, S.N. and Kurtz, T.G. (1986). Markov Processes: Characterization and Convergence. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. New York: Wiley.
  • [6] Fearnhead, P. and Prangle, D. (2012). Constructing summary statistics for approximate Bayesian computation: Semi-automatic approximate Bayesian computation. J. R. Stat. Soc. Ser. B. Stat. Methodol. 74 419–474.
  • [7] Kimura, M. and Crow, J. (1964). The number of alleles that can be maintained in a finite population. Genetics 49 725–738.
  • [8] McKinley, T.J., Ross, J.V., Deardon, R. and Cook, A.R. (2014). Simulation-based Bayesian inference for epidemic models. Comput. Statist. Data Anal. 71 434–447.
  • [9] Neal, P. and Roberts, G. (2005). A case study in non-centering for data augmentation: Stochastic epidemics. Stat. Comput. 15 315–327.
  • [10] Neal, P. and Roberts, G. (2006). Optimal scaling for partially updating MCMC algorithms. Ann. Appl. Probab. 16 475–515.
  • [11] Neal, P., Roberts, G. and Yuen, W.K. (2012). Optimal scaling of random walk Metropolis algorithms with discontinuous target densities. Ann. Appl. Probab. 22 1880–1927.
  • [12] Neal, P. and Terry Huang, C.L. (2015). Forward simulation Markov chain Monte Carlo with applications to stochastic epidemic models. Scand. J. Stat. 42 378–396.
  • [13] O’Neill, P.D. and Becker, N.G. (2001). Inference for an epidemic when susceptibility varies. Biostatistics 2 99–108.
  • [14] O’Neill, P.D. and Roberts, G.O. (1999). Bayesian inference for partially observed stochastic epidemics. J. Roy. Statist. Soc. Ser. A 162 121–129.
  • [15] Papaspiliopoulos, O., Roberts, G.O. and Sköld, M. (2003). Non-centered parameterizations for hierarchical models and data augmentation. In Bayesian Statistics, 7 (Tenerife, 2002) (J.M. Bernardo, M.J. Bayarri, J.O. Berger, A.P. Dawid, D. Heckerman, A.F.M. Smith and M. West, eds.) 307–326. New York: Oxford Univ. Press.
  • [16] Ripley, B.D. (1987). Stochastic Simulation. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. New York: Wiley.
  • [17] Roberts, G.O., Gelman, A. and Gilks, W.R. (1997). Weak convergence and optimal scaling of random walk Metropolis algorithms. Ann. Appl. Probab. 7 110–120.
  • [18] Roberts, G.O. and Rosenthal, J.S. (1998). Optimal scaling of discrete approximations to Langevin diffusions. J. R. Stat. Soc. Ser. B. Stat. Methodol. 60 255–268.
  • [19] Sherlock, C. and Roberts, G. (2009). Optimal scaling of the random walk Metropolis on elliptically symmetric unimodal targets. Bernoulli 15 774–798.
  • [20] Sisson, S.A., Fan, Y. and Tanaka, M.M. (2007). Sequential Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. USA 104 1760–1765 (electronic).
  • [21] Small, P.M., Hopewell, P.C., Singh, S.P., Paz, A., Parsonnet, J., Ruston, D.C., Schecter, G.F., Daley, C.L. and Schoolnik, G.K. (1994). The epidemiology of tuberculosis in San Francisco. A population-based study using conventional and molecular methods. New England Journal of Medicine 330 1703–1709.
  • [22] Tanaka, M.M., Francis, A.R., Luciani, F. and Sisson, S.A. (2006). Using approximate Bayesian computation to estimate tuberculosis transmission parameters from genotype data. Genetics 173 1511–1520.
  • [23] Tierney, L. (1994). Markov chains for exploring posterior distributions. Ann. Statist. 22 1701–1762.
  • [24] Xiang, F. and Neal, P. (2014). Efficient MCMC for temporal epidemics via parameter reduction. Comput. Statist. Data Anal. 80 240–250.