The Annals of Statistics

Improving SAMC using smoothing methods: Theory and applications to Bayesian model selection problems

Faming Liang

Full-text: Open access


Stochastic approximation Monte Carlo (SAMC) has recently been proposed by Liang, Liu and Carroll [J. Amer. Statist. Assoc. 102 (2007) 305–320] as a general simulation and optimization algorithm. In this paper, we propose to improve its convergence using smoothing methods and discuss the application of the new algorithm to Bayesian model selection problems. The new algorithm is tested through a change-point identification example. The numerical results indicate that the new algorithm can outperform SAMC and reversible jump MCMC significantly for the model selection problems. The new algorithm represents a general form of the stochastic approximation Markov chain Monte Carlo algorithm. It allows multiple samples to be generated at each iteration, and a bias term to be included in the parameter updating step. A rigorous proof for the convergence of the general algorithm is established under verifiable conditions. This paper also provides a framework on how to improve efficiency of Monte Carlo simulations by incorporating some nonparametric techniques.

Article information

Ann. Statist., Volume 37, Number 5B (2009), 2626-2654.

First available in Project Euclid: 17 July 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60J22: Computational methods in Markov chains [See also 65C40] 65C05: Monte Carlo methods

Model selection Markov chain Monte Carlo reversible jump smoothing stochastic approximation Monte Carlo


Liang, Faming. Improving SAMC using smoothing methods: Theory and applications to Bayesian model selection problems. Ann. Statist. 37 (2009), no. 5B, 2626--2654. doi:10.1214/07-AOS577.

Export citation


  • Aerts, M., Augustyns, I. and Janssen, P. (1997). Smoothing sparse multinomial data using local polynomial fitting. J. Nonparametr. Statist. 8 127–147.
  • Andrieu, C., Moulines, É. and Priouret, P. (2005). Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optim. 44 283–312.
  • Atchadé, Y. F. and Liu, J. S. (2007). The Wang–Landau algorithm in general state spaces: Applications and convergence analysis. Technical report, Dept. Statistics, Univ. Michigan.
  • Barry, D. and Hartigan, J. A. (1993). A Bayesian analysis for change point problems. J. Amer. Statist. Assoc. 88 309–319.
  • Benveniste, A., Métivier, M. and Priouret, P. (1990). Adaptive Algorithms and Stochastic Approximations. Springer, New York.
  • Berg, B. A. and Neuhaus, T. (1991). Multicanonical algorithms for 1st order phase-transitions. Phys. Lett. B 267 249–253.
  • Burman, P. (1987). Smoothing sparse contingency tables. Sankhyā Ser. A 49 24–36.
  • Chen, H.-F. (2002). Stochastic Approximation and Its Applications. Kluwer Academic Publishers, Boston.
  • Cheon, S. Y. and Liang, F. (2007). Phylogenetic tree reconstruction using stochastic approximation Monte Carlo. BioSystems. To appear.
  • Dong, J. and Simonoff, J. S. (1994). The construction and properties of boundary kernels for smoothing sparse multinomials. J. Comput. Graph. Statist. 3 57–66.
  • Fan, J., Heckman, N. E. and Wand, M. P. (1995). Local polynomial kernel regression for generalized linear models and quasi-likelihood functions. J. Amer. Statist. Assoc. 90 141–150.
  • Folland, G. B. (1990). Remainder estimates in Taylor’s theorem. American Math. Monthly 97 233–235.
  • Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Machine Intelligence 6 721–741.
  • Geyer, C. J. (1991). Markov chain Monte Carlo maximum likelihood. In Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface (E. M. Keramigas, ed.) 156–163. Interface Foundation, Fairfax.
  • Geyer, C. J. and Thompson, E. A. (1995). Annealing Markov chain Monte Carlo with applications to ancestral inference. J. Amer. Statist. Assoc. 90 909–920.
  • Gilks, W. R., Roberts, R. O. and Sahu, S. K. (1998). Adaptive Markov chain Monte Carlo through regeneration. J. Amer. Statist. Assoc. 93 1045–1054.
  • Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82 711–732.
  • Gu, M. G. and Kong, F. H. (1998). A stochastic approximation algorithm with Markov chain Monte Carlo method for incomplete data estimation problems. Proc. Natl. Acad. Sci. USA 95 7270–7274.
  • Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Application. Academic Press, New York.
  • Hall, P. and Titterington, D. M. (1987). On smoothing sparse multinomial data. Austral. J. Statist. 29 19–37.
  • Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57 97–109.
  • Hesselbo, B. and Stinchcomble, R. B. (1995). Monte Carlo simulation and global optimization without parameters. Phys. Rev. Lett. 74 2151–2155.
  • Hukushima K. and Nemoto, K. (1996). Exchange Monte Carlo method and application to spin glass simulations. J. Phys. Soc. Jpn. 65 1604–1608.
  • Kou, S. C., Zhou, Q. and Wong, W. H. (2006). Equi-energy sampler with applications to statistical inference and statistical mechanics (with discussion). Ann. Statist. 34 1581–1619.
  • Kushner, H. J. and Ying, G. G. (2003). Stochastic Approximation and Recursive Algorithms and Applications. Springer, New York.
  • Liang, F. (2005). Generalized Wang–Landau algorithm for Monte Carlo computation. J. Amer. Statist. Assoc. 100 1311–1327.
  • Liang, F. (2007). Annealing stochastic approximation Monte Carlo for neural network training. Machine Learning 68 201–233.
  • Liang, F., Liu, C. and Carroll, R. J. (2007). Stochastic approximation in Monte Carlo computation. J. Amer. Statist. Assoc. 102 305–320.
  • Liang, F. and Wong, W. H. (2000). Evolutionary Monte Carlo sampling: Applications to Cp model sampling and change-point problem. Statist. Sinica 10 317–342.
  • Liang, F. and Wong, W. H. (2001). Real parameter evolutionary Monte Carlo with applications in Bayesian mixture models. J. Amer. Statist. Assoc. 96 653–666.
  • Marinari, E. and Parisi, G. (1992). Simulated tempering: A new Monte Carlo scheme. Europhys. Lett. 19 451–458.
  • Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953). Equation of state calculations by fast computing machines. J. Chem. Phys. 21 1087–1091.
  • Mitsutake, A., Sugita, Y. and Okamoto, Y. (2003). Replica-exchange multicanonical and multicanonical replica exchange Monte Carlo simulations of peptides. I. Formula and benchmark test. J. Chem. Phys. 118 6676–6688.
  • Phillips, D. B. and Smith, A. F. M. (1996). Bayesian model comparison via jump diffusions. In Markov Chain Monte Carlo in Practice (W. R. Gilks, S. Richardson and D. J. Spiegelhalter, eds.) 215–239. Chapman and Hall, London.
  • Roberts, G. O. and Tweedie, R. L. (1996). Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms. Biometrika 83 95–110.
  • Rosenthal, J. S. (1995). Minorization conditions and convergence rate for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 90 558–566.
  • Simonoff, J. S. (1998). Three sides of smoothing: categorical data smoothing, nonparametric regression, and density estimation. Internat. Statist. Rev. 66 137–156.
  • Tadić, V. (1997). On the convergence of stochastic iterative algorithms and their applications to machine learning. A short version of this paper was published in Proc. 36th Conf. on Decision and Control 2281–2286. San Diego, USA.
  • Tibshirani, R. and Hastie, T. (1987). Local likelihood estimation. J. Amer. Statist. Assoc. 82 559–568.
  • Venables, W. N. and Ripley, B. D. (1999). Modern Applied Statistics with S-PLUS, 3rd ed. Springer, New York.
  • Wang, F. and Landau, D. P. (2001). Efficient, multiple-range random walk algorithm to calculate the density of states. Phys. Rev. Lett. 86 2050–2053.
  • Wong, W. H. and Liang, F. (1997). Dynamic weighting in Monte Carlo and optimization. Proc. Natl. Acad. Sci. 94 14220–14224.
  • Younes, L. (1999). On the converges of Markovian stochastic algorithms with rapidly decreasing ergodicity rates. Stochastic Stochastic Rep. 65 177–228.