Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning

Luo, Rui, Wang, Jianhong, Yang, Yaodong, Zhu, Zhanxing and Wang, Jun (2018) Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning. Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N. and Garnett, R. (eds.) In Advances in Neural Information Processing Systems 31 (NeurIPS 2018). vol. 31, Neural Information Processing Systems Foundation. 10 pp .

Abstract

In this paper, we propose a novel sampling method, the thermostat-assisted continuously-tempered Hamiltonian Monte Carlo, for the purpose of multimodal Bayesian learning. It simulates a noisy dynamical system by incorporating both a continuously-varying tempering variable and the Nos\'e-Hoover thermostats. A significant benefit is that it is not only able to efficiently generate i.i.d. samples when the underlying posterior distributions are multimodal, but also capable of adaptively neutralising the noise arising from the use of mini-batches. While the properties of the approach have been studied using synthetic datasets, our experiments on three real datasets have also shown its performance gains over several strong baselines for Bayesian learning with various types of neural networks plunged in.

This record has no associated files available for download.

Contributors

Download statistics