The University of Southampton
University of Southampton Institutional Repository

Approximate Laplace importance sampling for the estimation of expected Shannon information gain in high-dimensional Bayesian design for nonlinear models

Approximate Laplace importance sampling for the estimation of expected Shannon information gain in high-dimensional Bayesian design for nonlinear models
Approximate Laplace importance sampling for the estimation of expected Shannon information gain in high-dimensional Bayesian design for nonlinear models
One of the major challenges in Bayesian optimal design is to approximate the expected utility function in an accurate and computationally efficient manner. We focus on Shannon information gain, one of the most widely used utilities when the experimental goal is parameter inference. We compare the performance of various methods for approximating expected Shannon information gain in common nonlinear models from the statistics literature, with a particular emphasis on Laplace importance sampling (LIS) and approximate Laplace importance sampling (ALIS), a new method that aims to reduce the computational cost of LIS. Specifically, in order to centre the importance distributions LIS requires computation of the posterior mode for each of a large number of simulated possibilities for the response vector. ALIS substantially reduces the amount of numerical optimization that is required, in some cases eliminating all optimization, by centering the importance distributions on the data-generating parameter values wherever possible. Both methods are thoroughly compared with existing approximations including Double Loop Monte Carlo, nested importance sampling, and Laplace approximation. It is found that LIS and ALIS both give an efficient trade-off between mean squared error and computational cost for utility estimation, and ALIS can be up to 70% cheaper than LIS. Usually ALIS gives an approximation that is cheaper but less accurate than LIS, while still being efficient, giving a useful addition to the suite of efficient methods. However, we observed one case where ALIS is both cheaper and more accurate. In addition, for the first time we show that LIS and ALIS yield superior designs to existing methods in problems with large numbers of model parameters when combined with the approximate co-ordinate exchange algorithm for design optimization.
Importance sampling, Monte Carlo, Optimal design
0960-3174
Englezou, Yiolanda
49a1a99a-d7f4-4816-850a-ebcf64f12c9e
Waite, Timothy William
082a8fe9-b30a-4fdf-b392-8344639ae8f6
Woods, David
ae21f7e2-29d9-4f55-98a2-639c5e44c79c
Englezou, Yiolanda
49a1a99a-d7f4-4816-850a-ebcf64f12c9e
Waite, Timothy William
082a8fe9-b30a-4fdf-b392-8344639ae8f6
Woods, David
ae21f7e2-29d9-4f55-98a2-639c5e44c79c

Englezou, Yiolanda, Waite, Timothy William and Woods, David (2022) Approximate Laplace importance sampling for the estimation of expected Shannon information gain in high-dimensional Bayesian design for nonlinear models. Statistics and Computing, 32 (5), [82]. (doi:10.1007/s11222-022-10159-2).

Record type: Article

Abstract

One of the major challenges in Bayesian optimal design is to approximate the expected utility function in an accurate and computationally efficient manner. We focus on Shannon information gain, one of the most widely used utilities when the experimental goal is parameter inference. We compare the performance of various methods for approximating expected Shannon information gain in common nonlinear models from the statistics literature, with a particular emphasis on Laplace importance sampling (LIS) and approximate Laplace importance sampling (ALIS), a new method that aims to reduce the computational cost of LIS. Specifically, in order to centre the importance distributions LIS requires computation of the posterior mode for each of a large number of simulated possibilities for the response vector. ALIS substantially reduces the amount of numerical optimization that is required, in some cases eliminating all optimization, by centering the importance distributions on the data-generating parameter values wherever possible. Both methods are thoroughly compared with existing approximations including Double Loop Monte Carlo, nested importance sampling, and Laplace approximation. It is found that LIS and ALIS both give an efficient trade-off between mean squared error and computational cost for utility estimation, and ALIS can be up to 70% cheaper than LIS. Usually ALIS gives an approximation that is cheaper but less accurate than LIS, while still being efficient, giving a useful addition to the suite of efficient methods. However, we observed one case where ALIS is both cheaper and more accurate. In addition, for the first time we show that LIS and ALIS yield superior designs to existing methods in problems with large numbers of model parameters when combined with the approximate co-ordinate exchange algorithm for design optimization.

Text
ALIS-paper-r1-deblued - Accepted Manuscript
Restricted to Repository staff only until 10 September 2023.
Request a copy
Text
s11222-022-10159-2 - Version of Record
Download (496kB)

More information

Accepted/In Press date: 10 September 2022
e-pub ahead of print date: 30 September 2022
Keywords: Importance sampling, Monte Carlo, Optimal design

Identifiers

Local EPrints ID: 470495
URI: http://eprints.soton.ac.uk/id/eprint/470495
ISSN: 0960-3174
PURE UUID: 12a8a589-b05a-4222-878b-2229e334dd90
ORCID for David Woods: ORCID iD orcid.org/0000-0001-7648-429X

Catalogue record

Date deposited: 11 Oct 2022 17:01
Last modified: 14 Dec 2022 02:37

Export record

Altmetrics

Contributors

Author: Yiolanda Englezou
Author: Timothy William Waite
Author: David Woods ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×