The University of Southampton
University of Southampton Institutional Repository

Exploration and exploitation with insufficient resources

Exploration and exploitation with insufficient resources
Exploration and exploitation with insufficient resources
In physical experimentation, the resources available to discover new knowledge are typically extremely small in comparison to the size and dimensionality of the parameter spaces that can be searched. Additionally, due to the nature of physical experimentation, experimental errors will occur, particularly in biochemical experimentation where the reactants may undetectably denature, or reactant contamination could occur or equipment failure. These errors mean that not all experimental measurements and observations will be accurate or representative of the system being investigated. As the validity of observations is not guaranteed, resources must be split between exploration to discover new knowledge and exploitation to test the validity of the new knowledge. Currently we are investigating the automation of discovery in physical experimentation, with the aim of producing a fully autonomous closed-loop robotic machine capable of autonomous experimentation. This machine will build and evaluate hypotheses, determine experiments to perform and then perform them on an automated lab-on-chip experimentation platform for biochemical response characterisation. In the present work we examine how the trade-off between exploration and exploitation can occur in a situation where the number of experiments that can be performed is extremely small and where the observations returned are sometimes erroneous or unrepresentative of the behaviour being examined. To manage this trade-off we consider the use of a Bayesian notion of surprise, which is used to perform exploration experiments whilst observations are unsurprising from the predictions that can be made and exploits when observations are surprising as they do not match the predicted response
37-61
Lovell, Chris
1ac8eed7-512f-4082-a7ab-75b5e4950518
Jones, Gareth
469d05ca-944e-43cd-91bc-12074c13848e
Zauner, Klaus-Peter
c8b22dbd-10e6-43d8-813b-0766f985cc97
Gunn, Steve R.
306af9b3-a7fa-4381-baf9-5d6a6ec89868
Glowacka, Dorota
8e7748a7-46db-4ba4-8e88-400c5aad045b
Dorard, Louis
a8ef8ed1-81c1-4cea-ac50-fa85f7c0c81b
Shawe-Taylor, John
b1931d97-fdd0-4bc1-89bc-ec01648e928b
Lovell, Chris
1ac8eed7-512f-4082-a7ab-75b5e4950518
Jones, Gareth
469d05ca-944e-43cd-91bc-12074c13848e
Zauner, Klaus-Peter
c8b22dbd-10e6-43d8-813b-0766f985cc97
Gunn, Steve R.
306af9b3-a7fa-4381-baf9-5d6a6ec89868
Glowacka, Dorota
8e7748a7-46db-4ba4-8e88-400c5aad045b
Dorard, Louis
a8ef8ed1-81c1-4cea-ac50-fa85f7c0c81b
Shawe-Taylor, John
b1931d97-fdd0-4bc1-89bc-ec01648e928b

Lovell, Chris, Jones, Gareth, Zauner, Klaus-Peter and Gunn, Steve R. , Glowacka, Dorota, Dorard, Louis and Shawe-Taylor, John (eds.) (2012) Exploration and exploitation with insufficient resources. [in special issue: Proceedings of the Workshop on On-line Trading of Exploration and Exploitation 2, July 2, 2011, Bellevue, Washington, USA] JMLR: Workshop and Conference Proceedings, 26, 37-61.

Record type: Article

Abstract

In physical experimentation, the resources available to discover new knowledge are typically extremely small in comparison to the size and dimensionality of the parameter spaces that can be searched. Additionally, due to the nature of physical experimentation, experimental errors will occur, particularly in biochemical experimentation where the reactants may undetectably denature, or reactant contamination could occur or equipment failure. These errors mean that not all experimental measurements and observations will be accurate or representative of the system being investigated. As the validity of observations is not guaranteed, resources must be split between exploration to discover new knowledge and exploitation to test the validity of the new knowledge. Currently we are investigating the automation of discovery in physical experimentation, with the aim of producing a fully autonomous closed-loop robotic machine capable of autonomous experimentation. This machine will build and evaluate hypotheses, determine experiments to perform and then perform them on an automated lab-on-chip experimentation platform for biochemical response characterisation. In the present work we examine how the trade-off between exploration and exploitation can occur in a situation where the number of experiments that can be performed is extremely small and where the observations returned are sometimes erroneous or unrepresentative of the behaviour being examined. To manage this trade-off we consider the use of a Bayesian notion of surprise, which is used to perform exploration experiments whilst observations are unsurprising from the predictions that can be made and exploits when observations are surprising as they do not match the predicted response

PDF
lovell12a.pdf - Accepted Manuscript
Download (5MB)

More information

e-pub ahead of print date: 2011
Published date: 1 May 2012
Venue - Dates: Workshop on On-line Trading of Exploration and Exploitation 2, United States, 2011-01-01
Organisations: Electronic & Software Systems

Identifiers

Local EPrints ID: 337957
URI: https://eprints.soton.ac.uk/id/eprint/337957
PURE UUID: 62509a6b-5bac-41d4-8711-dec847acb1d3

Catalogue record

Date deposited: 04 May 2012 11:46
Last modified: 18 Jul 2017 05:59

Export record

Contributors

Author: Chris Lovell
Author: Gareth Jones
Author: Klaus-Peter Zauner
Author: Steve R. Gunn
Editor: Dorota Glowacka
Editor: Louis Dorard
Editor: John Shawe-Taylor

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×