The University of Southampton
University of Southampton Institutional Repository

Learning in unknown reward games: application to sensor networks

Learning in unknown reward games: application to sensor networks
Learning in unknown reward games: application to sensor networks
This paper demonstrates a decentralised method for optimisation using game-theoretic multi-agent techniques, applied to a sensor network management problem. Our first major contribution is to show how the marginal contribution utility design is used to construct a unknown-reward potential game formulation of the problem. This formulation exploits the sparse structure of sensor network
problems, and allows us to apply a bound to the price of anarchy of the Nash equilibria of the
induced game. Furthermore, since the game is a potential game, solutions can be found using multiagent
learning techniques. The techniques we derive use Q-learning to estimate an agent’s rewards,
while an action adaptation process responds to an agent’s opponents’ behaviour. However, there are
many different algorithmic configurations that could be used to solve these games. Thus, our second
major contribution is an extensive evaluation of several action adaptation processes. Specifically,
we compare six algorithms across a variety of parameter settings to ascertain the quality of the
solutions they produce, their speed of convergence, and their robustness to pre-specified parameter
choices. Our results show that they each perform similarly across a wide range of parameters.
There is, however, a significant effect from moving to a learning policy with sampling probabilities
that go to zero too quickly for rewards to be accurately estimated.
875-892
Chapman, Archie C.
5a804d19-1e05-4696-bbed-90dd8a1cb52e
Leslie, D
438df1f4-8817-47ad-a4a7-2a4b202759f6
Rogers, Alex
f9130bc6-da32-474e-9fab-6c6cb8077fdc
Jennings, Nicholas R.
ab3d94cc-247c-4545-9d1e-65873d6cdb30
Chapman, Archie C.
5a804d19-1e05-4696-bbed-90dd8a1cb52e
Leslie, D
438df1f4-8817-47ad-a4a7-2a4b202759f6
Rogers, Alex
f9130bc6-da32-474e-9fab-6c6cb8077fdc
Jennings, Nicholas R.
ab3d94cc-247c-4545-9d1e-65873d6cdb30

Chapman, Archie C., Leslie, D, Rogers, Alex and Jennings, Nicholas R. (2014) Learning in unknown reward games: application to sensor networks. The Computer Journal, 57 (6), 875-892.

Record type: Article

Abstract

This paper demonstrates a decentralised method for optimisation using game-theoretic multi-agent techniques, applied to a sensor network management problem. Our first major contribution is to show how the marginal contribution utility design is used to construct a unknown-reward potential game formulation of the problem. This formulation exploits the sparse structure of sensor network
problems, and allows us to apply a bound to the price of anarchy of the Nash equilibria of the
induced game. Furthermore, since the game is a potential game, solutions can be found using multiagent
learning techniques. The techniques we derive use Q-learning to estimate an agent’s rewards,
while an action adaptation process responds to an agent’s opponents’ behaviour. However, there are
many different algorithmic configurations that could be used to solve these games. Thus, our second
major contribution is an extensive evaluation of several action adaptation processes. Specifically,
we compare six algorithms across a variety of parameter settings to ascertain the quality of the
solutions they produce, their speed of convergence, and their robustness to pre-specified parameter
choices. Our results show that they each perform similarly across a wide range of parameters.
There is, however, a significant effect from moving to a learning policy with sampling probabilities
that go to zero too quickly for rewards to be accurately estimated.

Text
CJ_TagetApplication_reSub.pdf - Other
Download (892kB)

More information

Published date: 2014
Organisations: Agents, Interactions & Complexity

Identifiers

Local EPrints ID: 354633
URI: http://eprints.soton.ac.uk/id/eprint/354633
PURE UUID: 0dd3e89a-5335-41fe-94ec-9211d475c744

Catalogue record

Date deposited: 16 Jul 2013 08:47
Last modified: 14 Mar 2024 14:22

Export record

Contributors

Author: Archie C. Chapman
Author: D Leslie
Author: Alex Rogers
Author: Nicholas R. Jennings

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×