The University of Southampton
University of Southampton Institutional Repository

Reward Shaping for Valuing Communications During Multi-Agent Coordination

Reward Shaping for Valuing Communications During Multi-Agent Coordination
Reward Shaping for Valuing Communications During Multi-Agent Coordination
Decentralised coordination in multi-agent systems is typically achieved using communication. However, in many cases, communication is expensive to utilise because there is limited bandwidth, it may be dangerous to communicate, or communication may simply be unavailable at times. In this context, we argue for a rational approach to communication - if it has a cost, the agents should be able to calculate a value of communicating. By doing this, the agents can balance the need to communicate with the cost of doing so. In this research, we present a novel model of rational communication, that uses reward shaping to value communications, and employ this valuation in decentralised POMDP policy generation. In this context, reward shaping is the process by which expectations over joint actions are adjusted based on how coordinated the agent team is. An empirical evaluation of the benefits of this approach is presented in two domains. First, in the context of an idealised benchmark problem, the multiagent Tiger problem, our method is shown to require significantly less communication (up to 30% fewer messages) and still achieves a 30% performance improvement over the current state of the art. Second, in the context of a larger-scale problem, RoboCupRescue, our method is shown to scale well, and operate without recourse to significant amounts of domain knowledge.
641-648
Williamson, Simon A.
be7675ba-be67-4a69-98d2-32381c0cce90
Gerding, Enrico H.
d9e92ee5-1a8c-4467-a689-8363e7743362
Jennings, Nicholas R.
ab3d94cc-247c-4545-9d1e-65873d6cdb30
Williamson, Simon A.
be7675ba-be67-4a69-98d2-32381c0cce90
Gerding, Enrico H.
d9e92ee5-1a8c-4467-a689-8363e7743362
Jennings, Nicholas R.
ab3d94cc-247c-4545-9d1e-65873d6cdb30

Williamson, Simon A., Gerding, Enrico H. and Jennings, Nicholas R. (2009) Reward Shaping for Valuing Communications During Multi-Agent Coordination. Autonomous Agents And MultiAgent Systems, Budapest, Hungary. pp. 641-648 .

Record type: Conference or Workshop Item (Paper)

Abstract

Decentralised coordination in multi-agent systems is typically achieved using communication. However, in many cases, communication is expensive to utilise because there is limited bandwidth, it may be dangerous to communicate, or communication may simply be unavailable at times. In this context, we argue for a rational approach to communication - if it has a cost, the agents should be able to calculate a value of communicating. By doing this, the agents can balance the need to communicate with the cost of doing so. In this research, we present a novel model of rational communication, that uses reward shaping to value communications, and employ this valuation in decentralised POMDP policy generation. In this context, reward shaping is the process by which expectations over joint actions are adjusted based on how coordinated the agent team is. An empirical evaluation of the benefits of this approach is presented in two domains. First, in the context of an idealised benchmark problem, the multiagent Tiger problem, our method is shown to require significantly less communication (up to 30% fewer messages) and still achieves a 30% performance improvement over the current state of the art. Second, in the context of a larger-scale problem, RoboCupRescue, our method is shown to scale well, and operate without recourse to significant amounts of domain knowledge.

Text
rewardShaping.pdf - Version of Record
Download (242kB)

More information

Published date: May 2009
Additional Information: Event Dates: May, 2009
Venue - Dates: Autonomous Agents And MultiAgent Systems, Budapest, Hungary, 2009-05-01
Organisations: Agents, Interactions & Complexity

Identifiers

Local EPrints ID: 267076
URI: http://eprints.soton.ac.uk/id/eprint/267076
PURE UUID: c24e160b-1e84-4daf-94ae-0962fc5b8e2f
ORCID for Enrico H. Gerding: ORCID iD orcid.org/0000-0001-7200-552X

Catalogue record

Date deposited: 03 Feb 2009 15:05
Last modified: 15 Mar 2024 03:23

Export record

Contributors

Author: Simon A. Williamson
Author: Enrico H. Gerding ORCID iD
Author: Nicholas R. Jennings

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×