The University of Southampton
University of Southampton Institutional Repository

Cooperative information sharing to improve distributed learning in multi-agent systems

Cooperative information sharing to improve distributed learning in multi-agent systems
Cooperative information sharing to improve distributed learning in multi-agent systems
Effective coordination of agents' actions in partially-observable domains is a major challenge of multi-agent systems research. To address this, many researchers have developed techniques that allow the agents to make decisions based on estimates of the states and actions of other agents that are typically learnt using some form of machine learning algorithm. Nevertheless, many of these approaches fail to provide an actual means by which the necessary information is made available so that the estimates can be learnt. To this end, we argue that cooperative communication of state information between agents is one such mechanism. However, in a dynamically changing environment, the accuracy and timeliness of this communicated information determine the fidelity of the learned estimates and the usefulness of the actions taken based on these. Given this, we propose a novel information-sharing protocol, post-task-completion sharing, for the distribution of state information. We then show, through a formal analysis, the improvement in the quality of estimates produced using our strategy over the widely used protocol of sharing information between nearest neighbours. Moreover, communication heuristics designed around our information-sharing principle are subjected to empirical evaluation along with other benchmark strategies (including Littman's Q-routing and Stone's TPOT-RL) in a simulated call-routing application. These studies, conducted across a range of environmental settings, show that, compared to the different benchmarks used, our strategy generates an improvement of up to 60% in the call connection rate; of more than 1000% in the ability to connect long-distance calls; and incurs as low as 0.25 of the message overhead.
407-463
Dutta, P.S.
573e4782-1998-41d9-aa83-921bb4238b1c
Jennings, N. R.
ab3d94cc-247c-4545-9d1e-65873d6cdb30
Moreau, L.
033c63dd-3fe9-4040-849f-dfccbe0406f8
Dutta, P.S.
573e4782-1998-41d9-aa83-921bb4238b1c
Jennings, N. R.
ab3d94cc-247c-4545-9d1e-65873d6cdb30
Moreau, L.
033c63dd-3fe9-4040-849f-dfccbe0406f8

Dutta, P.S., Jennings, N. R. and Moreau, L. (2005) Cooperative information sharing to improve distributed learning in multi-agent systems. Journal of AI Research, 24, 407-463. (doi:10.1613/jair.1735).

Record type: Article

Abstract

Effective coordination of agents' actions in partially-observable domains is a major challenge of multi-agent systems research. To address this, many researchers have developed techniques that allow the agents to make decisions based on estimates of the states and actions of other agents that are typically learnt using some form of machine learning algorithm. Nevertheless, many of these approaches fail to provide an actual means by which the necessary information is made available so that the estimates can be learnt. To this end, we argue that cooperative communication of state information between agents is one such mechanism. However, in a dynamically changing environment, the accuracy and timeliness of this communicated information determine the fidelity of the learned estimates and the usefulness of the actions taken based on these. Given this, we propose a novel information-sharing protocol, post-task-completion sharing, for the distribution of state information. We then show, through a formal analysis, the improvement in the quality of estimates produced using our strategy over the widely used protocol of sharing information between nearest neighbours. Moreover, communication heuristics designed around our information-sharing principle are subjected to empirical evaluation along with other benchmark strategies (including Littman's Q-routing and Stone's TPOT-RL) in a simulated call-routing application. These studies, conducted across a range of environmental settings, show that, compared to the different benchmarks used, our strategy generates an improvement of up to 60% in the call connection rate; of more than 1000% in the ability to connect long-distance calls; and incurs as low as 0.25 of the message overhead.

Text
dutta05a.pdf - Accepted Manuscript
Available under License Creative Commons Attribution.
Download (1MB)

More information

Published date: October 2005
Organisations: Web & Internet Science, Agents, Interactions & Complexity

Identifiers

Local EPrints ID: 260810
URI: http://eprints.soton.ac.uk/id/eprint/260810
PURE UUID: f26eb41d-eaac-4fc2-9a49-b0dd3f70c9a5
ORCID for L. Moreau: ORCID iD orcid.org/0000-0002-3494-120X

Catalogue record

Date deposited: 29 Apr 2005
Last modified: 14 Mar 2024 06:43

Export record

Altmetrics

Contributors

Author: P.S. Dutta
Author: N. R. Jennings
Author: L. Moreau ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×