Decentralized Bayesian reinforcement learning for online agent collaboration


Teacy, W.T.L., Chalkiadakis, G., Farinelli, A., Rogers, A., Jennings, N.R., McClean, S. and Parr, G. (2012) Decentralized Bayesian reinforcement learning for online agent collaboration. In, 11th International Conference on Autonomous Agents and Multiagent Systems, Valencia, Spain, 04 - 08 Jun 2012. , 417-424.

Download

[img] PDF - Published Version
Download (410Kb)

Description/Abstract

Solving complex but structured problems in a decentralized manner via multiagent collaboration has received much attention in recent years. This is natural, as on one hand, multiagent systems usually possess a structure that determines the allowable interactions among the agents; and on the other hand, the single most pressing need in a cooperative multiagent system is to coordinate the local policies of autonomous agents with restricted capabilities to serve a system-wide goal. The presence of uncertainty makes this even more challenging, as the agents face the additional need to learn the unknown environment parameters while forming (and following) local policies in an online fashion. In this paper, we provide the first Bayesian reinforcement learning (BRL) approach for distributed coordination and learning in a cooperative multiagent system by devising two solutions to this type of problem. More specifically, we show how the Value of Perfect Information (VPI) can be used to perform efficient decentralised exploration in both model-based and model-free BRL, and in the latter case, provide a closed form solution for VPI, correcting a decade old result by Dearden, Friedman and Russell. To evaluate these solutions, we present experimental results comparing their relative merits, and demonstrate empirically that both solutions outperform an existing multiagent learning method, representative of the state-of-the-art.

Item Type: Conference or Workshop Item (Paper)
Keywords: multiagent learning, Bayesian techniques, uncertainty
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Faculty of Physical Sciences and Engineering > Electronics and Computer Science > Agents, Interactions & Complexity
ePrint ID: 273201
Date Deposited: 08 Feb 2012 13:11
Last Modified: 14 Apr 2014 11:36
Research Funder: EPSRC
Projects:
HUMAN-AGENT COLLECTIVES: FROM FOUNDATIONS TO APPLICATIONS [ORCHID]
Funded by: EPSRC (EP/I011587/1)
Led by: Nick Jennings
1 January 2011 to 31 December 2015
Contact Email Address: wtlt@ecs.soton.ac.uk
Further Information:Google Scholar
URI: http://eprints.soton.ac.uk/id/eprint/273201

Actions (login required)

View Item View Item