Conversational language models for human-in-the-loop multi-robot coordination
Conversational language models for human-in-the-loop multi-robot coordination
With the increasing prevalence and diversity of robots interacting in the real world, there is need for flexible, on-the-fly planning and cooperation. Large Language Models are starting to be explored in a multimodal setup for communication, coordination, and planning in robotics. Existing approaches generally use a single agent building a plan, or have multiple homogeneous agents coordinating for a simple task. We present a decentralised, dialogical approach in which a team of agents with different abilities plans solutions through peer-to-peer and human-robot discussion. We suggest that argument-style dialogues are an effective way to facilitate adaptive use of each agent's abilities within a cooperative team. Two robots discuss how to solve a cleaning problem set by a human, define roles, and agree on paths they each take. Each step can be interrupted by a human advisor and agents check their plans with the human. Agents then execute this plan in the real world, collecting rubbish from people in each room. Our implementation uses text at every step, maintaining transparency and effective human-multi-robot interaction.
2809 - 2811
International Foundation for Autonomous Agents and Multiagent Systems
Hunt, William
eec4ba79-8870-4657-a2ea-25511ae9dbaa
Godfrey, Toby
a79ab5bd-264c-4464-b82c-f19edf1a5f08
Soorati, Mohammad
35fe6bbb-ce52-4c21-a46e-9bb0e31d246c
6 May 2024
Hunt, William
eec4ba79-8870-4657-a2ea-25511ae9dbaa
Godfrey, Toby
a79ab5bd-264c-4464-b82c-f19edf1a5f08
Soorati, Mohammad
35fe6bbb-ce52-4c21-a46e-9bb0e31d246c
Hunt, William, Godfrey, Toby and Soorati, Mohammad
(2024)
Conversational language models for human-in-the-loop multi-robot coordination.
In AAMAS '24: Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems.
International Foundation for Autonomous Agents and Multiagent Systems.
.
Record type:
Conference or Workshop Item
(Paper)
Abstract
With the increasing prevalence and diversity of robots interacting in the real world, there is need for flexible, on-the-fly planning and cooperation. Large Language Models are starting to be explored in a multimodal setup for communication, coordination, and planning in robotics. Existing approaches generally use a single agent building a plan, or have multiple homogeneous agents coordinating for a simple task. We present a decentralised, dialogical approach in which a team of agents with different abilities plans solutions through peer-to-peer and human-robot discussion. We suggest that argument-style dialogues are an effective way to facilitate adaptive use of each agent's abilities within a cooperative team. Two robots discuss how to solve a cleaning problem set by a human, define roles, and agree on paths they each take. Each step can be interrupted by a human advisor and agents check their plans with the human. Agents then execute this plan in the real world, collecting rubbish from people in each room. Our implementation uses text at every step, maintaining transparency and effective human-multi-robot interaction.
This record has no associated files available for download.
More information
Published date: 6 May 2024
Venue - Dates:
The 23rd International Conference on Autonomous Agents and Multi-Agent Systems, Cordis Hotel, Auckland, New Zealand, 2024-05-06 - 2024-05-10
Identifiers
Local EPrints ID: 491931
URI: http://eprints.soton.ac.uk/id/eprint/491931
PURE UUID: 47d5fa25-0c7f-4ca0-bd93-ab4dd9a925e4
Catalogue record
Date deposited: 08 Jul 2024 17:11
Last modified: 11 Jul 2024 02:04
Export record
Contributors
Author:
William Hunt
Author:
Toby Godfrey
Author:
Mohammad Soorati
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics