Topological Q-learning with internally guided exploration for mobile robot navigation
Topological Q-learning with internally guided exploration for mobile robot navigation
Improving the learning convergence of reinforcement learning (RL) in mobile robot navigation has been the interest of many recent works that have investigated different approaches to obtain knowledge from effectively and efficiently exploring the robot’s environment. In RL, this knowledge is of great importance for reducing the high number of interactions required for updating the value function and to eventually find an optimal or a nearly optimal policy for the agent. In this paper, we propose a topological Q-learning (TQ-learning) algorithm that makes use of the topological ordering among the observed states of the environment in which the agent acts. This algorithm builds an incremental topological map of the environment using Instantaneous Topological Map model which we use for accelerating value function updates as well as providing a guided exploration strategy for the agent. We evaluate our algorithm against the original Q-learning and the Influence Zone algorithms in static and dynamic environments.
1939-1954
Hafez, Muhammad Burhan
e8c991ab-d800-46f2-abeb-cb169a1ed47e
Loo, Chu Kiong
1fcd7fed-c797-4536-b13e-3bd050db4a6d
28 February 2015
Hafez, Muhammad Burhan
e8c991ab-d800-46f2-abeb-cb169a1ed47e
Loo, Chu Kiong
1fcd7fed-c797-4536-b13e-3bd050db4a6d
Hafez, Muhammad Burhan and Loo, Chu Kiong
(2015)
Topological Q-learning with internally guided exploration for mobile robot navigation.
Neural Computing and Applications, 26 (8), .
(doi:10.1007/s00521-015-1861-8).
Abstract
Improving the learning convergence of reinforcement learning (RL) in mobile robot navigation has been the interest of many recent works that have investigated different approaches to obtain knowledge from effectively and efficiently exploring the robot’s environment. In RL, this knowledge is of great importance for reducing the high number of interactions required for updating the value function and to eventually find an optimal or a nearly optimal policy for the agent. In this paper, we propose a topological Q-learning (TQ-learning) algorithm that makes use of the topological ordering among the observed states of the environment in which the agent acts. This algorithm builds an incremental topological map of the environment using Instantaneous Topological Map model which we use for accelerating value function updates as well as providing a guided exploration strategy for the agent. We evaluate our algorithm against the original Q-learning and the Influence Zone algorithms in static and dynamic environments.
Text
s00521-015-1861-8
- Version of Record
Restricted to Repository staff only
Request a copy
More information
Accepted/In Press date: 17 February 2015
Published date: 28 February 2015
Identifiers
Local EPrints ID: 495803
URI: http://eprints.soton.ac.uk/id/eprint/495803
ISSN: 0941-0643
PURE UUID: a60a21e7-f9fe-4b25-a464-4e44d73c2a08
Catalogue record
Date deposited: 22 Nov 2024 18:05
Last modified: 23 Nov 2024 03:11
Export record
Altmetrics
Contributors
Author:
Muhammad Burhan Hafez
Author:
Chu Kiong Loo
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics