The University of Southampton
University of Southampton Institutional Repository

Topological Q-learning with internally guided exploration for mobile robot navigation

Topological Q-learning with internally guided exploration for mobile robot navigation
Topological Q-learning with internally guided exploration for mobile robot navigation
Improving the learning convergence of reinforcement learning (RL) in mobile robot navigation has been the interest of many recent works that have investigated different approaches to obtain knowledge from effectively and efficiently exploring the robot’s environment. In RL, this knowledge is of great importance for reducing the high number of interactions required for updating the value function and to eventually find an optimal or a nearly optimal policy for the agent. In this paper, we propose a topological Q-learning (TQ-learning) algorithm that makes use of the topological ordering among the observed states of the environment in which the agent acts. This algorithm builds an incremental topological map of the environment using Instantaneous Topological Map model which we use for accelerating value function updates as well as providing a guided exploration strategy for the agent. We evaluate our algorithm against the original Q-learning and the Influence Zone algorithms in static and dynamic environments.
0941-0643
1939-1954
Hafez, Muhammad Burhan
e8c991ab-d800-46f2-abeb-cb169a1ed47e
Loo, Chu Kiong
1fcd7fed-c797-4536-b13e-3bd050db4a6d
Hafez, Muhammad Burhan
e8c991ab-d800-46f2-abeb-cb169a1ed47e
Loo, Chu Kiong
1fcd7fed-c797-4536-b13e-3bd050db4a6d

Hafez, Muhammad Burhan and Loo, Chu Kiong (2015) Topological Q-learning with internally guided exploration for mobile robot navigation. Neural Computing and Applications, 26 (8), 1939-1954. (doi:10.1007/s00521-015-1861-8).

Record type: Article

Abstract

Improving the learning convergence of reinforcement learning (RL) in mobile robot navigation has been the interest of many recent works that have investigated different approaches to obtain knowledge from effectively and efficiently exploring the robot’s environment. In RL, this knowledge is of great importance for reducing the high number of interactions required for updating the value function and to eventually find an optimal or a nearly optimal policy for the agent. In this paper, we propose a topological Q-learning (TQ-learning) algorithm that makes use of the topological ordering among the observed states of the environment in which the agent acts. This algorithm builds an incremental topological map of the environment using Instantaneous Topological Map model which we use for accelerating value function updates as well as providing a guided exploration strategy for the agent. We evaluate our algorithm against the original Q-learning and the Influence Zone algorithms in static and dynamic environments.

Text
s00521-015-1861-8 - Version of Record
Restricted to Repository staff only
Request a copy

More information

Accepted/In Press date: 17 February 2015
Published date: 28 February 2015

Identifiers

Local EPrints ID: 495803
URI: http://eprints.soton.ac.uk/id/eprint/495803
ISSN: 0941-0643
PURE UUID: a60a21e7-f9fe-4b25-a464-4e44d73c2a08
ORCID for Muhammad Burhan Hafez: ORCID iD orcid.org/0000-0003-1670-8962

Catalogue record

Date deposited: 22 Nov 2024 18:05
Last modified: 23 Nov 2024 03:11

Export record

Altmetrics

Contributors

Author: Muhammad Burhan Hafez ORCID iD
Author: Chu Kiong Loo

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×