The University of Southampton
University of Southampton Institutional Repository

Adaptive UAV-trajectory optimization under quality of service constraints: a model-free solution

Adaptive UAV-trajectory optimization under quality of service constraints: a model-free solution
Adaptive UAV-trajectory optimization under quality of service constraints: a model-free solution
Unmanned aerial vehicles (UAVs) with the potential of providing reliable high-rate connectivity, are becoming a promising component of future wireless networks. A UAV collects data from a set of randomly distributed sensors, where both the locations of these sensors and their data volume to be transmitted are unknown to the UAV. In order to assist the UAV in finding the optimal motion trajectory in the face of the uncertainty without the above knowledge whilst aiming for maximizing the cumulative collected data, we formulate a reinforcement learning problem by modelling the motion-trajectory as a Markov decision process with the UAV acting as the learning agent.
Then, we propose a pair of novel trajectory optimization algorithms based on stochastic modelling and reinforcement learning, which allows the UAV to optimize its flight trajectory without the need for system identification. More specifically, by dividing the considered region into small tiles, we conceive state-action-reward-state-action (Sarsa) and $Q$-learning based UAV-trajectory optimization algorithms (i.e., SUTOA and QUTOA) aiming to maximize the cumulative data collected during the finite flight-time. Our simulation results demonstrate that both of the proposed approaches are capable of finding an optimal trajectory under the flight-time constraint. The preference for QUTOA vs. SUTOA depends on the relative position of the start and the end points of the UAVs.
2169-3536
1-14
Cui, Jingjing
dbe3c3ed-762f-4abf-bd7b-8d2737f2f0fc
Ding, Zhiguo
3880e390-c1a9-450e-940b-e184e6d532d2
Deng, Yansha
0d528ce8-41ad-4d89-b2f9-0159c61fd0a1
Nallanathan, Arumugam
d255cda5-a015-4bb9-9f17-88614a544396
Hanzo, Lajos
66e7266f-3066-4fc0-8391-e000acce71a1
Cui, Jingjing
dbe3c3ed-762f-4abf-bd7b-8d2737f2f0fc
Ding, Zhiguo
3880e390-c1a9-450e-940b-e184e6d532d2
Deng, Yansha
0d528ce8-41ad-4d89-b2f9-0159c61fd0a1
Nallanathan, Arumugam
d255cda5-a015-4bb9-9f17-88614a544396
Hanzo, Lajos
66e7266f-3066-4fc0-8391-e000acce71a1

Cui, Jingjing, Ding, Zhiguo, Deng, Yansha, Nallanathan, Arumugam and Hanzo, Lajos (2020) Adaptive UAV-trajectory optimization under quality of service constraints: a model-free solution. IEEE Access, 1-14. (doi:10.1109/ACCESS.2019.DOI).

Record type: Article

Abstract

Unmanned aerial vehicles (UAVs) with the potential of providing reliable high-rate connectivity, are becoming a promising component of future wireless networks. A UAV collects data from a set of randomly distributed sensors, where both the locations of these sensors and their data volume to be transmitted are unknown to the UAV. In order to assist the UAV in finding the optimal motion trajectory in the face of the uncertainty without the above knowledge whilst aiming for maximizing the cumulative collected data, we formulate a reinforcement learning problem by modelling the motion-trajectory as a Markov decision process with the UAV acting as the learning agent.
Then, we propose a pair of novel trajectory optimization algorithms based on stochastic modelling and reinforcement learning, which allows the UAV to optimize its flight trajectory without the need for system identification. More specifically, by dividing the considered region into small tiles, we conceive state-action-reward-state-action (Sarsa) and $Q$-learning based UAV-trajectory optimization algorithms (i.e., SUTOA and QUTOA) aiming to maximize the cumulative data collected during the finite flight-time. Our simulation results demonstrate that both of the proposed approaches are capable of finding an optimal trajectory under the flight-time constraint. The preference for QUTOA vs. SUTOA depends on the relative position of the start and the end points of the UAVs.

Text
Jingjing_IEEE_Access_20 - Version of Record
Available under License Creative Commons Attribution.
Download (2MB)

More information

Published date: 6 June 2020

Identifiers

Local EPrints ID: 441344
URI: http://eprints.soton.ac.uk/id/eprint/441344
ISSN: 2169-3536
PURE UUID: 4830204a-5d87-4a2b-badc-2a5ce379c7b0
ORCID for Lajos Hanzo: ORCID iD orcid.org/0000-0002-2636-5214

Catalogue record

Date deposited: 10 Jun 2020 16:30
Last modified: 18 Mar 2024 02:36

Export record

Altmetrics

Contributors

Author: Jingjing Cui
Author: Zhiguo Ding
Author: Yansha Deng
Author: Arumugam Nallanathan
Author: Lajos Hanzo ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×