The University of Southampton
University of Southampton Institutional Repository

Deep Reinforcement Learning Aided Packet-Routing for Aeronautical Ad-Hoc Networks Formed by Passenger Planes

Deep Reinforcement Learning Aided Packet-Routing for Aeronautical Ad-Hoc Networks Formed by Passenger Planes
Deep Reinforcement Learning Aided Packet-Routing for Aeronautical Ad-Hoc Networks Formed by Passenger Planes

Data packet routing in aeronautical ad-hoc networks (AANETs) is challenging due to their high-dynamic topology. In this paper, we invoke deep reinforcement learning for routing in AANETs aiming at minimizing the end-to-end (E2E) delay. Specifically, a deep Q-network (DQN) is conceived for capturing the relationship between the optimal routing decision and the local geographic information observed by the forwarding node. The DQN is trained in an offline manner based on historical flight data and then stored by each airplane for assisting their routing decisions during flight. To boost the learning efficiency and the online adaptability of the proposed DQN-routing, we further exploit the knowledge concerning the system's dynamics by using a deep value network (DVN) conceived with a feedback mechanism. Our simulation results show that both DQN-routing and DVN-routing achieve lower E2E delay than the benchmark protocol, and DVN-routing performs similarly to the optimal routing that relies on perfect global information.

AANET, Ad hoc networks, Airplanes, Delays, Network topology, Routing, Routing protocols, Topology, deep reinforcement learning, routing
0018-9545
5166-5171
Liu, Dong
889643f2-afeb-4479-bd41-3ccedd53d89d
Cui, Jingjing
dbe3c3ed-762f-4abf-bd7b-8d2737f2f0fc
Zhang, Jiankang
6add829f-d955-40ca-8214-27a039defc8a
Yang, Chenyang
d42a57f7-0b91-408e-97dc-e7ce7b92d000
Hanzo, Lajos
66e7266f-3066-4fc0-8391-e000acce71a1
Liu, Dong
889643f2-afeb-4479-bd41-3ccedd53d89d
Cui, Jingjing
dbe3c3ed-762f-4abf-bd7b-8d2737f2f0fc
Zhang, Jiankang
6add829f-d955-40ca-8214-27a039defc8a
Yang, Chenyang
d42a57f7-0b91-408e-97dc-e7ce7b92d000
Hanzo, Lajos
66e7266f-3066-4fc0-8391-e000acce71a1

Liu, Dong, Cui, Jingjing, Zhang, Jiankang, Yang, Chenyang and Hanzo, Lajos (2021) Deep Reinforcement Learning Aided Packet-Routing for Aeronautical Ad-Hoc Networks Formed by Passenger Planes. IEEE Transactions on Vehicular Technology, 70 (5), 5166-5171, [9408403]. (doi:10.1109/TVT.2021.3074015).

Record type: Article

Abstract

Data packet routing in aeronautical ad-hoc networks (AANETs) is challenging due to their high-dynamic topology. In this paper, we invoke deep reinforcement learning for routing in AANETs aiming at minimizing the end-to-end (E2E) delay. Specifically, a deep Q-network (DQN) is conceived for capturing the relationship between the optimal routing decision and the local geographic information observed by the forwarding node. The DQN is trained in an offline manner based on historical flight data and then stored by each airplane for assisting their routing decisions during flight. To boost the learning efficiency and the online adaptability of the proposed DQN-routing, we further exploit the knowledge concerning the system's dynamics by using a deep value network (DVN) conceived with a feedback mechanism. Our simulation results show that both DQN-routing and DVN-routing achieve lower E2E delay than the benchmark protocol, and DVN-routing performs similarly to the optimal routing that relies on perfect global information.

Text
Deep Reinforcement Learning Aided Routing in Aeronautical Ad Hoc Networks - Accepted Manuscript
Download (604kB)

More information

Accepted/In Press date: 15 April 2021
e-pub ahead of print date: 19 April 2021
Published date: May 2021
Additional Information: Funding Information: Manuscript received June 2, 2020; revised December 31, 2020 and March 9, 2021; accepted April 15, 2021. Date of publication April 19, 2021; date of current version June 9, 2021. This work was supported in part by the Engineering and Physical Sciences Research Council Projects under Grant EP/N004558/1, Grant EP/P034284/1, Grant EP/P034284/1, and Grant EP/P003990/1 (COALESCE), in part by the Royal Society’s Global Challenges Research Fund Grant, and in part by the European Research Council’s Advanced Fellow Grant QuantCom under Grant 789028. The review of this article was coordinated by Prof. Xiaoxia Huang. (Corresponding author: Lajos Hanzo.) Dong Liu, Jingjing Cui, and Lajos Hanzo are with the School of Electronics and Computer Science, the University of Southampton, Southampton SO17 1BJ, U.K. (e-mail: d.liu@soton.ac.uk; jingj.cui@soton.ac.uk; lh@ecs.soton.ac.uk). Publisher Copyright: © 1967-2012 IEEE.
Keywords: AANET, Ad hoc networks, Airplanes, Delays, Network topology, Routing, Routing protocols, Topology, deep reinforcement learning, routing

Identifiers

Local EPrints ID: 449654
URI: http://eprints.soton.ac.uk/id/eprint/449654
ISSN: 0018-9545
PURE UUID: 968b292c-187a-4bd1-89f9-603697a2855a
ORCID for Dong Liu: ORCID iD orcid.org/0000-0002-0619-1480
ORCID for Jiankang Zhang: ORCID iD orcid.org/0000-0001-5316-1711
ORCID for Lajos Hanzo: ORCID iD orcid.org/0000-0002-2636-5214

Catalogue record

Date deposited: 10 Jun 2021 16:31
Last modified: 18 Mar 2024 03:14

Export record

Altmetrics

Contributors

Author: Dong Liu ORCID iD
Author: Jingjing Cui
Author: Jiankang Zhang ORCID iD
Author: Chenyang Yang
Author: Lajos Hanzo ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×