Deep Reinforcement Learning Aided Packet-Routing for Aeronautical Ad-Hoc Networks Formed by Passenger Planes
Deep Reinforcement Learning Aided Packet-Routing for Aeronautical Ad-Hoc Networks Formed by Passenger Planes
Data packet routing in aeronautical ad-hoc networks (AANETs) is challenging due to their high-dynamic topology. In this paper, we invoke deep reinforcement learning for routing in AANETs aiming at minimizing the end-to-end (E2E) delay. Specifically, a deep Q-network (DQN) is conceived for capturing the relationship between the optimal routing decision and the local geographic information observed by the forwarding node. The DQN is trained in an offline manner based on historical flight data and then stored by each airplane for assisting their routing decisions during flight. To boost the learning efficiency and the online adaptability of the proposed DQN-routing, we further exploit the knowledge concerning the system's dynamics by using a deep value network (DVN) conceived with a feedback mechanism. Our simulation results show that both DQN-routing and DVN-routing achieve lower E2E delay than the benchmark protocol, and DVN-routing performs similarly to the optimal routing that relies on perfect global information.
AANET, Ad hoc networks, Airplanes, Delays, Network topology, Routing, Routing protocols, Topology, deep reinforcement learning, routing
5166-5171
Liu, Dong
889643f2-afeb-4479-bd41-3ccedd53d89d
Cui, Jingjing
dbe3c3ed-762f-4abf-bd7b-8d2737f2f0fc
Zhang, Jiankang
6add829f-d955-40ca-8214-27a039defc8a
Yang, Chenyang
d42a57f7-0b91-408e-97dc-e7ce7b92d000
Hanzo, Lajos
66e7266f-3066-4fc0-8391-e000acce71a1
May 2021
Liu, Dong
889643f2-afeb-4479-bd41-3ccedd53d89d
Cui, Jingjing
dbe3c3ed-762f-4abf-bd7b-8d2737f2f0fc
Zhang, Jiankang
6add829f-d955-40ca-8214-27a039defc8a
Yang, Chenyang
d42a57f7-0b91-408e-97dc-e7ce7b92d000
Hanzo, Lajos
66e7266f-3066-4fc0-8391-e000acce71a1
Liu, Dong, Cui, Jingjing, Zhang, Jiankang, Yang, Chenyang and Hanzo, Lajos
(2021)
Deep Reinforcement Learning Aided Packet-Routing for Aeronautical Ad-Hoc Networks Formed by Passenger Planes.
IEEE Transactions on Vehicular Technology, 70 (5), , [9408403].
(doi:10.1109/TVT.2021.3074015).
Abstract
Data packet routing in aeronautical ad-hoc networks (AANETs) is challenging due to their high-dynamic topology. In this paper, we invoke deep reinforcement learning for routing in AANETs aiming at minimizing the end-to-end (E2E) delay. Specifically, a deep Q-network (DQN) is conceived for capturing the relationship between the optimal routing decision and the local geographic information observed by the forwarding node. The DQN is trained in an offline manner based on historical flight data and then stored by each airplane for assisting their routing decisions during flight. To boost the learning efficiency and the online adaptability of the proposed DQN-routing, we further exploit the knowledge concerning the system's dynamics by using a deep value network (DVN) conceived with a feedback mechanism. Our simulation results show that both DQN-routing and DVN-routing achieve lower E2E delay than the benchmark protocol, and DVN-routing performs similarly to the optimal routing that relies on perfect global information.
Text
Deep Reinforcement Learning Aided Routing in Aeronautical Ad Hoc Networks
- Accepted Manuscript
More information
Accepted/In Press date: 15 April 2021
e-pub ahead of print date: 19 April 2021
Published date: May 2021
Additional Information:
Funding Information:
Manuscript received June 2, 2020; revised December 31, 2020 and March 9, 2021; accepted April 15, 2021. Date of publication April 19, 2021; date of current version June 9, 2021. This work was supported in part by the Engineering and Physical Sciences Research Council Projects under Grant EP/N004558/1, Grant EP/P034284/1, Grant EP/P034284/1, and Grant EP/P003990/1 (COALESCE), in part by the Royal Society’s Global Challenges Research Fund Grant, and in part by the European Research Council’s Advanced Fellow Grant QuantCom under Grant 789028. The review of this article was coordinated by Prof. Xiaoxia Huang. (Corresponding author: Lajos Hanzo.) Dong Liu, Jingjing Cui, and Lajos Hanzo are with the School of Electronics and Computer Science, the University of Southampton, Southampton SO17 1BJ, U.K. (e-mail: d.liu@soton.ac.uk; jingj.cui@soton.ac.uk; lh@ecs.soton.ac.uk).
Publisher Copyright:
© 1967-2012 IEEE.
Keywords:
AANET, Ad hoc networks, Airplanes, Delays, Network topology, Routing, Routing protocols, Topology, deep reinforcement learning, routing
Identifiers
Local EPrints ID: 449654
URI: http://eprints.soton.ac.uk/id/eprint/449654
ISSN: 0018-9545
PURE UUID: 968b292c-187a-4bd1-89f9-603697a2855a
Catalogue record
Date deposited: 10 Jun 2021 16:31
Last modified: 18 Mar 2024 03:14
Export record
Altmetrics
Contributors
Author:
Dong Liu
Author:
Jingjing Cui
Author:
Jiankang Zhang
Author:
Chenyang Yang
Author:
Lajos Hanzo
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics