Enhancing the fuel-economy of V2I-assisted autonomous driving: a reinforcement learning approach
Enhancing the fuel-economy of V2I-assisted autonomous driving: a reinforcement learning approach
A novel framework is proposed for enhancing the driving safety and fuel economy of autonomous vehicles (AVs) with the aid of vehicle-To-infrastructure (V2I) communication networks. The problem of driving trajectory design is formulated for minimizing the total fuel consumption, while enhancing driving safety (by obeying the traffic rules and avoiding obstacles). In an effort to solve this pertinent problem, a deep reinforcement learning (DRL) approach is proposed for making collision-free decisions. Firstly, a deep Q-network (DQN) aided algorithm is proposed for determining the trajectory and velocity of the AV by receiving real-Time traffic information from the base stations (BSs). More particularly, the AV acts as an agent to carry out optimal action such as lane change and velocity change by interacting with the environment. Secondly, to overcome the large overestimation of action values by the Q-learning model, a double deep Q-network (DDQN) algorithm is proposed by decomposing the max-Q-value operation into action selection and action evaluation. Additionally, three practical driving policies are also proposed as benchmarks. Numerical results are provided for demonstrating that the proposed trajectory design algorithms are capable of enhancing the driving safety and fuel economy of AVs. We demonstrate that the proposed DDQN based algorithm outperforms the DQN based algorithm. Additionally, it is also demonstrated that the proposed fuel-economy (FE) based driving policy derived from the DRL algorithm is capable of achieving in excess of 24% of fuel savings over the benchmarks.
Autonomous driving, V2I communications, deep reinforcement learning, fuel consumption, safe driving, trajectory design
8329-8342
Liu, Xiao
0a6c7a08-5930-4a75-a686-9ecfb7662d19
Liu, Yuanwei
4bff35d5-479f-4239-b4a3-a3eb918b304e
Chen, Yue
9b646fd4-7826-4d0b-b81a-c4bc5eae1be1
Hanzo, Lajos
66e7266f-3066-4fc0-8391-e000acce71a1
August 2020
Liu, Xiao
0a6c7a08-5930-4a75-a686-9ecfb7662d19
Liu, Yuanwei
4bff35d5-479f-4239-b4a3-a3eb918b304e
Chen, Yue
9b646fd4-7826-4d0b-b81a-c4bc5eae1be1
Hanzo, Lajos
66e7266f-3066-4fc0-8391-e000acce71a1
Liu, Xiao, Liu, Yuanwei, Chen, Yue and Hanzo, Lajos
(2020)
Enhancing the fuel-economy of V2I-assisted autonomous driving: a reinforcement learning approach.
IEEE Transactions on Vehicular Technology, 69 (8), , [9097944].
(doi:10.1109/TVT.2020.2996187).
Abstract
A novel framework is proposed for enhancing the driving safety and fuel economy of autonomous vehicles (AVs) with the aid of vehicle-To-infrastructure (V2I) communication networks. The problem of driving trajectory design is formulated for minimizing the total fuel consumption, while enhancing driving safety (by obeying the traffic rules and avoiding obstacles). In an effort to solve this pertinent problem, a deep reinforcement learning (DRL) approach is proposed for making collision-free decisions. Firstly, a deep Q-network (DQN) aided algorithm is proposed for determining the trajectory and velocity of the AV by receiving real-Time traffic information from the base stations (BSs). More particularly, the AV acts as an agent to carry out optimal action such as lane change and velocity change by interacting with the environment. Secondly, to overcome the large overestimation of action values by the Q-learning model, a double deep Q-network (DDQN) algorithm is proposed by decomposing the max-Q-value operation into action selection and action evaluation. Additionally, three practical driving policies are also proposed as benchmarks. Numerical results are provided for demonstrating that the proposed trajectory design algorithms are capable of enhancing the driving safety and fuel economy of AVs. We demonstrate that the proposed DDQN based algorithm outperforms the DQN based algorithm. Additionally, it is also demonstrated that the proposed fuel-economy (FE) based driving policy derived from the DRL algorithm is capable of achieving in excess of 24% of fuel savings over the benchmarks.
Text
Enhancing the Fuel-Economy of V2I-Assisted Autonomous Driving A Reinforcement Learning Approach double column
- Accepted Manuscript
More information
Accepted/In Press date: 8 May 2020
e-pub ahead of print date: 21 May 2020
Published date: August 2020
Additional Information:
Funding Information:
Manuscript received July 30, 2019; revised December 20, 2019 and April 12, 2020; accepted May 7, 2020. Date of publication May 21, 2020; date of current version August 13, 2020. The work of L. Hanzo was supported in part by the financial support of the Engineering and Physical Sciences Research Council projects EP/N004558/1, EP/P034284/1, EP/P034284/1, and EP/P003990/1 (COALESCE), of the Royal Society’s Global Challenges Research Fund Grant, and in part by the European Research Council’s Advanced Fellow Grant Quant-Com. This work was presented in part at the IEEE International Conference on Communications, Dublin, Ireland, June 2020. The review of this article was coordinated by Dr. E. Velenis. (Corresponding author: Lajos Hanzo.) Xiao Liu, Yuanwei Liu, and Yue Chen are with the School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, U.K. (e-mail: x.liu@qmul.ac.uk; yuanwei.liu@qmul.ac.uk; yue.chen@ qmul.ac.uk).
Publisher Copyright:
© 1967-2012 IEEE.
Keywords:
Autonomous driving, V2I communications, deep reinforcement learning, fuel consumption, safe driving, trajectory design
Identifiers
Local EPrints ID: 440998
URI: http://eprints.soton.ac.uk/id/eprint/440998
ISSN: 0018-9545
PURE UUID: b91edc60-d1ba-492e-ac6b-ac9b52cf3e29
Catalogue record
Date deposited: 27 May 2020 16:53
Last modified: 18 Mar 2024 02:36
Export record
Altmetrics
Contributors
Author:
Xiao Liu
Author:
Yuanwei Liu
Author:
Yue Chen
Author:
Lajos Hanzo
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics