Using reinforcement learning to combine Green Light Optimised Speed Advisory and responsive traffic control systems with non-autonomous vehicles: Effects of imperfect training and incomplete information

Abstract

The advantages of Green Light Optimal Speed Advisory (GLOSA) include improvements in travel times, fuel consumption, battery life, emissions, the number of stops, and ride smoothness for cars, buses, HGVs, HEVs, and EVs in scenarios ranging from isolated junctions to connected networks. However, GLOSA is a detriment when paired with responsive traffic control systems (RTCs), which are presently widespread, greatly reducing the number of locations where the advantages of GLOSA could be obtained. This is because RTCs cannot provide accurate long-term future signal plans without the introduction of some mitigating solution. However, as Reinforcement Learning methods can, by trial and error, identify unseen solutions to complicated problems, there appears to be no reason they couldn’t be used to create frameworks designed to combine GLOSA and RTCs and benefit (Connected Non-Autonomous Vehicles) CNAVs, allowing the advantages of GLOSA to be obtained with the present traffic and traffic control make-ups. Therefore, this thesis investigates the performance of such frameworks in both ideal and non-ideal conditions. To achieve this, an initial framework was constructed and evaluated on a simulated isolated junction. It was found that the initial framework had a positive impact on stopping time and junction entry speed when traffic densities were at 55% of saturation levels but a negative impact at traffic densities of 70% as the Reinforcement Learning Traffic Control (RLTC) system was unable to reach states where vehicles could approach the junction without the obstruction of queues regularly enough. Also, in the 55% scenario, when the training penetration rate (TPR) was lower than 20%, performance was degraded if the evaluation penetration rate (EPR) was increased, as vehicle behaviour differed greatly from what the models were expecting. To expand testing to arterial flows, a revised framework was created and tested on a simulated arterial testbed. Alterations were also made to the framework to account for lessons learned from the isolated junction experiments. It was found that the revised framework had a positive impact at all traffic densities, with vehicle speed increasing and stops and waiting time decreasing compared to benchmark systems, even when performance was degraded by the absence of vehicle route information. However, performance was still degraded at higher EPRs when the TPR was lower than 40%. Overall, this research shows that GLOSA can be combined with Reinforcement Learning Traffic Control (RLTC) systems in a way that benefits CNAVs in terms of waiting time, number of stops, junction entry speed and average speeds, and demonstrates that GLOSA could be installed despite present traffic and traffic control make-ups. It also highlights the need for future frameworks to be tested at a variety of EPRs and with/without route information to avoid decreases in performance or outright failures as conditions at the site of deployments change over time.

More information

Identifiers

ORCID for William Paine: orcid.org/0000-0003-4894-9680

ORCID for Ben Waterson: orcid.org/0000-0001-9817-7119

ORCID for Tom Cherrett: orcid.org/0000-0003-0394-5459

Catalogue record

Export record