Realisation of early-exit dynamic neural networks on reconfigurable hardware

Abstract

Early-Exiting is a strategy that’s becoming popular in Deep Neural Networks (DNNs), as it can lead to faster execution and a reduction in the computational intensity of inference. To achieve this intermediate classifiers abstract information from the input samples to strategically stop forward propagation and generate an output at an earlier stage. Confidence criteria are used to identify easier to recognise samples over the ones that need further filtering. However, such dynamic DNNs have only been realised in conventional computing systems (CPU+GPU) using libraries designed for static networks. In this paper, we do a first exploration to efficiently realise early-exit dynamic DNNs on FPGAs, a platform already proven to be highly effective for neural network applications. We consider two approaches for implementing and executing the intermediate classifiers: pipeline, which uses existing hardware, and parallel, which uses additional dedicated modules. We model their energy needs and execution time and explore their performance using the BranchyNet early exit approach on LeNet-5, AlexNet, VGG19 and ResNet32, and a Xilinx ZCU106 Evaluation Board. We found that the dynamic approaches are at least 24% faster than a static network executed on a FPGA, consuming a minimum of 1.32x lower energy. We further observe that FPGAs can enhance the performance of early-exit dynamic DNNs, by minimising the complexities introduced by the decision intermediate classifiers, through parallel execution. Finally we compare the two approaches, and identify which is best for different network types and confidence levels.

More information

Identifiers

ORCID for Jonathon Hare: orcid.org/0000-0003-2921-4283

ORCID for Geoff V. Merrett: orcid.org/0000-0003-4980-3894

Catalogue record

Export record

Altmetrics