Optimising hardware accelerated neural networks with quantisation and a knowledge distillation evolutionary algorithm

This paper compares the latency, accuracy, training time and hardware costs of neural networks compressed with our new multi-objective evolutionary algorithm called NEMOKD, and with quantisation. We evaluate NEMOKD on Intel’s Movidius Myriad X VPU processor, and quantisation on Xilinx’s programmable Z7020 FPGA hardware. Evolving models with NEMOKD increases inference accuracy by up to 82% at the cost of 38% increased latency, with throughput performance of 100–590 image frames-per-second (FPS). Quantisation identifies a sweet spot of 3 bit precision in the trade-off between latency, hardware requirements, training time and accuracy. Parallelising FPGA implementations of 2 and 3 bit quantised neural networks increases throughput from 6 k FPS to 373 k FPS, a 62× speedup.

Evolutionary algorithm, FPGA, Movidius VPU, Neural network, Quantisation

10.3390/electronics10040396

2079-9292

1-21

Stewart, Robert

3b99f51f-d1fe-4783-a845-5668e67b72bb

Nowlan, Andrew

d2343b2a-a504-4cd9-8d7a-31a7ed8dc1d3

Bacchus, Pascal

eb49c519-4327-46a6-9bb8-1d426e561767

Ducasse, Quentin

805341d9-7bca-43c1-a572-50db9eac16ad

Komendantskaya, Ekaterina

f12d9c23-5589-40b8-bcf9-a04fe9dedf61

Stewart, Robert

3b99f51f-d1fe-4783-a845-5668e67b72bb

Nowlan, Andrew

d2343b2a-a504-4cd9-8d7a-31a7ed8dc1d3

Bacchus, Pascal

eb49c519-4327-46a6-9bb8-1d426e561767

Ducasse, Quentin

805341d9-7bca-43c1-a572-50db9eac16ad

Komendantskaya, Ekaterina

f12d9c23-5589-40b8-bcf9-a04fe9dedf61

Stewart, Robert, Nowlan, Andrew, Bacchus, Pascal, Ducasse, Quentin and Komendantskaya, Ekaterina (2021) Optimising hardware accelerated neural networks with quantisation and a knowledge distillation evolutionary algorithm. Electronics (Switzerland), 10 (4), 1-21, [396]. (doi:10.3390/electronics10040396).

Record type: Article

Abstract

This record has no associated files available for download.

More information

Accepted/In Press date: 1 February 2021

e-pub ahead of print date: 5 February 2021

Additional Information: Funding Information: Funding: This research was funded by EPSRC project “Border Patrol: Improving Smart Device Security through Type-Aware Systems Design (EP/N028201/1)”; EPSRC project “Serious Coding: A Game Approach To Security For The New Code-Citizens (EP/T017511/1)”; National Cyber Security Center, UK, Grant “SecConn-NN: Neural Networks with Security Contracts—towards lightweight, modular security for neural networks”; UK Research Institute in Verified Trustworthy Software Systems research project “CONVENER: Continuous Verification of Neural Networks” (from the “Digital Security Through Verification” call). Publisher Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland.

Keywords: Evolutionary algorithm, FPGA, Movidius VPU, Neural network, Quantisation

Learn more about Cyber Physical Systems research