The University of Southampton
University of Southampton Institutional Repository

Fluid dynamic DNNs for reliable and adaptive distributed inference on edge devices

Fluid dynamic DNNs for reliable and adaptive distributed inference on edge devices
Fluid dynamic DNNs for reliable and adaptive distributed inference on edge devices
Distributed inference is a popular approach for efficient DNN inference at the edge. However, traditional Static and Dynamic DNNs are not distribution-friendly, causing system reliability and adaptability issues. In this paper, we introduce Fluid Dynamic DNNs (Fluid DyDNNs), tailored for distributed inference. Distinct from Static and Dynamic DNNs, Fluid DyDNNs utilize a novel nested incremental training algorithm to enable independent and combined operation of its sub-networks, enhancing system reliability and adaptability. Evaluation on embedded Arm CPUs with a DNN model and the MNIST dataset, shows that in scenarios of single device failure, Fluid DyDNNs ensure continued inference, whereas Static and Dynamic DNNs fail. When devices are fully operational, Fluid DyDNNs can operate in either a High-Accuracy mode and achieve comparable accuracy with Static DNNs, or in a High-Throughput mode and achieve 2.5x and 2x throughput compared with Static and Dynamic DNNs, respectively.
Xun, Lei
d30d0c37-7c17-4eed-b02c-1a0f81844f17
Hu, Mingyu
686551f3-f76b-471d-b424-71a5c68851da
Zhao, Hengrui
9a7e2ba5-4932-4188-8aef-ae39d17fca46
Singh, Amit Kumar
bded7886-24ab-4a24-8539-f8fe106426ac
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Merrett, Geoff V.
89b3a696-41de-44c3-89aa-b0aa29f54020
Xun, Lei
d30d0c37-7c17-4eed-b02c-1a0f81844f17
Hu, Mingyu
686551f3-f76b-471d-b424-71a5c68851da
Zhao, Hengrui
9a7e2ba5-4932-4188-8aef-ae39d17fca46
Singh, Amit Kumar
bded7886-24ab-4a24-8539-f8fe106426ac
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Merrett, Geoff V.
89b3a696-41de-44c3-89aa-b0aa29f54020

Xun, Lei, Hu, Mingyu, Zhao, Hengrui, Singh, Amit Kumar, Hare, Jonathon and Merrett, Geoff V. (2024) Fluid dynamic DNNs for reliable and adaptive distributed inference on edge devices. Design, Automation and Test in Europe, , Valencia, Spain. 25 - 27 Mar 2024. 2 pp . (In Press)

Record type: Conference or Workshop Item (Paper)

Abstract

Distributed inference is a popular approach for efficient DNN inference at the edge. However, traditional Static and Dynamic DNNs are not distribution-friendly, causing system reliability and adaptability issues. In this paper, we introduce Fluid Dynamic DNNs (Fluid DyDNNs), tailored for distributed inference. Distinct from Static and Dynamic DNNs, Fluid DyDNNs utilize a novel nested incremental training algorithm to enable independent and combined operation of its sub-networks, enhancing system reliability and adaptability. Evaluation on embedded Arm CPUs with a DNN model and the MNIST dataset, shows that in scenarios of single device failure, Fluid DyDNNs ensure continued inference, whereas Static and Dynamic DNNs fail. When devices are fully operational, Fluid DyDNNs can operate in either a High-Accuracy mode and achieve comparable accuracy with Static DNNs, or in a High-Throughput mode and achieve 2.5x and 2x throughput compared with Static and Dynamic DNNs, respectively.

Text
Fluid Dynamic DNNs - Accepted Manuscript
Restricted to Repository staff only until 27 March 2024.
Available under License Creative Commons Attribution.
Request a copy

More information

Accepted/In Press date: 12 January 2024
Venue - Dates: Design, Automation and Test in Europe, , Valencia, Spain, 2024-03-25 - 2024-03-27

Identifiers

Local EPrints ID: 486257
URI: http://eprints.soton.ac.uk/id/eprint/486257
PURE UUID: bed7e9a4-0c96-40c4-85ea-75c27a10a5f1
ORCID for Jonathon Hare: ORCID iD orcid.org/0000-0003-2921-4283
ORCID for Geoff V. Merrett: ORCID iD orcid.org/0000-0003-4980-3894

Catalogue record

Date deposited: 16 Jan 2024 17:33
Last modified: 17 Jan 2024 02:40

Export record

Contributors

Author: Lei Xun
Author: Mingyu Hu
Author: Hengrui Zhao
Author: Amit Kumar Singh
Author: Jonathon Hare ORCID iD
Author: Geoff V. Merrett ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×