The University of Southampton
University of Southampton Institutional Repository

Dynamic-OFA: Runtime DNN architecture switching for performance scaling on heterogeneous embedded platforms

Dynamic-OFA: Runtime DNN architecture switching for performance scaling on heterogeneous embedded platforms
Dynamic-OFA: Runtime DNN architecture switching for performance scaling on heterogeneous embedded platforms
Mobile and embedded platforms are increasingly required to efficiently execute computationally demanding DNNs across heterogeneous processing elements. At runtime, the available hardware resources to DNNs can vary considerably due to other concurrently running applications. The performance requirements of the applications could also change under different scenarios. To achieve the desired performance, dynamic DNNs have been proposed in which the number of channels/layers can be scaled in real time to meet different requirements under varying resource constraints. However, the training process of such dynamic DNNs can be costly, since platform-aware models of different deployment scenarios must be retrained to become dynamic. This paper proposes Dynamic-OFA, a novel dynamic DNN approach for state-of-the-art platform-aware NAS models (i.e. Once-for-all network (OFA)). Dynamic-OFA pre-samples a family of sub-networks from a static OFA backbone model, and contains a runtime manager to choose different sub-networks under different runtime environments. As such, Dynamic-OFA does not need the traditional dynamic DNN training pipeline. Compared to the state-of-the-art, our experimental results using ImageNet on a Jetson Xavier NX show that the approach is up to 3.5x (CPU), 2.4x (GPU) faster for similar Top-1 accuracy, or 3.8% (CPU), 5.1% (GPU) higher accuracy at similar latency.
2160-7508
3104-3112
Lou, Wei
016731cf-d26e-4c03-b722-72519754d9e9
Xun, Lei
51a0da82-6979-49a8-8eff-ada011f5aff5
Sabetsarvestani, Mohammadamin
f5c0e55f-6f0c-4f56-9d6d-7de19d6fb136
Bi, Jia
8b23da1b-a6d6-43f4-9752-04a825093b3b
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Merrett, Geoff
89b3a696-41de-44c3-89aa-b0aa29f54020
Lou, Wei
016731cf-d26e-4c03-b722-72519754d9e9
Xun, Lei
51a0da82-6979-49a8-8eff-ada011f5aff5
Sabetsarvestani, Mohammadamin
f5c0e55f-6f0c-4f56-9d6d-7de19d6fb136
Bi, Jia
8b23da1b-a6d6-43f4-9752-04a825093b3b
Hare, Jonathon
65ba2cda-eaaf-4767-a325-cd845504e5a9
Merrett, Geoff
89b3a696-41de-44c3-89aa-b0aa29f54020

Lou, Wei, Xun, Lei, Sabetsarvestani, Mohammadamin, Bi, Jia, Hare, Jonathon and Merrett, Geoff (2021) Dynamic-OFA: Runtime DNN architecture switching for performance scaling on heterogeneous embedded platforms. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2021. pp. 3104-3112 . (doi:10.1109/CVPRW53098.2021.00347).

Record type: Conference or Workshop Item (Paper)

Abstract

Mobile and embedded platforms are increasingly required to efficiently execute computationally demanding DNNs across heterogeneous processing elements. At runtime, the available hardware resources to DNNs can vary considerably due to other concurrently running applications. The performance requirements of the applications could also change under different scenarios. To achieve the desired performance, dynamic DNNs have been proposed in which the number of channels/layers can be scaled in real time to meet different requirements under varying resource constraints. However, the training process of such dynamic DNNs can be costly, since platform-aware models of different deployment scenarios must be retrained to become dynamic. This paper proposes Dynamic-OFA, a novel dynamic DNN approach for state-of-the-art platform-aware NAS models (i.e. Once-for-all network (OFA)). Dynamic-OFA pre-samples a family of sub-networks from a static OFA backbone model, and contains a runtime manager to choose different sub-networks under different runtime environments. As such, Dynamic-OFA does not need the traditional dynamic DNN training pipeline. Compared to the state-of-the-art, our experimental results using ImageNet on a Jetson Xavier NX show that the approach is up to 3.5x (CPU), 2.4x (GPU) faster for similar Top-1 accuracy, or 3.8% (CPU), 5.1% (GPU) higher accuracy at similar latency.

Text
Dynamic-OFA_CVPR’W 2021_Accepted - Accepted Manuscript
Download (558kB)

More information

Accepted/In Press date: 20 April 2021
Published date: 20 June 2021
Additional Information: Funding Information: This work was supported in part by the Engineering and Physical Sciences Research Council (EPSRC) under Grant EP/S030069/1. Experimental data can be found at: https://doi.org/10.5258/SOTON/D1804. Code is available open-source on https://github.com/UoS-EEC. Publisher Copyright: © 2021 IEEE.

Identifiers

Local EPrints ID: 448645
URI: http://eprints.soton.ac.uk/id/eprint/448645
ISSN: 2160-7508
PURE UUID: 7d0ba869-04da-416e-8ba3-7c015c9f526e
ORCID for Jonathon Hare: ORCID iD orcid.org/0000-0003-2921-4283
ORCID for Geoff Merrett: ORCID iD orcid.org/0000-0003-4980-3894

Catalogue record

Date deposited: 29 Apr 2021 16:30
Last modified: 17 Mar 2024 03:05

Export record

Altmetrics

Contributors

Author: Wei Lou
Author: Lei Xun
Author: Mohammadamin Sabetsarvestani
Author: Jia Bi
Author: Jonathon Hare ORCID iD
Author: Geoff Merrett ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×