The University of Southampton
University of Southampton Institutional Repository

A neural network architecture optimizer based on DARTS and generative adversarial learning

A neural network architecture optimizer based on DARTS and generative adversarial learning
A neural network architecture optimizer based on DARTS and generative adversarial learning
Neural network architecture search automatically configures a set of network architectures according to the targeted rules. Thus, it relieves the human-dependent effort and repetitive resources consumption for designing neural network architectures and makes the task of finding the optimum network architecture with better performance much more accessible. Network architecture search methods based on differentiable architecture search (DARTS), however, introduces parameter redundancy. To address this issue, this work presents a novel method for optimizing network architectures that combines DARTS with generative adversarial learning (GAL). We first find the module structures utilizing the DARTS algorithm. Afterwards, the retrieved modules are stacked to derive the initial neural network architecture. Next, the GAL is used to prune some branches of the initial neural network, thereby obtaining the final neural network architecture. The proposed DARTS-GAL method re-optimizes the network architecture searched by DARTS to simplify the network connection and reduce network parameters without compromising network performance. Experimental results on benchmark datasets, i.e., Mixed National Institute of Standards and Technology (MNIST), FashionMNIST, Canadian Institute for Advanced Research10 (CIFAR10), Canadian Institute for Advanced Research100 (CIAFR100), Cats vs Dogs, and voiceprint recognition datasets, indicate that the test accuracies of the DARTS-GAL are higher than those of the DARTS in the majority of the cases. In particular, the proposed solution exhibits an improvement in accuracy by 7.35% on CIFAR10 compared with DARTS, attaining the state-of-the-art result of 99.60%. Additionally, the number of network parameters derived by the DARTS-GAL is significantly lower than that by the DARTS method, with a pruning rate of 62.3% at the highest case.
Deep learning, Differentiable architecture search (DARTS), Generative adversarial learning (GAL), Neural network architecture search, Pruning
0020-0255
448-468
Zhang, Ting
9afe33e5-998b-4b25-962e-2b2d376d062a
Waqas, Muhammad
ddc5a9a9-4c75-4f24-bb31-18f1974a0560
Shen, Hao
a165a3ce-bda9-441a-8226-21a6a2b91821
Liu, Zhaoying
20c67d51-e992-4083-ab63-6dec8a97c4a3
Zhang, Xiangyu
f450adff-04de-4e27-87e0-e3c570f5234f
Li, Yujian
77efa252-9f54-4f2d-a7ea-8ef79b42978f
Halim, Zahid
f0286481-b905-4851-be41-ffda91535148
Chen, Sheng
9310a111-f79a-48b8-98c7-383ca93cbb80
Zhang, Ting
9afe33e5-998b-4b25-962e-2b2d376d062a
Waqas, Muhammad
ddc5a9a9-4c75-4f24-bb31-18f1974a0560
Shen, Hao
a165a3ce-bda9-441a-8226-21a6a2b91821
Liu, Zhaoying
20c67d51-e992-4083-ab63-6dec8a97c4a3
Zhang, Xiangyu
f450adff-04de-4e27-87e0-e3c570f5234f
Li, Yujian
77efa252-9f54-4f2d-a7ea-8ef79b42978f
Halim, Zahid
f0286481-b905-4851-be41-ffda91535148
Chen, Sheng
9310a111-f79a-48b8-98c7-383ca93cbb80

Zhang, Ting, Waqas, Muhammad, Shen, Hao, Liu, Zhaoying, Zhang, Xiangyu, Li, Yujian, Halim, Zahid and Chen, Sheng (2021) A neural network architecture optimizer based on DARTS and generative adversarial learning. Information Sciences, 581, 448-468.

Record type: Article

Abstract

Neural network architecture search automatically configures a set of network architectures according to the targeted rules. Thus, it relieves the human-dependent effort and repetitive resources consumption for designing neural network architectures and makes the task of finding the optimum network architecture with better performance much more accessible. Network architecture search methods based on differentiable architecture search (DARTS), however, introduces parameter redundancy. To address this issue, this work presents a novel method for optimizing network architectures that combines DARTS with generative adversarial learning (GAL). We first find the module structures utilizing the DARTS algorithm. Afterwards, the retrieved modules are stacked to derive the initial neural network architecture. Next, the GAL is used to prune some branches of the initial neural network, thereby obtaining the final neural network architecture. The proposed DARTS-GAL method re-optimizes the network architecture searched by DARTS to simplify the network connection and reduce network parameters without compromising network performance. Experimental results on benchmark datasets, i.e., Mixed National Institute of Standards and Technology (MNIST), FashionMNIST, Canadian Institute for Advanced Research10 (CIFAR10), Canadian Institute for Advanced Research100 (CIAFR100), Cats vs Dogs, and voiceprint recognition datasets, indicate that the test accuracies of the DARTS-GAL are higher than those of the DARTS in the majority of the cases. In particular, the proposed solution exhibits an improvement in accuracy by 7.35% on CIFAR10 compared with DARTS, attaining the state-of-the-art result of 99.60%. Additionally, the number of network parameters derived by the DARTS-GAL is significantly lower than that by the DARTS method, with a pruning rate of 62.3% at the highest case.

Text
InfSci2021-12 - Author's Original
Download (4MB)
Text
InformationSciencesPaper - Accepted Manuscript
Restricted to Repository staff only until 16 September 2023.
Request a copy

More information

e-pub ahead of print date: 17 September 2021
Published date: December 2021
Keywords: Deep learning, Differentiable architecture search (DARTS), Generative adversarial learning (GAL), Neural network architecture search, Pruning

Identifiers

Local EPrints ID: 453382
URI: http://eprints.soton.ac.uk/id/eprint/453382
ISSN: 0020-0255
PURE UUID: 7f0df7bd-db6f-494b-a134-279dec494e3f

Catalogue record

Date deposited: 13 Jan 2022 18:18
Last modified: 27 Apr 2022 11:44

Export record

Contributors

Author: Ting Zhang
Author: Muhammad Waqas
Author: Hao Shen
Author: Zhaoying Liu
Author: Xiangyu Zhang
Author: Yujian Li
Author: Zahid Halim
Author: Sheng Chen

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×