Detecting operational adversarial examples for reliable deep learning

The utilisation of Deep Learning (DL) raises new challenges regarding its dependability in critical applications. Sound verification and validation methods are needed to assure the safe and reliable use of DL. However, state-of-The-Art debug testing methods on DL that aim at detecting adversarial examples (AEs) ignore the operational profile, which statistically depicts the software's future operational use. This may lead to very modest effectiveness on improving the software's delivered reliability, as the testing budget is likely to be wasted on detecting AEs that are unrealistic or encountered very rarely in real-life operation. In this paper, we first present the novel notion of 'operational AEs' which are AEs that have relatively high chance to be seen in future operation. Then an initial design of a new DL testing method to efficiently detect 'operational AEs' is provided, as well as some insights on our prospective research plan.

Deep Learning robustness, operational profile, robustness testing, safe AI, software reliability, software testing

10.1109/DSN-S52858.2021.00013

5-6

IEEE

Zhao, Xingyu

56d69104-77e5-4741-bca1-c0fa13f433fe

Huang, Wei

bd1464ed-9914-4bab-8eb0-37e1bd50f9bf

Schewe, Sven

eabccdbc-088f-4bc7-b101-1b83af6b6185

Dong, Yi

355a62d9-5d1a-4c14-a900-9911e8c62453

Huang, Xiaowei

ea80b217-6df4-4708-970d-93303f2a17e5

2 September 2021

Zhao, Xingyu

56d69104-77e5-4741-bca1-c0fa13f433fe

Huang, Wei

bd1464ed-9914-4bab-8eb0-37e1bd50f9bf

Schewe, Sven

eabccdbc-088f-4bc7-b101-1b83af6b6185

Dong, Yi

355a62d9-5d1a-4c14-a900-9911e8c62453

Huang, Xiaowei

ea80b217-6df4-4708-970d-93303f2a17e5

Zhao, Xingyu, Huang, Wei, Schewe, Sven, Dong, Yi and Huang, Xiaowei (2021) Detecting operational adversarial examples for reliable deep learning. In Proceedings of the 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume, DSN-S 2021. IEEE. pp. 5-6 . (doi:10.1109/DSN-S52858.2021.00013).

Record type: Conference or Workshop Item (Paper)

Abstract

This record has no associated files available for download.

More information

Published date: 2 September 2021

Additional Information: Funding Information: This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 956123. This work is partially supported by the UK Dstl (through the project of Safety Argument for Learning-enabled Autonomous Underwater Vehicles). XZ’s contribution is partially supported through Fellowships at the Assuring Autonomy International Programme. We thank Lorenzo Strigini for his insightful comments.

Venue - Dates: 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume, DSN-S 2021, , Virtual, Taipei, Taiwan, 2021-06-21 - 2021-06-24

Keywords: Deep Learning robustness, operational profile, robustness testing, safe AI, software reliability, software testing