Strengthened Circle and Popov Criteria and the analysis of ReLU neural networks
Strengthened Circle and Popov Criteria and the analysis of ReLU neural networks
Many systems involving neural networks (NNs) can be framed as Lurie systems: feedback systems consisting of a linear time-invariant (LTI) part and a static nonlinearity. Examples of these include the interconnection of LTI systems with L-layer feedforward NNs [1], [2] and continuous time recurrent neural networks (RNN) [3]. Stability analysis of a Lurie system lends itself to a range of criteria from absolute stability; however, in NN analysis, the size of m (see Fig. 1 where u, y in Rem) is typically large. As a result, existing absolute stability criteria suffer from greater conservatism and/or computational complexity [4]. This paper addresses this problem by strengthening the low complexity classical Circle and Popov Criteria for the specialised case of the repeated ReLU nonlinearity (a popular NN activation function). The results are cast as a set of linear matrix inequalities (LMIs) with less restrictive conditions on the matrix variables than their classical counterparts. A full version of this paper has recently been in published in [5].
LMIs, Lyapunov methods, Neural networks, Robust control, stability of nonlinear systems
127-128
Richardson, Carl R.
3406b6af-f00d-410b-8051-a0ecc27baba5
Turner, Matthew C.
6befa01e-0045-4806-9c91-a107c53acba0
Gunn, Steve R.
306af9b3-a7fa-4381-baf9-5d6a6ec89868
22 May 2024
Richardson, Carl R.
3406b6af-f00d-410b-8051-a0ecc27baba5
Turner, Matthew C.
6befa01e-0045-4806-9c91-a107c53acba0
Gunn, Steve R.
306af9b3-a7fa-4381-baf9-5d6a6ec89868
Richardson, Carl R., Turner, Matthew C. and Gunn, Steve R.
(2024)
Strengthened Circle and Popov Criteria and the analysis of ReLU neural networks.
In 2024 UKACC 14th International Conference on Control, CONTROL 2024.
IEEE.
.
(doi:10.1109/CONTROL60310.2024.10531900).
Record type:
Conference or Workshop Item
(Paper)
Abstract
Many systems involving neural networks (NNs) can be framed as Lurie systems: feedback systems consisting of a linear time-invariant (LTI) part and a static nonlinearity. Examples of these include the interconnection of LTI systems with L-layer feedforward NNs [1], [2] and continuous time recurrent neural networks (RNN) [3]. Stability analysis of a Lurie system lends itself to a range of criteria from absolute stability; however, in NN analysis, the size of m (see Fig. 1 where u, y in Rem) is typically large. As a result, existing absolute stability criteria suffer from greater conservatism and/or computational complexity [4]. This paper addresses this problem by strengthening the low complexity classical Circle and Popov Criteria for the specialised case of the repeated ReLU nonlinearity (a popular NN activation function). The results are cast as a set of linear matrix inequalities (LMIs) with less restrictive conditions on the matrix variables than their classical counterparts. A full version of this paper has recently been in published in [5].
Text
UKACC_EA
- Accepted Manuscript
Restricted to Repository staff only until 22 May 2026.
Request a copy
More information
Published date: 22 May 2024
Additional Information:
Funding Information:
This work was supported in part by the Defence Science and Technology Laboratory (DSTL) and in part by the U.K. Research and Innovation (UKRI) Centre of Machine Intelligence for Nano-Electronic Devices and Systems under Grant EP/S024298/1.
Publisher Copyright:
© 2024 IEEE.
Publisher Copyright:
© 2024 IEEE.
Keywords:
LMIs, Lyapunov methods, Neural networks, Robust control, stability of nonlinear systems
Identifiers
Local EPrints ID: 490608
URI: http://eprints.soton.ac.uk/id/eprint/490608
PURE UUID: 2dd1e990-b315-48ba-b538-976372c34f58
Catalogue record
Date deposited: 31 May 2024 16:37
Last modified: 25 Jul 2024 02:02
Export record
Altmetrics
Contributors
Author:
Carl R. Richardson
Author:
Matthew C. Turner
Author:
Steve R. Gunn
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics