Gauss-Newton-type methods for bilevel optimization

Abstract

This document is designed to provide an overview of the three papers that analyze application of Gauss-Newton-type methods to find solutions of bilevel programming problems. [Paper 1] and [Paper 2] consider the framework for nonlinear bilevel problems, while [Paper 3] extends the analysis to the linear class of bilevel problems. All of the papers are mainly based on the lower-level
value function (LLVF) reformulation of bilevel programming problems. With some appropriate assumptions optimality conditions are stated in the context of this reformulation for nonlinear case in [Paper 1, Paper 2] and for linear case in [Paper 3]. In [Paper 3] LLVF reformulation is compared with Karush-Kuhn-Tucker (KKT) reformulation as the latter approach is very popular to solve linear bilevel problems (BLPs). There we also discuss which assumptions are automatically satisfied for these reformulations for the linear class of bilevel problems. In all three works NCP-functions are used to substitute complementarity constraints and state optimality conditions in the form of a system of equations. Smoothing is then discussed to be the best technique to obtain differentiability of such system. One of the main property of our framework is that introduced system is overdetermined. As Jacobian of such system is non-square, Gauss-Newton-type methods are considered to solve the system. In particular, [Paper 1] examines Gauss-Newton method and Newton method with Moore-Penrose pseudo inverse in the context of bilevel optimization. Levenberg-Marquardt method for nonlinear bilevel problems is discussed in [Paper 2] together with the detailed discussion on the parameters choice. In [Paper 3] Levenberg-Marquardt method is analyzed for the class of linear bilevel optimization problems. It is worth noting that these well-known methods have not yet been studied for bilevel optimization framework. We prove that all these methods can be well-defined
for the introduced formulation of optimality conditions. In numerical part of the papers we present the implementation results of the methods for a good number of problems, choosing different value of penalty parameter λ on the go. The analysis has shown that all these methods perform well, recovering known optimal solutions for most of the tested examples with very small average CPU
time required by the algorithms. This demonstrates that not only the methods are valid for the chosen framework, but also that they can compete with popular methods used in bilevel optimization. Further, results of [Paper 1] has shown that Newton method with pseudo inverse could very well be a better option to implement in practice than Gauss-Newton method in its classic formulation.
Analysis in [Paper 2] has shown that introduced Levenberg-Marquardt algorithm is very sensitive to the choice of the penalty parameter. The results there lead to some non-trivial suggestions on choosing penalty parameter, which could be useful for anyone dealing with a framework of similar nature. In numerical part of [Paper 3], it has been shown that algorithm performs slightly better for LLVF-based approach than for KKT-based approach for the linear bilevel problems. However, the comparison was only done for 24 linear test problems of relatively small size. We believe that more extensive experiments would be more reliable. For this reason, we create the basis for such comparison by transforming 50 integer examples and 124 binary examples to ordinary linear bilevel problems. Finally, results of implementing Levenberg-Marquardt method for KKT and LLVF reformulations of these problems is presented in the final section of [Paper 3].

More information

Identifiers

ORCID for Alain Zemkoho: orcid.org/0000-0003-1265-4178

ORCID for Joerg Fliege: orcid.org/0000-0002-4459-5419

ORCID for Shenglong Zhou: orcid.org/0000-0003-2843-1614

Catalogue record

Export record