The University of Southampton
University of Southampton Institutional Repository

Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces

Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces
Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces
We consider monotone inclusions defined on a Hilbert space where the operator is given by the sum of a maximal monotone operator T and a single-valued monotone, Lipschitz continuous, and expectation-valued operator V. We draw motivation from the seminal work by Attouch and Cabot (Attouch in AMO 80:547–598, 2019, Attouch in MP 184: 243–287) on relaxed inertial methods for monotone inclusions and present a stochastic extension of the relaxed inertial forward–backward-forward method. Facilitated by an online variance reduction strategy via a mini-batch approach, we show that our method produces a sequence that weakly converges to the solution set. Moreover, it is possible to estimate the rate at which the discrete velocity of the stochastic process vanishes. Under strong monotonicity, we demonstrate strong convergence, and give a detailed assessment of the iteration and oracle complexity of the scheme. When the mini-batch is raised at a geometric (polynomial) rate, the rate statement can be strengthened to a linear (suitable polynomial) rate while the oracle complexity of computing an ϵ-solution improves to O(1/ϵ). Importantly, the latter claim allows for possibly biased oracles, a key theoretical advancement allowing for far broader applicability. By defining a restricted gap function based on the Fitzpatrick function, we prove that the expected gap of an averaged sequence diminishes at a sublinear rate of O(1/k) while the oracle complexity of computing a suitably defined ϵ-solution is O(1/ϵ1+a) where a>1. Numerical results on two-stage games and an overlapping group Lasso problem illustrate the advantages of our method compared to competitors.
Complexity, Dynamic sampling, Monotone operator splitting, Stochastic approximation, Variance reduction
0926-6003
Cui, Shisheng
56dd0856-dc0f-40ad-8c55-9fdbac4aa1e8
Shanbhag, Uday
9d7babef-cf38-4d51-a969-447a188ba0d5
Staudigl, Mathias
d5078a05-5eaf-48ee-9789-360d4f27902d
Vuong, Phan Tu
52577e5d-ebe9-4a43-b5e7-68aa06cfdcaf
Cui, Shisheng
56dd0856-dc0f-40ad-8c55-9fdbac4aa1e8
Shanbhag, Uday
9d7babef-cf38-4d51-a969-447a188ba0d5
Staudigl, Mathias
d5078a05-5eaf-48ee-9789-360d4f27902d
Vuong, Phan Tu
52577e5d-ebe9-4a43-b5e7-68aa06cfdcaf

Cui, Shisheng, Shanbhag, Uday, Staudigl, Mathias and Vuong, Phan Tu (2022) Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces. Computational Optimization and Applications. (doi:10.1007/s10589-022-00399-3).

Record type: Article

Abstract

We consider monotone inclusions defined on a Hilbert space where the operator is given by the sum of a maximal monotone operator T and a single-valued monotone, Lipschitz continuous, and expectation-valued operator V. We draw motivation from the seminal work by Attouch and Cabot (Attouch in AMO 80:547–598, 2019, Attouch in MP 184: 243–287) on relaxed inertial methods for monotone inclusions and present a stochastic extension of the relaxed inertial forward–backward-forward method. Facilitated by an online variance reduction strategy via a mini-batch approach, we show that our method produces a sequence that weakly converges to the solution set. Moreover, it is possible to estimate the rate at which the discrete velocity of the stochastic process vanishes. Under strong monotonicity, we demonstrate strong convergence, and give a detailed assessment of the iteration and oracle complexity of the scheme. When the mini-batch is raised at a geometric (polynomial) rate, the rate statement can be strengthened to a linear (suitable polynomial) rate while the oracle complexity of computing an ϵ-solution improves to O(1/ϵ). Importantly, the latter claim allows for possibly biased oracles, a key theoretical advancement allowing for far broader applicability. By defining a restricted gap function based on the Fitzpatrick function, we prove that the expected gap of an averaged sequence diminishes at a sublinear rate of O(1/k) while the oracle complexity of computing a suitably defined ϵ-solution is O(1/ϵ1+a) where a>1. Numerical results on two-stage games and an overlapping group Lasso problem illustrate the advantages of our method compared to competitors.

Text
s10589-022-00399-3 - Version of Record
Available under License Creative Commons Attribution.
Download (4MB)

More information

Accepted/In Press date: 30 June 2022
e-pub ahead of print date: 30 July 2022
Published date: 30 July 2022
Keywords: Complexity, Dynamic sampling, Monotone operator splitting, Stochastic approximation, Variance reduction

Identifiers

Local EPrints ID: 469068
URI: http://eprints.soton.ac.uk/id/eprint/469068
ISSN: 0926-6003
PURE UUID: 760655ea-dd2e-46cb-b64a-1128d4881ce5
ORCID for Phan Tu Vuong: ORCID iD orcid.org/0000-0002-1474-994X

Catalogue record

Date deposited: 06 Sep 2022 17:57
Last modified: 07 Sep 2022 02:00

Export record

Altmetrics

Contributors

Author: Shisheng Cui
Author: Uday Shanbhag
Author: Mathias Staudigl
Author: Phan Tu Vuong ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×