The University of Southampton
University of Southampton Institutional Repository
Warning ePrints Soton is experiencing an issue with some file downloads not being available. We are working hard to fix this. Please bear with us.

Quadratic convergence of smoothing Newton's Method for 0/1 loss optimization

Quadratic convergence of smoothing Newton's Method for 0/1 loss optimization
Quadratic convergence of smoothing Newton's Method for 0/1 loss optimization
It has been widely recognized that the $0/1$-loss function is one of the most natural choices for modelling classification errors, and it has a wide range of applications including support vector machines and $1$-bit compressed sensing.
Due to the combinatorial nature of the $0/1$-loss function, methods based on convex relaxations or smoothing approximations have dominated the existing research and are often able to provide approximate solutions of good quality.
However, those methods are not optimizing the $0/1$-loss function directly and hence no optimality has been established for the original problem. This paper aims to study the optimality conditions of the $0/1$ function minimization, and for the first time to develop Newton's method that directly optimizes the $0/1$ function with a local quadratic convergence under reasonable conditions. Extensive numerical experiments demonstrate its superior performance as one would expect from Newton-type methods.
$0/1$-loss function, optimality conditions, Newton's method, locally quadratic convergence
1052-6234
1-28
Zhou, Shenglong
d183edc9-a9f6-4b07-a140-a82213dbd8c3
Pan, Lili
b3d275cf-f42f-49ca-a927-bbb1ae0812ba
Xiu, Naihua
8b5770f7-ae35-4dbe-884a-02fb4ea27bee
Qi, Hou-Duo
e9789eb9-c2bc-4b63-9acb-c7e753cc9a85
Zhou, Shenglong
d183edc9-a9f6-4b07-a140-a82213dbd8c3
Pan, Lili
b3d275cf-f42f-49ca-a927-bbb1ae0812ba
Xiu, Naihua
8b5770f7-ae35-4dbe-884a-02fb4ea27bee
Qi, Hou-Duo
e9789eb9-c2bc-4b63-9acb-c7e753cc9a85

Zhou, Shenglong, Pan, Lili, Xiu, Naihua and Qi, Hou-Duo (2021) Quadratic convergence of smoothing Newton's Method for 0/1 loss optimization. SIAM Journal on Optimization, 1-28. (In Press)

Record type: Article

Abstract

It has been widely recognized that the $0/1$-loss function is one of the most natural choices for modelling classification errors, and it has a wide range of applications including support vector machines and $1$-bit compressed sensing.
Due to the combinatorial nature of the $0/1$-loss function, methods based on convex relaxations or smoothing approximations have dominated the existing research and are often able to provide approximate solutions of good quality.
However, those methods are not optimizing the $0/1$-loss function directly and hence no optimality has been established for the original problem. This paper aims to study the optimality conditions of the $0/1$ function minimization, and for the first time to develop Newton's method that directly optimizes the $0/1$ function with a local quadratic convergence under reasonable conditions. Extensive numerical experiments demonstrate its superior performance as one would expect from Newton-type methods.

Text
Newton01_PURE - Accepted Manuscript
Restricted to Repository staff only
Request a copy

More information

Accepted/In Press date: 5 September 2021
Keywords: $0/1$-loss function, optimality conditions, Newton's method, locally quadratic convergence

Identifiers

Local EPrints ID: 451313
URI: http://eprints.soton.ac.uk/id/eprint/451313
ISSN: 1052-6234
PURE UUID: 482013dc-2f15-41c7-a6a1-cb59587b5da2
ORCID for Shenglong Zhou: ORCID iD orcid.org/0000-0003-2843-1614
ORCID for Hou-Duo Qi: ORCID iD orcid.org/0000-0003-3481-4814

Catalogue record

Date deposited: 20 Sep 2021 16:32
Last modified: 21 Sep 2021 01:38

Export record

Contributors

Author: Shenglong Zhou ORCID iD
Author: Lili Pan
Author: Naihua Xiu
Author: Hou-Duo Qi ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×