The University of Southampton
University of Southampton Institutional Repository

Subspace Newton method for sparse SVM

Subspace Newton method for sparse SVM
Subspace Newton method for sparse SVM
Kernel-based methods for support vector machines (SVM) have seen a great advantage in various applications. However, they may incur prohibitive computational costs when the involved sample size is on a large scale. Therefore, reducing the number of support vectors (or say sample reduction) appears to be necessary, which gives rise to the topic of the sparse SVM. Motivated by this, we aim at solving a sparsity constrained kernel SVM optimization, which is capable of controlling the number of the support vectors. Based on the established optimality conditions associated with the stationary equations, a subspace Newton method is cast to tackle the sparsity constrained problem and enjoys one-step convergence property if the starting point is close to a local region of a stationary point, leading to a super-fast computational speed. Numerical comparisons with some other excellent solvers demonstrate that the proposed method performs exceptionally well, especially for datasets with large numbers of samples, in terms of a much fewer number of support vectors and shorter computational time.
Zhou, Shenglong
d183edc9-a9f6-4b07-a140-a82213dbd8c3
Zhou, Shenglong
d183edc9-a9f6-4b07-a140-a82213dbd8c3

Zhou, Shenglong (2020) Subspace Newton method for sparse SVM. arXiv, (2005.13771). (In Press)

Record type: Article

Abstract

Kernel-based methods for support vector machines (SVM) have seen a great advantage in various applications. However, they may incur prohibitive computational costs when the involved sample size is on a large scale. Therefore, reducing the number of support vectors (or say sample reduction) appears to be necessary, which gives rise to the topic of the sparse SVM. Motivated by this, we aim at solving a sparsity constrained kernel SVM optimization, which is capable of controlling the number of the support vectors. Based on the established optimality conditions associated with the stationary equations, a subspace Newton method is cast to tackle the sparsity constrained problem and enjoys one-step convergence property if the starting point is close to a local region of a stationary point, leading to a super-fast computational speed. Numerical comparisons with some other excellent solvers demonstrate that the proposed method performs exceptionally well, especially for datasets with large numbers of samples, in terms of a much fewer number of support vectors and shorter computational time.

Text
SVM-Newton - Author's Original
Download (799kB)

More information

Accepted/In Press date: 3 July 2020

Identifiers

Local EPrints ID: 443627
URI: http://eprints.soton.ac.uk/id/eprint/443627
PURE UUID: babe012b-2ace-468e-aa67-831be8bd639b
ORCID for Shenglong Zhou: ORCID iD orcid.org/0000-0003-2843-1614

Catalogue record

Date deposited: 04 Sep 2020 16:34
Last modified: 16 Mar 2024 08:30

Export record

Contributors

Author: Shenglong Zhou ORCID iD

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×