The University of Southampton
University of Southampton Institutional Repository

Covering Numbers for Support Vector Machines

Covering Numbers for Support Vector Machines
Covering Numbers for Support Vector Machines
Support vector (SV) machines are linear classifiers that use the maximum margin hyperplane in a feature space defined by a kernel function. Until recently, the only bounds on the generalization performance of SV machines (within Valiant’s probably approximately correct framework) took no account of the kernel used except in its effect on the margin and radius. More recently, it has been shown that one can bound the relevant covering numbers using tools from functional analysis. In this paper, we show that the resulting bound can be greatly simplified. The new bound involves the eigenvalues of the integral operator induced by the kernel. It shows that the effective dimension depends on the rate of decay of these eigenvalues. We present an explicit calculation of covering numbers for an SV machine using a Gaussian kernel, which is significantly better than that implied by previous results.
Covering numbers, entropy numbers, kernel machines, statistical learning theory, support vector (SV) machines.
1-58113-167-4
267-277
ACM Press
Guo, Ying
7f152e2a-4203-4d54-81fa-7888bfe77b11
Bartlett, Peter L.
41a9704e-e7ce-4c9f-8418-86563c1d67f9
Shawe-Taylor, John
b1931d97-fdd0-4bc1-89bc-ec01648e928b
Williamson, Robert C.
74ed9ea2-d4bc-4355-aadd-3589150f3dfb
Guo, Ying
7f152e2a-4203-4d54-81fa-7888bfe77b11
Bartlett, Peter L.
41a9704e-e7ce-4c9f-8418-86563c1d67f9
Shawe-Taylor, John
b1931d97-fdd0-4bc1-89bc-ec01648e928b
Williamson, Robert C.
74ed9ea2-d4bc-4355-aadd-3589150f3dfb

Guo, Ying, Bartlett, Peter L., Shawe-Taylor, John and Williamson, Robert C. (1999) Covering Numbers for Support Vector Machines. In Proceedings of COLT'99. ACM Press. pp. 267-277 .

Record type: Conference or Workshop Item (Paper)

Abstract

Support vector (SV) machines are linear classifiers that use the maximum margin hyperplane in a feature space defined by a kernel function. Until recently, the only bounds on the generalization performance of SV machines (within Valiant’s probably approximately correct framework) took no account of the kernel used except in its effect on the margin and radius. More recently, it has been shown that one can bound the relevant covering numbers using tools from functional analysis. In this paper, we show that the resulting bound can be greatly simplified. The new bound involves the eigenvalues of the integral operator induced by the kernel. It shows that the effective dimension depends on the rate of decay of these eigenvalues. We present an explicit calculation of covering numbers for an SV machine using a Gaussian kernel, which is significantly better than that implied by previous results.

Text
CoveringNumbersForSupportVectorMachines.pdf - Other
Download (380kB)

More information

Published date: 1999
Additional Information: Also published in IEEE Transactions on Information Theory, Vol 48, No 1, January 2002
Keywords: Covering numbers, entropy numbers, kernel machines, statistical learning theory, support vector (SV) machines.
Organisations: Electronics & Computer Science

Identifiers

Local EPrints ID: 259653
URI: http://eprints.soton.ac.uk/id/eprint/259653
ISBN: 1-58113-167-4
PURE UUID: 16241056-58d8-43ef-8327-355423c33ac4

Catalogue record

Date deposited: 04 Aug 2004
Last modified: 02 Dec 2019 21:09

Export record

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×