Covering Numbers for Support Vector Machines
Covering Numbers for Support Vector Machines
Support vector (SV) machines are linear classifiers that use the maximum margin hyperplane in a feature space defined by a kernel function. Until recently, the only bounds on the generalization performance of SV machines (within Valiant’s probably approximately correct framework) took no account of the kernel used except in its effect on the margin and radius. More recently, it has been shown that one can bound the relevant covering numbers using tools from functional analysis. In this paper, we show that the resulting bound can be greatly simplified. The new bound involves the eigenvalues of the integral operator induced by the kernel. It shows that the effective dimension depends on the rate of decay of these eigenvalues. We present an explicit calculation of covering numbers for an SV machine using a Gaussian kernel, which is significantly better than that implied by previous results.
Covering numbers, entropy numbers, kernel machines, statistical learning theory, support vector (SV) machines.
1-58113-167-4
267-277
Association for Computing Machinery
Guo, Ying
7f152e2a-4203-4d54-81fa-7888bfe77b11
Bartlett, Peter L.
41a9704e-e7ce-4c9f-8418-86563c1d67f9
Shawe-Taylor, John
b1931d97-fdd0-4bc1-89bc-ec01648e928b
Williamson, Robert C.
74ed9ea2-d4bc-4355-aadd-3589150f3dfb
1999
Guo, Ying
7f152e2a-4203-4d54-81fa-7888bfe77b11
Bartlett, Peter L.
41a9704e-e7ce-4c9f-8418-86563c1d67f9
Shawe-Taylor, John
b1931d97-fdd0-4bc1-89bc-ec01648e928b
Williamson, Robert C.
74ed9ea2-d4bc-4355-aadd-3589150f3dfb
Guo, Ying, Bartlett, Peter L., Shawe-Taylor, John and Williamson, Robert C.
(1999)
Covering Numbers for Support Vector Machines.
In Proceedings of COLT'99.
Association for Computing Machinery.
.
Record type:
Conference or Workshop Item
(Paper)
Abstract
Support vector (SV) machines are linear classifiers that use the maximum margin hyperplane in a feature space defined by a kernel function. Until recently, the only bounds on the generalization performance of SV machines (within Valiant’s probably approximately correct framework) took no account of the kernel used except in its effect on the margin and radius. More recently, it has been shown that one can bound the relevant covering numbers using tools from functional analysis. In this paper, we show that the resulting bound can be greatly simplified. The new bound involves the eigenvalues of the integral operator induced by the kernel. It shows that the effective dimension depends on the rate of decay of these eigenvalues. We present an explicit calculation of covering numbers for an SV machine using a Gaussian kernel, which is significantly better than that implied by previous results.
Text
CoveringNumbersForSupportVectorMachines.pdf
- Other
More information
Published date: 1999
Additional Information:
Also published in IEEE Transactions on Information Theory, Vol 48, No 1, January 2002
Keywords:
Covering numbers, entropy numbers, kernel machines, statistical learning theory, support vector (SV) machines.
Organisations:
Electronics & Computer Science
Identifiers
Local EPrints ID: 259653
URI: http://eprints.soton.ac.uk/id/eprint/259653
ISBN: 1-58113-167-4
PURE UUID: 16241056-58d8-43ef-8327-355423c33ac4
Catalogue record
Date deposited: 04 Aug 2004
Last modified: 14 Mar 2024 06:27
Export record
Contributors
Author:
Ying Guo
Author:
Peter L. Bartlett
Author:
John Shawe-Taylor
Author:
Robert C. Williamson
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics