Structural Risk Minimization over Data-Dependent Hierarchies
Structural Risk Minimization over Data-Dependent Hierarchies
The paper introduces some generalizations of Vapnik's (1982) method of structural risk minimization (SRM). As well as making explicit some of the details on SRM, it provides a result that allows one to trade off errors on the training sample against improved generalization performance. It then considers the more general case when the hierarchy of classes is chosen in response to the data. A result is presented on the generalization performance of classifiers with a “large margin”. This theoretically explains the impressive generalization performance of the maximal margin hyperplane algorithm of Vapnik and co-workers (which is the basis for their support vector machines). The paper concludes with a more general result in terms of “luckiness” functions, which provides a quite general way for exploiting serendipitous simplicity in observed data to obtain better prediction accuracy from small training sets. Four examples are given of such functions, including the Vapnik-Chervonenkis (1971) dimension measured on the sample
approximation theory learning systems minimisation probability risk management set theory Vapnik's method Vapnik-Chervonenkis dimension classifiers data-dependent hierarchies generalization performance luckiness functions maximal margin hyperplane algorithm prediction accuracy probably approximately correct model small training sets structural risk minimization support vector machines training sample errors
1926-1940
Shawe-Taylor, J.
c32d0ee4-b422-491f-8c28-78663851d6db
Bartlett, P. L.
3a7d0643-e56a-487d-889b-0fdbb2402a29
Williamson, R. C.
62d57e88-c730-4503-9aa8-c57d61072c51
Anthony, M.
44cc9b8c-f199-4df9-a6c5-8b4a37c238b2
September 1998
Shawe-Taylor, J.
c32d0ee4-b422-491f-8c28-78663851d6db
Bartlett, P. L.
3a7d0643-e56a-487d-889b-0fdbb2402a29
Williamson, R. C.
62d57e88-c730-4503-9aa8-c57d61072c51
Anthony, M.
44cc9b8c-f199-4df9-a6c5-8b4a37c238b2
Shawe-Taylor, J., Bartlett, P. L., Williamson, R. C. and Anthony, M.
(1998)
Structural Risk Minimization over Data-Dependent Hierarchies.
IEEE Transactions on Information Theory, 44 (5), .
Abstract
The paper introduces some generalizations of Vapnik's (1982) method of structural risk minimization (SRM). As well as making explicit some of the details on SRM, it provides a result that allows one to trade off errors on the training sample against improved generalization performance. It then considers the more general case when the hierarchy of classes is chosen in response to the data. A result is presented on the generalization performance of classifiers with a “large margin”. This theoretically explains the impressive generalization performance of the maximal margin hyperplane algorithm of Vapnik and co-workers (which is the basis for their support vector machines). The paper concludes with a more general result in terms of “luckiness” functions, which provides a quite general way for exploiting serendipitous simplicity in observed data to obtain better prediction accuracy from small training sets. Four examples are given of such functions, including the Vapnik-Chervonenkis (1971) dimension measured on the sample
This record has no associated files available for download.
More information
Published date: September 1998
Keywords:
approximation theory learning systems minimisation probability risk management set theory Vapnik's method Vapnik-Chervonenkis dimension classifiers data-dependent hierarchies generalization performance luckiness functions maximal margin hyperplane algorithm prediction accuracy probably approximately correct model small training sets structural risk minimization support vector machines training sample errors
Organisations:
Electronics & Computer Science
Identifiers
Local EPrints ID: 259793
URI: http://eprints.soton.ac.uk/id/eprint/259793
ISSN: 0018-9448
PURE UUID: 8bb0830d-7ac7-42eb-9f75-48332590390f
Catalogue record
Date deposited: 17 Aug 2004
Last modified: 27 Apr 2022 12:05
Export record
Contributors
Author:
J. Shawe-Taylor
Author:
P. L. Bartlett
Author:
R. C. Williamson
Author:
M. Anthony
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics