The University of Southampton
University of Southampton Institutional Repository

Structural Risk Minimization over Data-Dependent Hierarchies

Structural Risk Minimization over Data-Dependent Hierarchies
Structural Risk Minimization over Data-Dependent Hierarchies
The paper introduces some generalizations of Vapnik's (1982) method of structural risk minimization (SRM). As well as making explicit some of the details on SRM, it provides a result that allows one to trade off errors on the training sample against improved generalization performance. It then considers the more general case when the hierarchy of classes is chosen in response to the data. A result is presented on the generalization performance of classifiers with a “large margin”. This theoretically explains the impressive generalization performance of the maximal margin hyperplane algorithm of Vapnik and co-workers (which is the basis for their support vector machines). The paper concludes with a more general result in terms of “luckiness” functions, which provides a quite general way for exploiting serendipitous simplicity in observed data to obtain better prediction accuracy from small training sets. Four examples are given of such functions, including the Vapnik-Chervonenkis (1971) dimension measured on the sample
approximation theory learning systems minimisation probability risk management set theory Vapnik's method Vapnik-Chervonenkis dimension classifiers data-dependent hierarchies generalization performance luckiness functions maximal margin hyperplane algorithm prediction accuracy probably approximately correct model small training sets structural risk minimization support vector machines training sample errors
0018-9448
1926-1940
Shawe-Taylor, J.
c32d0ee4-b422-491f-8c28-78663851d6db
Bartlett, P. L.
3a7d0643-e56a-487d-889b-0fdbb2402a29
Williamson, R. C.
62d57e88-c730-4503-9aa8-c57d61072c51
Anthony, M.
44cc9b8c-f199-4df9-a6c5-8b4a37c238b2
Shawe-Taylor, J.
c32d0ee4-b422-491f-8c28-78663851d6db
Bartlett, P. L.
3a7d0643-e56a-487d-889b-0fdbb2402a29
Williamson, R. C.
62d57e88-c730-4503-9aa8-c57d61072c51
Anthony, M.
44cc9b8c-f199-4df9-a6c5-8b4a37c238b2

Shawe-Taylor, J., Bartlett, P. L., Williamson, R. C. and Anthony, M. (1998) Structural Risk Minimization over Data-Dependent Hierarchies. IEEE Transactions on Information Theory, 44 (5), 1926-1940.

Record type: Article

Abstract

The paper introduces some generalizations of Vapnik's (1982) method of structural risk minimization (SRM). As well as making explicit some of the details on SRM, it provides a result that allows one to trade off errors on the training sample against improved generalization performance. It then considers the more general case when the hierarchy of classes is chosen in response to the data. A result is presented on the generalization performance of classifiers with a “large margin”. This theoretically explains the impressive generalization performance of the maximal margin hyperplane algorithm of Vapnik and co-workers (which is the basis for their support vector machines). The paper concludes with a more general result in terms of “luckiness” functions, which provides a quite general way for exploiting serendipitous simplicity in observed data to obtain better prediction accuracy from small training sets. Four examples are given of such functions, including the Vapnik-Chervonenkis (1971) dimension measured on the sample

Full text not available from this repository.

More information

Published date: September 1998
Keywords: approximation theory learning systems minimisation probability risk management set theory Vapnik's method Vapnik-Chervonenkis dimension classifiers data-dependent hierarchies generalization performance luckiness functions maximal margin hyperplane algorithm prediction accuracy probably approximately correct model small training sets structural risk minimization support vector machines training sample errors
Organisations: Electronics & Computer Science

Identifiers

Local EPrints ID: 259793
URI: http://eprints.soton.ac.uk/id/eprint/259793
ISSN: 0018-9448
PURE UUID: 8bb0830d-7ac7-42eb-9f75-48332590390f

Catalogue record

Date deposited: 17 Aug 2004
Last modified: 16 Jul 2019 22:52

Export record

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×