The University of Southampton
University of Southampton Institutional Repository

Enlarging the Margins in Perceptron Decision Trees

Enlarging the Margins in Perceptron Decision Trees
Enlarging the Margins in Perceptron Decision Trees
Capacity control in perceptron decision trees is typically performed by controlling their size. We prove that other quantities can be as relevant to reduce their flexibility and combat overfitting. In particular, we provide an upper bound on the generalization error which depends both on the size of the tree and on the margin of the decision nodes. So enlarging the margin in perceptron decision trees will reduce the upper bound on generalization error. Based on this analysis, we introduce three new algorithms, which can induce large margin perceptron decision trees. To assess the effect of the large margin bias, OC1 (Journal of Artificial Intelligence Research, 1994, 2, 1–32.) of Murthy, Kasif and Salzberg, a well-known system for inducing perceptron decision trees, is used as the baseline algorithm. An extensive experimental study on real world data showed that all three new algorithms perform better or at least not significantly worse than OC1 on almost every dataset with only one exception. OC1 performed worse than the best margin-based method on every dataset.
capacity control, decision trees, perceptron, learning theory, learning algorithm
295-313
Wu, D.
325ef387-856f-49f6-bc2d-2ef44d7e6f82
Bennett, K.P.
1bed919b-5ab0-478d-a90d-bdad737efb55
Cristianini, N.
00885da7-7833-4f0c-b8a0-3f385d89f642
Shawe-Taylor, J.
c32d0ee4-b422-491f-8c28-78663851d6db
Wu, D.
325ef387-856f-49f6-bc2d-2ef44d7e6f82
Bennett, K.P.
1bed919b-5ab0-478d-a90d-bdad737efb55
Cristianini, N.
00885da7-7833-4f0c-b8a0-3f385d89f642
Shawe-Taylor, J.
c32d0ee4-b422-491f-8c28-78663851d6db

Wu, D., Bennett, K.P., Cristianini, N. and Shawe-Taylor, J. (2000) Enlarging the Margins in Perceptron Decision Trees. Machine Learning, 41 (3), 295-313.

Record type: Article

Abstract

Capacity control in perceptron decision trees is typically performed by controlling their size. We prove that other quantities can be as relevant to reduce their flexibility and combat overfitting. In particular, we provide an upper bound on the generalization error which depends both on the size of the tree and on the margin of the decision nodes. So enlarging the margin in perceptron decision trees will reduce the upper bound on generalization error. Based on this analysis, we introduce three new algorithms, which can induce large margin perceptron decision trees. To assess the effect of the large margin bias, OC1 (Journal of Artificial Intelligence Research, 1994, 2, 1–32.) of Murthy, Kasif and Salzberg, a well-known system for inducing perceptron decision trees, is used as the baseline algorithm. An extensive experimental study on real world data showed that all three new algorithms perform better or at least not significantly worse than OC1 on almost every dataset with only one exception. OC1 performed worse than the best margin-based method on every dataset.

Text
EnlargingTheMarginsInPerceptronDecisionTrees.pdf - Other
Download (184kB)

More information

Published date: December 2000
Keywords: capacity control, decision trees, perceptron, learning theory, learning algorithm
Organisations: Electronics & Computer Science

Identifiers

Local EPrints ID: 259791
URI: http://eprints.soton.ac.uk/id/eprint/259791
PURE UUID: 76dfc956-576e-418d-be82-987edec44e6d

Catalogue record

Date deposited: 02 Mar 2005
Last modified: 14 Mar 2024 06:28

Export record

Contributors

Author: D. Wu
Author: K.P. Bennett
Author: N. Cristianini
Author: J. Shawe-Taylor

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×