The University of Southampton
University of Southampton Institutional Repository

RULEM: A novel heuristic rule learning approach for ordinal classification with monotonicity constraints

RULEM: A novel heuristic rule learning approach for ordinal classification with monotonicity constraints
RULEM: A novel heuristic rule learning approach for ordinal classification with monotonicity constraints
In many real world applications classification models are required to be in line with domain knowledge and to respect monotone relations between predictor variables and the target class, in order to be acceptable for implementation. This paper presents a novel heuristic approach, called RULEM, to induce monotone ordinal rule based classification models. The proposed approach can be applied in combination with any rule- or tree-based classification technique, since monotonicity is guaranteed in a post-processing step. RULEM checks whether a rule set or decision tree violates the imposed monotonicity constraints and existing violations are resolved by inducing a set of additional rules which enforce monotone classification. The approach is able to handle non-monotonic noise, and can be applied to both partially and totally monotone problems with an ordinal target variable. Two novel justifiability measures are introduced which are based on RULEM and allow to calculate the extent to which a classification model is in line with domain knowledge expressed in the form of monotonicity constraints. An extensive benchmarking experiment and subsequent statistical analysis of the results on 14 public data sets indicates that RULEM preserves the predictive power of a rule induction technique while guaranteeing monotone classification. On the other hand, the post-processed rule sets are found to be significantly larger which is due to the induction of additional rules. E.g., when combined with Ripper a median performance difference was observed in terms of PCC equal to zero and an average difference equal to −0.66%, with on average 5 rules added to the rule sets. The average and minimum justifiability of the original rule sets equal respectively 92.66% and 34.44% in terms of the RULEMF justifiability index, and 91.28% and 40.1% in terms of RULEMS, indicating the effective need for monotonizing the rule sets.
1568-4946
858-873
Verbeke, Wouter
57c0d98a-130a-4202-b6dd-cdc6914f4732
Martens, David
42e7e141-fb3d-4ead-8e3a-96b39bab65f9
Baesens, Bart
f7c6496b-aa7f-4026-8616-ca61d9e216f0
Verbeke, Wouter
57c0d98a-130a-4202-b6dd-cdc6914f4732
Martens, David
42e7e141-fb3d-4ead-8e3a-96b39bab65f9
Baesens, Bart
f7c6496b-aa7f-4026-8616-ca61d9e216f0

Verbeke, Wouter, Martens, David and Baesens, Bart (2017) RULEM: A novel heuristic rule learning approach for ordinal classification with monotonicity constraints. Applied Soft Computing, 60, 858-873. (doi:10.1016/j.asoc.2017.01.042).

Record type: Article

Abstract

In many real world applications classification models are required to be in line with domain knowledge and to respect monotone relations between predictor variables and the target class, in order to be acceptable for implementation. This paper presents a novel heuristic approach, called RULEM, to induce monotone ordinal rule based classification models. The proposed approach can be applied in combination with any rule- or tree-based classification technique, since monotonicity is guaranteed in a post-processing step. RULEM checks whether a rule set or decision tree violates the imposed monotonicity constraints and existing violations are resolved by inducing a set of additional rules which enforce monotone classification. The approach is able to handle non-monotonic noise, and can be applied to both partially and totally monotone problems with an ordinal target variable. Two novel justifiability measures are introduced which are based on RULEM and allow to calculate the extent to which a classification model is in line with domain knowledge expressed in the form of monotonicity constraints. An extensive benchmarking experiment and subsequent statistical analysis of the results on 14 public data sets indicates that RULEM preserves the predictive power of a rule induction technique while guaranteeing monotone classification. On the other hand, the post-processed rule sets are found to be significantly larger which is due to the induction of additional rules. E.g., when combined with Ripper a median performance difference was observed in terms of PCC equal to zero and an average difference equal to −0.66%, with on average 5 rules added to the rule sets. The average and minimum justifiability of the original rule sets equal respectively 92.66% and 34.44% in terms of the RULEMF justifiability index, and 91.28% and 40.1% in terms of RULEMS, indicating the effective need for monotonizing the rule sets.

Text
RULEM Rule Learning with Monotonicity Constraints for Ordinal Classification - Accepted Manuscript
Restricted to Repository staff only
Request a copy

More information

Accepted/In Press date: 26 January 2017
e-pub ahead of print date: 9 February 2017
Published date: November 2017

Identifiers

Local EPrints ID: 416586
URI: https://eprints.soton.ac.uk/id/eprint/416586
ISSN: 1568-4946
PURE UUID: 6f7e32d8-5342-4ce8-8502-1fdb7b364a1a
ORCID for Bart Baesens: ORCID iD orcid.org/0000-0002-5831-5668

Catalogue record

Date deposited: 03 Jan 2018 17:30
Last modified: 10 Sep 2019 00:45

Export record

Altmetrics

Contributors

Author: Wouter Verbeke
Author: David Martens
Author: Bart Baesens ORCID iD

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×