The University of Southampton
University of Southampton Institutional Repository

Performance measures of models to predict Loss Given Default: a critical review

Performance measures of models to predict Loss Given Default: a critical review
Performance measures of models to predict Loss Given Default: a critical review
The need for quantitative models in banking is largely related to the Basel Accords and, in particular, to the Advanced Internal Ratings-Based (AIRB) approach. Under the AIRB approach, lenders are allowed to use their own predictions of risk parameters. The most important risk parameters are Probability of Default (PD) and Loss Given Default (LGD). LGD is the lender’s loss on a loan due to the customer’s default, i.e. failure to meet the credit commitment. Unlike with PD models, whose performance is almost always measured with the Gini coefficient or the Kolmogorov-Smirnov (KS) statistic, there are no standard performance measures of models to predict LGD. Currently, at least ten different performance measures are used. For the purpose of this review, they are classified as either error measures or non-error measures. Among the former are Mean Square Error (MSE), Mean Absolute Error (MAE) etc. The latter include e.g. coefficient of determination (R-squared) and correlation coefficients between the observed and predicted LGD. Understandably, the error measures should be relatively low in a good LGD model, whereas the non-error measures should be relatively high. The advantages and disadvantages of each measure are discussed. It is argued that R-squared should only be used to evaluate the performance of linear models, although it is commonly applied to all sorts of LGD models. Another popular measure is the Area Over the Regression Error Characteristic Curve (AOC), which can be defined twofold, depending on whether squared or absolute residuals are used. It is pointed out that AOC and MSE/MAE are practically identical even for reasonably small samples. Finally, the application of the Area Under the Receiver Operating Characteristic Curve (AUC) to LGD models is critically discussed. As LGD is represented by a continuous variable, AUC requires its arbitrary classification, e.g. below-the-mean and over-the-mean. An alternative is proposed that is free from this drawback. The review is illustrated with examples of evaluating the performance of some LGD models built on the data provided by a UK bank
credit scoring, Loss Given Default, model performance measures
Bijak, Katarzyna
5130b6b9-fbf1-44e8-9106-1dd69c6692a6
Bijak, Katarzyna
5130b6b9-fbf1-44e8-9106-1dd69c6692a6

Bijak, Katarzyna (2014) Performance measures of models to predict Loss Given Default: a critical review. 34th International Symposium on Forecasting, Rotterdam, Netherlands. 28 Jun - 01 Jul 2014.

Record type: Conference or Workshop Item (Other)

Abstract

The need for quantitative models in banking is largely related to the Basel Accords and, in particular, to the Advanced Internal Ratings-Based (AIRB) approach. Under the AIRB approach, lenders are allowed to use their own predictions of risk parameters. The most important risk parameters are Probability of Default (PD) and Loss Given Default (LGD). LGD is the lender’s loss on a loan due to the customer’s default, i.e. failure to meet the credit commitment. Unlike with PD models, whose performance is almost always measured with the Gini coefficient or the Kolmogorov-Smirnov (KS) statistic, there are no standard performance measures of models to predict LGD. Currently, at least ten different performance measures are used. For the purpose of this review, they are classified as either error measures or non-error measures. Among the former are Mean Square Error (MSE), Mean Absolute Error (MAE) etc. The latter include e.g. coefficient of determination (R-squared) and correlation coefficients between the observed and predicted LGD. Understandably, the error measures should be relatively low in a good LGD model, whereas the non-error measures should be relatively high. The advantages and disadvantages of each measure are discussed. It is argued that R-squared should only be used to evaluate the performance of linear models, although it is commonly applied to all sorts of LGD models. Another popular measure is the Area Over the Regression Error Characteristic Curve (AOC), which can be defined twofold, depending on whether squared or absolute residuals are used. It is pointed out that AOC and MSE/MAE are practically identical even for reasonably small samples. Finally, the application of the Area Under the Receiver Operating Characteristic Curve (AUC) to LGD models is critically discussed. As LGD is represented by a continuous variable, AUC requires its arbitrary classification, e.g. below-the-mean and over-the-mean. An alternative is proposed that is free from this drawback. The review is illustrated with examples of evaluating the performance of some LGD models built on the data provided by a UK bank

This record has no associated files available for download.

More information

Published date: 30 June 2014
Venue - Dates: 34th International Symposium on Forecasting, Rotterdam, Netherlands, 2014-06-28 - 2014-07-01
Keywords: credit scoring, Loss Given Default, model performance measures
Organisations: Centre of Excellence for International Banking, Finance & Accounting

Identifiers

Local EPrints ID: 366899
URI: http://eprints.soton.ac.uk/id/eprint/366899
PURE UUID: 90a893c7-01f5-4a54-ae60-82918a04bc60
ORCID for Katarzyna Bijak: ORCID iD orcid.org/0000-0003-1416-9045

Catalogue record

Date deposited: 18 Jul 2014 15:20
Last modified: 12 Dec 2021 03:46

Export record

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×