Performance measures of models to predict Loss Given Default: a critical review

Bijak, Katarzyna (2014) Performance measures of models to predict Loss Given Default: a critical review. 34th International Symposium on Forecasting, Rotterdam, Netherlands. 28 Jun - 01 Jul 2014.

Record type: Conference or Workshop Item (Other)

Abstract

The need for quantitative models in banking is largely related to the Basel Accords and, in particular, to the Advanced Internal Ratings-Based (AIRB) approach. Under the AIRB approach, lenders are allowed to use their own predictions of risk parameters. The most important risk parameters are Probability of Default (PD) and Loss Given Default (LGD). LGD is the lender’s loss on a loan due to the customer’s default, i.e. failure to meet the credit commitment. Unlike with PD models, whose performance is almost always measured with the Gini coefficient or the Kolmogorov-Smirnov (KS) statistic, there are no standard performance measures of models to predict LGD. Currently, at least ten different performance measures are used. For the purpose of this review, they are classified as either error measures or non-error measures. Among the former are Mean Square Error (MSE), Mean Absolute Error (MAE) etc. The latter include e.g. coefficient of determination (R-squared) and correlation coefficients between the observed and predicted LGD. Understandably, the error measures should be relatively low in a good LGD model, whereas the non-error measures should be relatively high. The advantages and disadvantages of each measure are discussed. It is argued that R-squared should only be used to evaluate the performance of linear models, although it is commonly applied to all sorts of LGD models. Another popular measure is the Area Over the Regression Error Characteristic Curve (AOC), which can be defined twofold, depending on whether squared or absolute residuals are used. It is pointed out that AOC and MSE/MAE are practically identical even for reasonably small samples. Finally, the application of the Area Under the Receiver Operating Characteristic Curve (AUC) to LGD models is critically discussed. As LGD is represented by a continuous variable, AUC requires its arbitrary classification, e.g. below-the-mean and over-the-mean. An alternative is proposed that is free from this drawback. The review is illustrated with examples of evaluating the performance of some LGD models built on the data provided by a UK bank

This record has no associated files available for download.