How much do we see? On the explainability of partial dependence plots for credit risk scoring

• Autorzy: Gero Szepannek , Karsten Lübke
• Języki publikacji: EN

Abstrakty

EN

Risk prediction models in credit scoring have to fulfil regulatory requirements, one of which consists in the interpretability of the model. Unfortunately, many popular modern machine learning algorithms result in models that do not satisfy this business need, whereas the research activities in the field of explainable machine learning have strongly increased in recent years. Partial dependence plots denote one of the most popular methods for model-agnostic interpretation of a feature's effect on the model outcome, but in practice they are usually applied without answering the question of how much can actually be seen in such plots. For this purpose, in this paper a methodology is presented in order to analyse to what extent arbitrary machine learning models are explainable by partial dependence plots. The proposed framework provides both a visualisation, as well as a measure to quantify the explainability of a model on an understandable scale. A corrected version of the German credit data, one of the most popular data sets of this application domain, is used to demonstrate the proposed methodology.(original abstract)

Słowa kluczowe

EN

Credit scoring; Risk assessment models

PL

Skoring kredytowy; Modele oceny ryzyka

• Czasopismo: Argumenta Oeconomica
• Rocznik: 2023
• Numer: nr 1 (50)
• Strony: 137--150

Twórcy

autor

Gero Szepannek

• Stralsund University of Applied Sciences, Germany

autor

Karsten Lübke

• FOM University of Applied Sciences, Dortmund, Germany

Bibliografia

• Apley, D. (2016). Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models. arXiv:1612.08468.
• Baesens, B., Van Gestel, T., Viaene, S., Stepanova, M., Suykens, J. and Vanthienen J. (2002). Benchmarking State-of-the-Art Classification Algorithms for Credit Scoring. Journal of the Operational Research Society, 54(6), 627-635.
• Banasik, J. and Crook, J. (2007). Reject inference, augmentation and sample selection. European Journal of Operational Research, 183, 1582-1594.
• Basel Committee on Banking Supervision. International Convergence of Capital Measurement and Capital Standards: A Revised Framework. Bank for International Settlements, 2005.
• Biecek, P. (2018). DALEX: Explainers for Complex Predictive Models. R. Journal of Machine Learning Research, 19(84), 1-5.
• Biecek, P., Chlebus, M., Gajda, J., Gosiewska, A., Kozak, A., Ogonowski, D., Sztachelski, J. and Wojewnik, P. (2021). Enabling Machine Learning Algorithms for Credit Scoring - Explainable Artificial Intelligence (XAI) methods for clear understanding complex predictive models. arXiv:2104.06735.
• Bischl, B., Kühn, T. and Szepannek, G. (2014). On Class Imbalance Correction for Classification Algorithms in Credit Scoring. In Operations Research Proceedings 2014. Selected Papers of the Annual International Conference of the German Operations Research Society (GOR), 37-43, 2016.
• Breiman, L. (2001). Random Forests. Machine Learning, 45(1), 5-32.
• Britton, M. (2019). VINE: Visualizing Statistical Interactions in Black Box Models. arXiv:1904.00561.
• Brown, I. and Mues, C. (2012). An experimental comparison of classification algorithms for imbalanced credit scoring data sets. Expert Systems with Applications, 39(3), 3446-3453.
• Bücker, M., Szepannek, G., Gosiewska, A. and Biecek, P. (2021). Transparency, auditability and explainability of machine learning models in credit scoring. Journal of the Operational Research Society, 73(1), 70-90.
• Bussmann, N., Giudici, P., Marinelli, D. and Papenbrock, J. (2020). Explainable AI in fintech risk management. Frontiers in Artificial Intelligence, 3.
• Cleveland, W. (1993). Visualizing Data. Hobart Press.
• Crook, J., Edelman, D., and Thomas, L. (2007). Recent developments in consumer credit risk assessment. European Journal of Operational Research, 183, 1447-1465.
• Dastile, X., and Celik, T. (2021). Making deep learning-based predictions for credit scoring explainable. IEEE Access, 9.
• Demajo, L., Vella, V. and Dingli, A. (2020). Explainable AI for interpretable credit scoring. arXiv:2012.03749.
• Dua, D., and Graff, C. (2017). UCI Machine Learning Repository. Available at: https://archive.ics.uci.edu/ml/index.php, 2017.
• European Banking Authority. Guidelines on PD estimation, LGD estimation and the treatment of defaulted exposures.
• Friedman, J. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29, 1189-1232.
• Friedman, J. and Popescu, B. (2008). Predictive learning via rule ensembles. Annals of Applied Statistics, 2(3), 916-954.
• Goldstein, A., Kapelner, A., Bleich, J. and Pitkin, E. (2015). Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. Journal of Computational and Graphical Statistics, 24(1), 44-65.
• Gosiewska, A. and Biecek, P. (2019). iBreakDown: Uncertainty of model explanations for non-additive predictive models. arXiv:1903.11420.
• Greenwell, B. (2017). An R Package for constructing partial dependence plots. The R Journal, 9(1), 421-436.
• Groemping, U. (2019). South German credit data: Correcting a widely used data set. Department II, Beuth University of Applied Sciences Berlin.
• Hooker, G. and Mentch, L. (2019). Please stop permuting features: An explanation and alternatives. arXiv:1905.03151.
• Hutter, F., Kotthoff, L. and Vanschoren, J. (2018). Automated Machine Learning: Methods, Systems, Challenges. Springer.
• Kusner, K. and Loftus, J. (2020). The long road to fairer algorithms. Nature, 534, 34-36.
• Lang, M., Binder, M., Richter, J., Schratz, P., Pfisterer, F., Coors, S., Au, Q., Casalicchio, G., Kotthoff, L. and Bischl, B. (2019). mlr3: A modern object-oriented machine learning framework. R. Journal of Open Source Software.
• Lessmann, S., Baesens, B., Seow, H.V., and Thomas, L. (2015). Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research. European Journal of Operational Research, 247(1), 124-136.
• Liaw, A. and Wiener, M. (2002). Classification and regression by random Forest. R News, 2(3), 18-22.
• Louzada, F., Ara, A. and Fernandes, G. (2016). Classification methods applied to credit scoring: A systematic review and overall comparison. Surveys in OR and Management Science, 21(2), 117-134.
• Luebke, K., Gehrke, M. Horst, J. and Szepannek, G. (2020). Why we should teach causal inference: Examples in linear regression with simulated data. Journal of Statistics Education, 28(2), 133-139.
• Miller, G. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2).
• Molnar, C., Bischl, B and Casalicchio, G. (2018). iml: An R package for interpretable machine learning. Journal of Open Source Software, 3(26).
• Molnar, C., Casalicchio, G. and Bischl, B. (2020a). Quantifying model complexity via functional decomposition for better post-hoc interpretability. In Machine Learning and Knowledge Discovery in Databases (pp. 193-204). Springer International Publishing.
• Molnar, C, König, G., Herbinger, J., Freiesleben, T., Dandl, S., Scholbeck, C., Casalicchio, G., Grosse-Wentrup, M. and Bischl, B. (2020b). Pitfalls to avoid when interpreting machine learning models. arXiv:2007.04131.
• Ribeiro, M., Singh, S. and Guestrin, C. (2016). "Why should I trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135-1144.
• Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1, 206-215.
• Staniak, M. and Biecek, P. (2018). Explanations of model predictions with live and break Down Packages. The R Journal, 10(2), 395-409.
• Strumbelj, E. and Kononenko, I. (2014). Explaining prediction models and individual predictions with feature contributions. Knowledge and Information Systems, 41(3), 647-665.
• Szepannek, G. (2017). On the practical relevance of modern machine learning algorithms for credit scoring applications. WIAS Report Series, 29, 88-96.
• Szepannek, G. (2022). An Overview on the landscape of R packages for open source scorecard modelling. Risks, 10(3), 1-33.
• Szepannek G. and Luebke, K. (2021). Facing the challenges of developing fair risk scoring models. Frontiers in Artificial Intelligence, 4.
• Szepannek G. and Aschenbruck, R. (2020). Predicting eBay prices: selecting and interpreting machine learning models - Results of the AG DANK 2018 Data Science Competition. Archives of Data Science A, 7(1), 1-17.
• Torrent, N., Visani, G. and Enrico Bagli, E. (2020). PSD2 Explainable AI model for credit scoring. arXiv:2011.10367.
• Verbraken, T., Bravo, C., Weber Richard, and Baesens, B. (2014). Development and application of consumer credit scoring models using profit-based classification measures. European Journal of Operational Research, 238(2), 505-513.
• Vincotti, V. and Hand, D. (2002). Scorecard construction with unbalanced class sizes. Journal of the Iranian Statistical Society, 2, 189-205.
• Zhao, Q. and Hastie, T. (2019). Causal interpretations of black-box models. Journal of Business and Economic Statistics, 2019.

Identyfikatory

DOI

DOI: 10.15611/aoe.2023.1.07