Enhancing transparency and fairness in automated credit decisions: an explainable novel hybrid machine learning approach

Abstract

This paper uses a generalised stacking method to introduce a novel hybrid model that combines a one-dimensional convolutional neural network 1DCNN with extreme gradient boosting XGBoost. We compared the predictive accuracies of the proposed hybrid architecture with three conventional algorithms-1DCNN, XGBoost and logistic regression (LR) using a dataset of over twenty thousand peer-to-peer (P2P) consumer credit observations. By leveraging the SHAP algorithm, the research provides a detailed analysis of feature importance, contributing to the model’s predictions and offering insights into the overall and individual significance of different features. The findings demonstrate that the hybrid model outperforms the LR, XGBoost and 1DCNN models in terms of classification accuracy. Furthermore, the research addresses concern regarding fairness and bias by showing that removing potentially discriminatory features, such as age and gender, does not significantly impact the hybrid model’s classification capabilities. This suggests that fair and unbiased credit scoring models can achieve high effectiveness levels without compromising accuracy. This paper makes significant contributions to academic research and practical applications in credit risk management by presenting a hybrid model that offers superior classification accuracy and promotes interpretability using the model agnostic SHAP framework. Supplementary Information The online version contains supplementary material available at 10.1038/s41598-024-75026-8.