(RandomOverSampler, SMOTE, Cluster Centroids, SMOTEENN, BalancedRandomForestClassifier, EasyEnsembleClassifier)

Credit risk is an inherently unbalanced classification problem, as the number of good loans easily outnumber the number of risky loans. Therefore, I employed different techniques to train and evaluate models with unbalanced classes. I used imbalanced-learn and scikit-learn libraries to build and evaluate models using resampling. I also evaluated the performance of these models and made a recommendation on whether they should be used to predict credit risk.

• Extract credit data
• Encode Obj Data Types into Int/Float Data Types through LabelEncoder from sklearn.preprocessing
• Implement machine learning models. Train the model and generate predictions.
• Use resampling to attempt to address class imbalance.
• Oversample the data using the RandomOverSampler and SMOTE algorithms.
• Undersample the data using the cluster centroids algorithm.
• Use a combination approach with the SMOTEENN algorithm.
• Train a logistic regression classifier (from Scikit-learn) using the resampled data.
• Evaluate the performance of machine learning models.
• Calculate the balanced accuracy score using balanced_accuracy_score from sklearn.metrics.
• Generate a confusion matrix.
• Print the classification report (classification_report_imbalanced from imblearn.metrics).
• Write an analysis of the models’ performance. Describe the precision and recall scores, as well as the balanced accuracy score.

• Python (Jupyter Notebook)
• Pandas
• sklearn
• imblearn

Confusion Matrix

imbalanced classification report

Balanced Accuracy Score: 0.608 Precision TP/(TP+FP): 0.10 Recall / Sensitivity TP/(TP+FN): 0.77

Confusion Matrix

imbalanced classification report

Balanced Accuracy Score: 0.534 Precision TP/(TP+FP): 0.006 Recall / Sensitivity TP/(TP+FN): 0.77

Confusion Matrix

imbalanced classification report

Balanced Accuracy Score: 0.650 Precision TP/(TP+FP): 0.009 Recall / Sensitivity TP/(TP+FN): 0.77

While comparing the balanced accuracy scores, it is clear that the Combination approach with SMOTEEN algorithms has the highest accuracy. This is also true with the recall/sensitivity, which takes into consideration how the ML model is capable of correctly predicting the credit risk of a specific individual. That said, Oversampling with RandomOverSampler and SMOTE algorithms has a 0.001 higher likely hood of predicting the credit risk of specifical individual when measured against false negative cases. That said, with recall and accuracy much higher than other models, Combination approach with SMOTEEN algorithms seems to be the best approach.

Even though we employed random over, under, and combined sampling appoarches to offset the small number of actual high-risk credit loans, there is no by-passing that high-risk credit loans is a minority data set within the population of the dataset. This has a big effect on the performance on our model even when we have employed techniques to offset this distribution. A recommendation would be to locate more high-risk credit loan data. The reason for this is so that the ML model can be better trained in identifying the loans through the input features.