This project consists of one main modules: interpret_model.py.

• In interpret_model.py all models are being trained and evaluated.

• Python: 3.7.4
  • Tensorflow: 1.13.1
  • xai: 0.0.5
  • Scikit-learn: 0.20.1
  • numpy: 1.16.0
  • eli5: 0.10.1
  • pandas: 0.25.3
  • lime: 0.1.1.36

There are models such as linear regression, decision trees, etc. that have the advantage to be easily interpretable. These models can have properties such as linearity and monotonicity, interactions between its features and are also suitable for different tasks (regression, classification or both).

• linearity - if the association between features and target is modelled linearly.
• monotonicity - an increase in the feature value always affects the target outcome in the same way (increase or decrease).

However, these models also have one big disadvantage that predictive performance is lost compared to other machine learning models.

(See https://christophm.github.io/interpretable-ml-book/simple.html#simple)

In our code example Logistic Regression and Decision Tree Classifiers are being trained with the Census Income Data Set to predict weather a person makes over 50K a year. Four different models per classifier are being trained - (1.) one with the raw data set without any modifications, (2.) one with two removed features: marital-status, relationship, (3.) one with the same features removed but a balanced (on gender) data set (with xai toolbox) and (4.) one more balanced (on ethnicity) data set with removed: marital-status, relationship and gender. Then these models are being interpreted using the eli5 library.

• Model 1. accuracy: 0.8108301770907974
• Classification report:

• Model 2. accuracy: 0.7800184256321016
• Classification report:

• Model 3. accuracy: 0.6966666666666667
• Classification report:

• Model 4. accuracy: 0.7763422337065748
• Classification report:

From the classifications reports we could identify some specifics about the four models. Models 1. and 2. show almost equal performance in accuracy, precision and recall. On the other hand, the Model 3. performs slightly worse than the first two and Model 4. has almost the same accuracy as 1. and 2. but shows results on precision and recall closer to Model 3.

• Precision is the ability of a classifier not to label an instance positive that is actually negative -> The precision percent of people that earn >= 50k is higher (less false negatives) in Model 3. and 4. in contrast to the other two models but the precision percent of people that earn < 50k is lower (in 3. and 4.) than the other two models.

• Recall is the ability of a classifier to find all positive instances. Model 4. performs better regarding the recall for people that < 50k (~89% of the people that earn < 50k were truly classified) than the other three models (which show almost the same values). But on the contrary Model 3. and 4. perform worse on the people that earn >= 50k and were truly classified (less false negatives for 1. and 2.).

• Feature importance table for each of the models. (Weight: Feature)

NOTE: Features with largest coefficients. Caveats:

1. Be careful with features which are not independent - weights don't show their importance.
2. If scale of input features is different then scale of coefficients will also be different, making direct comparison between coefficient values incorrect.
3. Depending on regularization, rare features sometimes may have high coefficients; this doesn't mean they contribute much to the classification result for most examples.

We can now evaluate the feature importance of each model:

• On the one hand we could see that in Model 1. the columns: marital-status and relationship play an important role in deciding whether a person should earn >= 50k. In my opinion these two columns should not be part of the models and were therefore removed in Models 2., 3. and 4. Looking at other models we could now see that the most important role in deciding whether a person earns >= 50k is played by the columns: capital-gain, occupation and education-num.

• On the other hand we see that in Model 2. a key influencer on deciding whether a person should earn < 50k is the gender. The feature gender__ Female affects the prediction in a negative way, and, therefore, I have balanced the data set in Model 3. to have equal number of males and females that earn >= 50k or < 50k (see support in the classification report). As we can see in the feature importance column of Model 3., this have not made any positive impact or the negativity of the gender__ Female feature. Thus, in Model 4. I have also removed the gender column and balanced the model on ethnicity.

Let us now interpret and evaluate some concrete examples of the test data set.

From my point of view this person should be classified as one earning <50k, bacause of his age, hours-per-week and education. That are also the results of Models 2., 3. and 4. Due to the huge impact of the marital-status on the decision, Model 1. has predicted, the otherwise unexpected result, right.

This example is interesting because only Model 4. classifies it wrong, because the information about the capital-gain and capital-loss is not convincing and the most valuable feature is the ethnicity, which should also have been removed.

There are quite a lot of such examples in which Model 1. got the wrong result and Models 2., 3. and 4. guessed it right. In this case we could see that the decision for the person's income to be <50k of the first Model is mainly based on the fact that the person has never married. In the other three cases there are not strong indicators for a negative decision.

In this example we see that only Model 2. classified the person's income wrong, bacause of his occupation as Farming-fishing. Model 3. and 4. have balanced data sets and in Model 1. the prediction was strongly influenced by the material-status.

In this example we can see that only Model 4. classified the person's income right, because it was not biased by columns (that otherwise should not play role in the decision) such as: marital-status, relationship, gender.

• Interpret and evaluate the results of the models trained with a Decision Tree Classifier

Currently the Decision Tree Classifier's code is commented out.

For our black-box interpretations we are going to use LIME. LIME explanations are based on local surrogate models. These models are interpretable models (like a linear model or decision tree) that are learned on the predictions of the original black box model. But instead of trying to fit a global surrogate model, LIME focuses on fitting local surrogate models to explain why single predictions were made.

(XG-)Boosting, in contrast to Logistic Regression and Decision Trees, takes a more iterative approach. It is an ensemble technique in that many models are combined together to perform the final one, but rather than training all of the models in isolation of one another, boosting trains models in succession, with each new model being trained to correct the errors made by the previous ones.

(See Explainable AI Driving business value through greater understanding, p.24-25)

In our code example Random Forest and XGBoost classifiers are being trained with the Census Income Data Set to predict weather a person makes over 50K a year. Two different models per classifier are being trained - (1.) one with the raw data set without any modifications and (4.) one more data set with removed: marital-status, relationship, ethnicity and gender. Then these models are being interpreted using the LIME.

• Model 1. accuracy: 0.8626266762206981
• Classification report:

• Model 2. (without marital-status, relationship, ethnicity and gender) accuracy: 0.8462483365748797
• Classification report:

The classification reports for both models look pretty similar. One notices that they perform better than the other models achieving accuracy of more than 84%.

From the examples' results one notices that LIME chooses the five most important features and their importance. Comparing both models it is apparent that the only feature that plays an important role in the predictions, that was removed from Model 2., is marital-status. And in some examples this feature plays a crucial role.

In the example above it is easily seen that material-status has enough impact to switch the decision from <50k to >= 50k, which in this case generates the wrong answer. Similar results also show the examples below.

Both models show almost equal results, despite the second one is missing four features, because only one feature plays an important part in the decisions of the classifier.

• Interpret and evaluate the results of the models trained with a Random Forest Classifier

Currently the Random Forest Classifier's code is commented out.