In this lab, you'll build upon previous lessons on precision, recall and accuracy and create a confusion matrix visualization. You may remember seeing confusion matrices from our KNN work! Now, we'll put that together into a more cohesive visual using matplotlib.
You will be able to:
- Understand and assess precision recall and accuracy of classifiers
- Evaluate classification models using various metrics
Recall that the confusion matrix represents the counts (or normalized counts) of our True Positives, False Positives, True Negatives and False Negatives. This can further be visualized when analyzing the effectiveness of our classification algorithm.
Here's an example of generating a confusion matrix:
With that, let's look at some code for generating this visual.
As usual, we start by fitting a model to our data by importing, normalizing, splitting into train and test sets and then calling our algorithm.
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
import pandas as pd
#Load the data
df = pd.read_csv('heart.csv')
#Define appropriate X and y
X = df[df.columns[:-1]]
y = df.target
#Normalize the Data
for col in df.columns:
df[col] = (df[col]-min(df[col]))/ (max(df[col]) - min(df[col]))
# Split the data into train and test sets.
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
#Fit a model
logreg = LogisticRegression(fit_intercept = False, C = 1e12) #Starter code
model_log = logreg.fit(X_train, y_train)
print(model_log) #Preview model params
#Predict
y_hat_test = logreg.predict(X_test)
#Data Preview
df.head()
LogisticRegression(C=1000000000000.0, class_weight=None, dual=False,
fit_intercept=False, intercept_scaling=1, max_iter=100,
multi_class='ovr', n_jobs=1, penalty='l2', random_state=None,
solver='liblinear', tol=0.0001, verbose=0, warm_start=False)
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
age | sex | cp | trestbps | chol | fbs | restecg | thalach | exang | oldpeak | slope | ca | thal | target | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 0.708333 | 1.0 | 1.000000 | 0.481132 | 0.244292 | 1.0 | 0.0 | 0.603053 | 0.0 | 0.370968 | 0.0 | 0.0 | 0.333333 | 1.0 |
1 | 0.166667 | 1.0 | 0.666667 | 0.339623 | 0.283105 | 0.0 | 0.5 | 0.885496 | 0.0 | 0.564516 | 0.0 | 0.0 | 0.666667 | 1.0 |
2 | 0.250000 | 0.0 | 0.333333 | 0.339623 | 0.178082 | 0.0 | 0.0 | 0.770992 | 0.0 | 0.225806 | 1.0 | 0.0 | 0.666667 | 1.0 |
3 | 0.562500 | 1.0 | 0.333333 | 0.245283 | 0.251142 | 0.0 | 0.5 | 0.816794 | 0.0 | 0.129032 | 1.0 | 0.0 | 0.666667 | 1.0 |
4 | 0.583333 | 0.0 | 0.000000 | 0.245283 | 0.520548 | 0.0 | 0.5 | 0.702290 | 1.0 | 0.096774 | 1.0 | 0.0 | 0.666667 | 1.0 |
From there it's very easy to create the raw confusion matrix using built in methods:
from sklearn.metrics import confusion_matrix
cnf_matrix = confusion_matrix(y_hat_test, y_test)
print('Confusion Matrix:\n',cnf_matrix)
Confusion Matrix:
[[24 4]
[ 9 39]]
Creating a pretty visual is a little more complicated. Generating the initial image is simple but we have to use the itertools package to iterate over the matrix and append labels to the individual cells.
import numpy as np
import itertools
import matplotlib.pyplot as plt
%matplotlib inline
plt.imshow(cnf_matrix, cmap=plt.cm.Blues) #Create the basic matrix.
#Add title and Axis Labels
plt.title('Confusion Matrix')
plt.ylabel('True label')
plt.xlabel('Predicted label')
#Add appropriate Axis Scales
class_names = set(y) #Get class labels to add to matrix
tick_marks = np.arange(len(class_names))
plt.xticks(tick_marks, class_names, rotation=45)
plt.yticks(tick_marks, class_names)
#Add Labels to Each Cell
thresh = cnf_matrix.max() / 2. #Used for text coloring below
#Here we iterate through the confusion matrix and append labels to our visualization.
for i, j in itertools.product(range(cnf_matrix.shape[0]), range(cnf_matrix.shape[1])):
plt.text(j, i, cnf_matrix[i, j],
horizontalalignment="center",
color="white" if cnf_matrix[i, j] > thresh else "black")
#Add a Side Bar Legend Showing Colors
plt.colorbar()
<matplotlib.colorbar.Colorbar at 0x1a16667ac8>
Generalize the above code into a function that you can reuse to create confusion matrix visuals going forward.
def plot_confusion_matrix(cm, classes,
title='Confusion matrix',
cmap=plt.cm.Blues):
#Pseudocode/Outline:
#Print the confusion matrix (optional)
#Create the basic matrix.
#Add title and Axis Labels
#Add appropriate Axis Scales
#Add Labels to Each Cell
#Add a Side Bar Legend Showing Colors
When the normalization parameter is set to True, your function should return percentages for the each label class in the visual rather then raw counts.
def plot_confusion_matrix(cm, classes,
normalize=False,
title='Confusion matrix',
cmap=plt.cm.Blues):
#Check if Normalization Option is Set to True. If so, normalize the raw confusion matrix before visualizing
#Other code should be equivalent to your previous function
#Print the confusion matrix (optional)
#Create the basic matrix.
#Add title and Axis Labels
#Add appropriate Axis Scales
#Add Labels to Each Cell
#Add a Side Bar Legend Showing Colors
Call you function to create a normalized confusion matrix for the model above.
# Plot normalized confusion matrix
# Your code here
Well done! In this lab we previewed the confusion matrix and practice our matplotlib skills for producing visualizations!