In this lab, you'll practice the concepts you learned in the last lesson, bias-variance tradeoff.
In this lab you will:
- Demonstrate the tradeoff between bias and variance by way of fitting a machine learning model
In this lab, you'll try to predict some movie revenues based on certain factors, such as ratings and movie year. Start by running the following cell which imports all the necessary functions and the dataset:
import numpy as np
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
import matplotlib.pyplot as plt
%matplotlib inline
df = pd.read_excel('movie_data_detailed_with_ols.xlsx')
df.head()Subset the df DataFrame to only keep the 'domgross', 'budget', 'imdbRating', 'Metascore', and 'imdbVotes' columns.
# Subset the DataFrame
df = None- First, assign the predictors to
Xand the outcome variable,'domgross'toy - Split the data into training and test sets. Set the seed to 42 and the
test_sizeto 0.25
# domgross is the outcome variable
X = None
y = None
X_train , X_test, y_train, y_test = None# create a scaler
scaler = NoneUse the MinMaxScaler to scale the training set. Remember you can fit and transform in a single method using .fit_transform().
Then, use .transform() to apply the scaler to the test set.
# fit and transform X_train
# X_train_scaled = scaler.fit_transform(X_train)
X_train_scaled = None
# transform X_test
X_test_scaled = None# Your code
# create an instance of linear regression
linreg = None
# fit the model to X_train and y_trainUse the model to make predictions on both the training and test sets:
# Training set predictions
lm_train_predictions = None
# Test set predictions
lm_test_predictions = NonePlot predictions for the training set against the actual data:
# Run this cell - vertical distance between the points and the line denote the errors
plt.figure(figsize=(8, 5))
plt.scatter(y_train, lm_train_predictions, label='Model')
plt.plot(y_train, y_train, label='Actual data')
plt.title('Model vs data for training set')
plt.legend();Plot predictions for the test set against the actual data:
# Run this cell - vertical distance between the points and the line denote the errors
plt.figure(figsize=(8, 5))
plt.scatter(y_test, lm_test_predictions, label='Model')
plt.plot(y_test, y_test, label='Actual data')
plt.title('Model vs data for test set')
plt.legend();Create a function bias() to calculate the bias of a model's predictions given the actual data:
(The expected value can simply be taken as the mean or average value.)
import numpy as np
def bias(y, y_hat):
passCreate a function variance() to calculate the variance of a model's predictions:
def variance(y_hat):
pass# Bias and variance for training set
b = None
v = None
print(f'Train bias: {b} \nTrain variance: {v}')
# Train bias (approximate): -4.063953052867542e-09
# Train variance (approximate): 3406811040986517.0# Bias and variance for test set
b = None
v = None
print(f'Test bias: {b} \nTest variance: {v}')
# Test bias: -10982393.918069275
# Test variance: 1518678846127932.0Use PolynomialFeatures with degree 3 and transform X_train_scaled and X_test_scaled.
Important note: By including this, you don't only take polynomials of single variables, but you also combine variables, eg:
Budget times MetaScore squared
What you're essentially doing is taking interactions and creating polynomials at the same time! Have a look at how many columns we get using np.shape()!
# Your code here
poly = None
X_train_poly = None
X_test_poly = None# Check the shapeFit a regression model to the training data:
# Your code here
polyreg = LinearRegression()Use the model to make predictions on both the training and test sets:
# Training set predictions
poly_train_predictions = None
# Test set predictions
poly_test_predictions = NonePlot predictions for the training set against the actual data:
# Run this cell - vertical distance between the points and the line denote the errors
plt.figure(figsize=(8, 5))
plt.scatter(y_train, poly_train_predictions, label='Model')
plt.plot(y_train, y_train, label='Actual data')
plt.title('Model vs data for training set')
plt.legend();Plot predictions for the test set against the actual data:
# Run this cell - vertical distance between the points and the line denote the errors
plt.figure(figsize=(8, 5))
plt.scatter(y_test, poly_test_predictions, label='Model')
plt.plot(y_test, y_test, label='Actual data')
plt.title('Model vs data for test set')
plt.legend();Calculate the bias and variance for the training set:
# Bias and variance for training set
b = None
v = None
print('Train bias: {} \nTrain variance: {}'.format(b, v))
# Train bias (approximate): -2.0997090773148971e-07
# Train variance (approximate): 7394168636697528.0Calculate the bias and variance for the test set:
# Bias and variance for test set
b = None
v = None
print('Test bias: {} \nTest variance: {}'.format(b, v))
# Test bias: -68166032.47666144
# Test variance: 4.798244829435879e+16# Your description hereIn this lab we went from 4 predictors to 35 by adding polynomials and interactions, using PolynomialFeatures. That being said, where 35 leads to overfitting, there are probably ways to improve by adding just a few polynomials. Feel free to experiment and see how bias and variance improve!
This lab gave you insight into how bias and variance change for a training and a test set by using both simple and complex models.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent