The Gymnasium library is supported on Linux and Mac OS. If you have trouble installing on Windows (Failed building wheels for box2d-py?), check out my guide Install Gymnasium on Windows

frozen_lake_q.py
Solves the FrozenLake-v1 8x8 map with Q-Learning.
Companion Youtube video: Solve FrozenLake-v1 8x8 Tutorial

frozen_lake_qe.py
This file is almost identical to frozen_lake_q.py above, except this uses the frozen_lake_enhanced.py environment.

frozen_lake_enhanced.py
This is the FrozenLake-v1 environment "enhanced" to help you better understand Q-Learning. Features:

• The Q values are overlayed on top of each cell of the map. Visually see the Q values update in realtime!
• The map is enlarged to fill the whole screen so that it is easier to read the Q values.
• Shortcut keys to speed up or slow down the animation.

Companion Youtube video: See Q-Learning in Realtime on FrozenLake-v1

mountain_car_q.py
Solves the MountainCar-v0 environment with Q-Learning.
Companion Youtube video: Solve MountainCar-v0 Tutorial

cartpole_q.py
Solves the CartPole-v1 environment with Q-Learning.
Companion Youtube video: Solve CartPole-v1 Tutorial

sb3.py
Solves the Humanoid-v4 environment with Stable Baselines3 Soft Actor-Critic.
Companion Youtube video: Solve Humanoid-v4 Tutorial