Gymnasium environment for the Game 2048 and game-playing agents using temporal difference learning of n-tuple networks.

To install gymnasium-2048 with pip, execute:

pip install gymnasium_2048

From source:

git clone https://github.com/Quentin18/gymnasium-2048
cd gymnasium-2048/
pip install -e .

The action is an integer representing the direction to slide the tiles:

The observation is a 3D ndarray encoding the board state. It is encoded into 16 channels, where each channel is a 4x4 binary image. The i-th channel marks each cell of the game position that contains the i-th tile as 1, and 0 otherwise. Each channel represents the positions of empty cells, 2-tiles, 4-tiles, ... , and 32768-tiles, respectively.

This representation is mostly used for deep convolutional neural networks (DCNN).

At each step, for each tile merge, the player gains a reward equal to the value of the new tile. The total reward, corresponding to the game score, is the sum of rewards obtained throughout the game.

The game starts with two randomly generated tiles. A 2-tile can be generated with probability 0.9 and a 4-tile with probability 0.1.

The episode ends if there are no legal moves, i.e., all squares are occupied and there are no two adjacent tiles sharing the same value.

• size: the size of the game board. The default value is 4.
• max_pow: the maximum power of 2 allowed. The default value is 16.

import gymnasium as gym

gym.make("gymnasium_2048:gymnasium_2048/TwentyFortyEight-v0", size=4, max_pow=16)

To use the training and evaluation scripts, install the training dependencies:

pip install .[training]

To play the game manually with the four arrows of your keyboard, execute:

python -m scripts.play

See the arguments with the help command:

python -m scripts.play -h

To train an agent using temporal difference learning of n-tuple networks, execute:

python -m scripts.train \
  --algo tdl \
  -n 100000 \
  --eval-freq 5000 \
  --eval-episode 1000 \
  --save-freq 5000 \
  --seed 42 \
  -o models/tdl

See the arguments with the help command:

python -m scripts.train -h

To plot training metrics from logs, execute:

python -m scripts.plot \
  -i train.log \
  -t "Temporal Difference Learning" \
  -o figures/training_tdl.png

See the arguments with the help command:

python -m scripts.plot -h

Here are the training metrics of trained policies over episodes:

TDL small	TDL

To see a trained agent in action, execute:

python -m scripts.enjoy \
  --algo tdl \
  -i models/tdl/best_n_tuple_network_policy.zip \
  -n 1 \
  --seed 42

See the arguments with the help command:

python -m scripts.enjoy -h

To evaluate the performance of a trained agent, execute:

python -m scripts.evaluate \
  --algo tdl \
  -i models/tdl/best_n_tuple_network_policy.zip \
  -n 1000 \
  --seed 42 \
  -t "Temporal Difference Learning" \
  -o figures/stats_tdl.png

See the arguments with the help command:

python -m scripts.evaluate -h

Here are the performances of trained policies:

TDL small	TDL

To run tests, execute:

pytest

To cite the repository in publications:

@misc{gymnasium-2048,
  author = {Quentin Deschamps},
  title = {Gymnasium 2048},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Quentin18/gymnasium-2048}},
}

• Szubert and Jaśkowski: Temporal Difference Learning of N-Tuple Networks for the Game 2048
• Guei and Wu: On Reinforcement Learning for the Game of 2048

Quentin Deschamps