For this project, you will train an agent to navigate (and collect bananas!) in a large, square world.

A reward of +1 is provided for collecting a yellow banana, and a reward of -1 is provided for collecting a blue banana. Thus, the goal of your agent is to collect as many yellow bananas as possible while avoiding blue bananas.

The state space has 37 dimensions and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction. Given this information, the agent has to learn how to best select actions. Four discrete actions are available, corresponding to:

• 0 - move forward.
• 1 - move backward.
• 2 - turn left.
• 3 - turn right.

The task is episodic, and in order to solve the environment, your agent must get an average score of +13 over 100 consecutive episodes.

• Python 3.6 or later
• Conda

1. Clone the repository
2. Create a conda environment

conda create --name drlnd python=3.12

3. Activate the environment

conda activate drlnd

4. Install the required packages

pip install -r requirements.txt

5. Install the requirements from the python folder

cd python
pip install .

6. Download the Unity environments from one of the links below. You need only select the environment that matches your operating system:
   • Linux: click here
   • Mac OSX: click here
   • Windows (32-bit): click here
   • Windows (64-bit): click here
7. Unzip the file and place it in the root directory of the repository

In the project, your agent learned from information such as its velocity, along with ray-based perception of objects around its forward direction. A more challenging task would be to learn directly from pixels!

This environment is almost identical to the project environment, where the only difference is that the state is an 84 x 84 RGB image, corresponding to the agent's first-person view.

You need only select the environment that matches your operating system:

• Linux: click here
• Mac OSX: click here
• Windows (32-bit): click here
• Windows (64-bit): click here

Unzip the file and place it in the root directory of the repository

There are already pre-trained weights in the checkpoint.pth and checkpoint_visual.pth file for 2000 episodes.

The current implementation uses the environment with the ray-based perception for linux. Feel free to change it to the environment with the pixel-based perception and/or for your operating system. You can do this by changing the env_filename and visual variables in the test.py and train.py scripts.

To test the agent, run the test.py script.

python src/test.py

To train the agent, run the train.py script.

python src/train.py

The implementation is based on the Double Deep Q-Network (DDQN) algorithm with some modifications. The modifications include the use of a dueling network architecture and prioritized experience replay.

The report for this project can be found in the src/Report.ipynb file.