faizansana / intersection-carla-gym Goto Github PK

Intersection Gym environment in CARLA Town 3

License: MIT License

Python 96.87% Shell 0.88% Dockerfile 2.25%

autonomous-driving carla carla-reinforcement-learning carla-simulator gym gymnasium reinforcement-learning

intersection-carla-gym's Introduction

Intersection Environment for Training Reinforcement Learning Algorithms

OpenAI gym and Gymnasium environment for CARLA Simulator, particularly for a 4-way unsignalized intersection environment.

Getting Started

System Requirements

The following are the requirements for running this repository using the provided docker files:

Operating System: Linux (tested on Ubuntu 20.04)
NVIDIA GPU with CUDA support (tested on NVIDIA GeForce RTX 3060/3080/3090)

Prerequisites

Setup

Clone the repository

git clone https://github.com/faizansana/intersection-carla-gym.git

(Optional) If you want to use the gymnasium environment, then use the main branch. To use gym v0.21, checkout the following branch.
```
git checkout gym-v0.21
```
From within the working directory, open the dev_config.sh file to change any specific requirements such as CARLA version, CUDA version etc.
Run the dev_config.sh file to set the environment variables for docker.
```
bash dev_config.sh
```
Pull the already built containers from docker hub if they are available.
```
docker compose pull
```
After the containers have been pulled, start them using the following command.
```
docker compose up -d
```
(Optional) Open the main_container, and attach it to VS Code using the Remote Explorer extension

Usage (from within main container)

Setup a configuration file based on your requirements:

exp_name: "first_test" # Name of experiment
output_dir: "output" # Name of output directory for logs
env:
    obs_space: "dict" # Choose from "dict" or "normal"
    continuous: True # If False then discrete mode is used
    target_speeds: [0, 3, 6, 9, 12] # For discrete speed control
    desired_speed: 12 # For continuous speed control
    dt: 0.05 
    render: false
    ego_vehicle_filter: "vehicle.lincoln*" # Vehicle to use for ego vehicle
    num_veh: 1 # Number of vehicles in each intersection except ego vehicle
    num_ped: 1 # Number of pedestrians at each crosswalk
    max_steps: 500 # Maximum number of steps per episode
    CAM_RES: 1024 # Camera resolution for rendering
    max_waypt: 200 # Maximum number of waypoints
    pedestrian_proximity_threshold: 2.0 # Threshold to give negative reward when vehicle distance to pedestrian is less than this value
    vehicle_proximity_threshold: 2.5 # Threshold to give negative reward when vehicle distance to other vehicle is less than this value
    reward_weights:
        # Route completion reward
        c_completion: 100.0
        # Collision penalty with vehicle
        c_terminal_collision: -100.0
        # Collision penalty with pedestrian
        c_terminal_pedestrian_collision: -200.0
        # Timeout penalty
        c_terminal_timeout: -10.0
        # Velocity reward constants
        c_v_eff_under_limit: 1.0
        c_v_eff_over_limit: -2.0
        # Penalty for needing another step
        r_step: -0.0
        # Penalty for non-smooth actions
        c_action_reg: -0.0
        # Penalty for yaw delta w.r.t. road heading
        c_yaw_delta: -0.0
        # Penalty for lateral deviation
        c_lat_dev: -0.0
        # Distance from goal penalty
        c_dist_from_goal: 3.5
        # Progress reward
        c_progress: 0.0
        # Penalty for being close to pedestrians
        c_pedestrian_proximity: -10.0
        # Penalty for being close to vehicles
        c_vehicle_proximity: -5.0

Save the config file as config_name.yaml.

Setup the environment using the following code snippet. This is also found in test_env.py.

import yaml

import carla_env_custom

if __name__ == "__main__":
    cfg = yaml.safe_load(open("config_name.yaml", "r"))
    env = carla_env_custom.CarlaEnv(cfg=cfg, host="HOST", tm_port=9000)

    obs, info = env.reset()

    while True:
            obs, reward, done, _, info = env.step(np.array([1.0], dtype=np.float32))
            if done:
                obs, info = env.reset()

A demo video of the environment with num_veh set to 1 and num_ped set to 2 is shown below.

2024-02-22_20-43-29.mp4

The following video shows interfacing with PlotJuggler. 26 values are available to be viewed within PlotJuggler.

2024-02-23_15-52-03.mp4

intersection-carla-gym's People

Contributors

Stargazers

Watchers

Forkers

marinbao

intersection-carla-gym's Issues

Create functional test for overall repo

There is a test_env.py script that runs the environment and checks the returns are compatible with the gym API. Use this as a CI test.

Remove local_carla_agents folder and directly import from carla folder

Currently, there is a local_carla_agents folder which is a copy of that found in the carla Python API folder.

Since containers are being used, we can just refer it directly by adding it to PythonPath

Enhance logging system

Currently the logs are created using the experiment name and output dir params in the config file. This doesn't seem too useful so think of a better way of enhancing the logging and reduce the tree structure of config yaml file.

Remove port field in config file

Currently, the host is defined within code whereas port is called from the config file. This is confusing.

Look into setting no render mode for CARLA

In this case, since we are not depending on sensor states (LiDAR, camera etc.) and only care about other vehicle states, pedestrians etc. using no render mode makes sense. This would also reduce resource consumption and increase FPS. Documentation is here

Installation of carla library in Docker container for 0.9.10.1 does not work

The egg file created only contains 0.9.10 and not the minor 0.1 version. Since we are passing the version directly, this causes the egg file path to be incorrect.

Need to think of a better way of adding this to path.

Upgrade reward function to incorporate pedestrian penalty

At the moment, the model, even after learning for 1.5 million steps, still crashes into pedestrians.

Incorporating penalties for getting close to pedestrians in dense rewards might help.
Larger negative reward for crashing into pedestrian.

Add Discrete space to environment

Currently the environment can only take Box (continuous) action space. Upgrade env to also allow for discrete action spaces.

Scale the rewards between -1 and 1

Based on current research, it was seen that scaling the rewards between -1 and 1 allowed for more stable training.

Update state space to only include vehicles within specific observation area

Currently, the ego vehicle technically has a "Gods-eye" view of the environment. In order to make it more realistic, it should only contain info within specific surroundings.

Potentially add negative reward for distance to other vehicles

Update reward function to give negative reward if too close to other vehicle. This might improve collision percentage.

Unify `desired_speed` and `target_speeds` within config file

Currently target_speeds is of type List[floats] and desired_speed is of type float. Since the latter is for continuous while earlier for discrete, unify into a single config desired_speed which can be of type List[floats] or float.

Look into integration with Foxglove for analysis

A sample project is shown here: https://github.com/collabora/carlafox/tree/main

By doing this, we can potentially get the velocity, acceleration and other details of adversarial vehicles without having to explicitly code them.

However, this will require significant effort as it will require ROS integration through CARLA ROS Bridge

Incorporate Vx for other vehicles and pedestrians into observation space

Currently, it seems like the observation space only contains v_y. Having v_x or tangential velocity might be better.

Segmentation fault occurring after n random timesteps

The environment exits with a segmentation fault that occurs after n timesteps. This is an indeterministic number failing within even 2000 timesteps and as late as never as seen.

The following has been observed.

It is seen to fail faster in python 3.8.16 (environment.yml) as compared to 3.7
It was thought that this might be due to the collision sensor. However, the collision sensor was completely removed from the script and it stil continued to fail.
This seg fault might be occurring due to failure of destroying the actor. It seems that the seg fault is occurring due to the carla library. However since the carla python library is a wrapper for C code, it is hard to debug exactly what is going wrong.
After the traffic manager instantiation was moved to after setting it to synchronous mode, it seems these seg faults occur less. However, this could be unrelated.