carla-simulator / reinforcement-learning Goto Github PK

Hi,

One of the baselines in the CARLA legacy paper is modular pipeline. Is there any open source code for this baseline?

Thanks.

I just want to konw what is the network architeture in you A3C experiment ? And what is the image size used ?
I try to output 320 * 240 image with 3 instances on one GPU, but the TCP connection is always broken. I guess it's because of the amount of data ? And how can I solve it ?

Currently I only use image 84 * 84, which I think it's too small .

Hi,

I got error while running the code:
ValueError: pvals < 0, pvals > 1 or pvals contains NaNs

The error occurs at line 27 in policy_output.py

When I tried to print the values contained in batch_probs, I got this:

[ 2.29848295e-07 4.51255920e-01 1.97014214e-05 3.09768734e-05 5.48567290e-01 -1.18768529e-07 1.16773965e-04 -1.15810205e-07 8.16957236e-06]

There are some negative values there... Can you help me please ?

I found this interesting article : (https://numpy.org/doc/1.17/reference/random/generated/numpy.random.mtrand.RandomState.multinomial.html)

Thank you.

Thank you @felipecode for sharing reinforcement-learning code and I think the trained result is good. I want to use the train code, so do you have plan to share the training code and when?
I hope to recieve your response, thanks for your job !

Hi,

I can't find a requirement for 'tensorflow-gpu'. Am I correct to assume that the reinforcement learning experiment doesn't require it?

Hello,
I am getting the following error when trying to run the reinforcement learning:

E:\Build_NewCarlaWithGreenLaserBeam\PythonClient>python run_RL.py --city-name To wn01 --host 127.0.0.1 --port 2000 --corl-2017 ERROR: (127.0.0.1:2000) failed to read data: timed out ERROR: (127.0.0.1:2000) failed to read data: timed out ERROR: (127.0.0.1:2000) failed to read data: [WinError 10054] An existing connec tion was forcibly closed by the remote host ERROR: (127.0.0.1:2000) failed to connect: [WinError 10061] No connection could be made because the target machine actively refused it
Server's been loaded up as:
E:\Build_NewCarlaWithGreenLaserBeam>CarlaUE4.exe /Game/Maps/Town01 -carla-server -benchmark -fps=10 -windowed -ResX=480 -ResY=240 -carla-world-port=2000
Anything else I should try?

Hey , it is weird when I start run_RL.py. The script throws an error with

print(sensory, dir(sensory['CameraRGB']))
KeyError: 'CameraRGB'

Hey , I run the RL benchmark with

python run_RL.py --corl-2017

In which my python environment is set by

conda create -n carla_rl python=3.6 chainer=1.24.0 cached-property=1.4.2 pillow=5.1.0 opencv=3.3.1 h5py=2.7.1

An error occurs with

Traceback (most recent call last):
  File "run_RL.py", line 89, in <module>
    args.host, args.port)
  File "/home/ng/Projects/CARLA_0.8.2/PythonClient/carla/driving_benchmark/driving_benchmark.py", line 294, in run_driving_benchmark
    benchmark_summary = benchmark.benchmark_agent(experiment_suite, agent, client)
  File "/home/ng/Projects/CARLA_0.8.2/PythonClient/carla/driving_benchmark/driving_benchmark.py", line 121, in benchmark_agent
    self._get_shortest_path(positions[start_index], positions[end_index]))
  File "/home/ng/Projects/CARLA_0.8.2/PythonClient/carla/driving_benchmark/driving_benchmark.py", line 182, in _get_shortest_path
    end_point.orientation.x, end_point.orientation.y, end_point.orientation.z])
  File "/home/ng/Projects/CARLA_0.8.2/PythonClient/carla/planner/planner.py", line 114, in get_shortest_path_distance
    track_target, target_ori)
  File "/home/ng/Projects/CARLA_0.8.2/PythonClient/carla/planner/city_track.py", line 91, in compute_route
    route = a_star.solve()
  File "/home/ng/Projects/CARLA_0.8.2/PythonClient/carla/planner/astar.py", line 142, in solve
    return self.get_path()
  File "/home/ng/Projects/CARLA_0.8.2/PythonClient/carla/planner/astar.py", line 111, in get_path
    path.append((cell.x, cell.y))
AttributeError: 'NoneType' object has no attribute 'x'

Any ideas?

from carla.driving_benchmark.experiment_suite import CoRL2017, BasicExperimentSuite

should be

from carla.driving_benchmark.experiment_suites import CoRL2017, BasicExperimentSuite

Hi,

I want to train an RL agent in Carla. Is there any way to run the simulation faster?

I use:
carla9.6
ubuntu 18.04
rtx 2080ti

Thank you

Hi, I'm using trying out this code in windows. I always get this error : ERROR: (localhost:2000) failed to read data: timed out. This is the error trace.
runfile('C:/Users/cvaram/Documents/CARLA_0.9.5/PythonAPI/run_RL.py', args='--city-name Town01 --corl-2017', wdir='C:/Users/cvaramba/Documents/CARLA_0.9.5/PythonAPI')
Reloaded modules: carla, carla.libcarla, carla.driving_benchmark, carla.driving_benchmark.driving_benchmark, carla.client, carla.sensor, carla.transform, carla.carla_server_pb2, carla.tcp, carla.util, carla.driving_benchmark.metrics, carla.planner, carla.planner.planner, carla.planner.city_track, carla.planner.graph, carla.planner.astar, carla.planner.map, carla.planner.grid, carla.planner.converter, carla.settings, carla.driving_benchmark.results_printer, carla.driving_benchmark.recording, carla.driving_benchmark.experiment_suites, carla.driving_benchmark.experiment_suites.basic_experiment_suite, carla.driving_benchmark.experiment, carla.driving_benchmark.experiment_suites.experiment_suite, carla.driving_benchmark.experiment_suites.corl_2017, agent.runnable_model, agent.asyncrl, agent.asyncrl.run_train_test, agent.asyncrl.policy, agent.asyncrl.policy_output, agent.asyncrl.fc_net, agent.asyncrl.nonlinearity, agent.asyncrl.dqn_head, agent.asyncrl.a3c, agent.asyncrl.weight_init, carla.agent, carla.agent.forward_agent, carla.agent.agent

ERROR: (localhost:2000) failed to read data: timed out
ERROR: (localhost:2000) failed to read data: timed out
ERROR: (localhost:2000) failed to read data: timed out
Traceback (most recent call last):

File "", line 1, in
runfile('C:/Users/vara/Documents/CARLA_0.9.5/PythonAPI/run_RL.py', args='--city-name Town01 --corl-2017', wdir='C:/Users/vara/Documents/CARLA_0.9.5/PythonAPI')

File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 786, in runfile
execfile(filename, namespace)

File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "C:/Users/vara/Documents/CARLA_0.9.5/PythonAPI/run_RL.py", line 89, in
args.host, args.port)

File "C:\Users\vara\Documents\CARLA_0.9.5\PythonAPI\carla\driving_benchmark\driving_benchmark.py", line 280, in run_driving_benchmark
client.load_settings(CarlaSettings())

File "C:\Users\vara\Documents\CARLA_0.9.5\PythonAPI\carla\client.py", line 75, in load_settings
return self._request_new_episode(carla_settings)

File "C:\Users\vara\Documents\CARLA_0.9.5\PythonAPI\carla\client.py", line 160, in _request_new_episode
data = self._world_client.read()

File "C:\Users\vara\Documents\CARLA_0.9.5\PythonAPI\carla\tcp.py", line 73, in read
header = self._read_n(4)

File "C:\Users\vara\Documents\CARLA_0.9.5\PythonAPI\carla\tcp.py", line 89, in _read_n
data = self._socket.recv(length)

Please any help would be appreciated.

Thank you @felipecode for sharing this code, When you can share the training code, hoping it will be in the next few days.

when i test your code it will just go straight, do I have a problem somewhere?

Hi,

We have been working to reproduce the results of this repository by training an A2C agent from scratch in the CARLA simulator. You can see our code here: https://github.com/seansegal/carla-rl.

While we do see a considerable increase in mean episode reward over the course of our training, we have not be able to match the performance in the benchmarks that was reported in the original CARLA paper. We matched all the hyperparameters given in the paper and for those that are not given, we have tried to infer those from the agent/trained_model/args.txt file inside this repository.

Would it be possible to release the training code for this agent? Given that this is part of an open-source project, is there a reason that it has not been released yet?

If releasing the code is not possible, could you please provide some of the following training details:

• Which tasks were used for training (no turns, one turn, routing, dynamic objects)? Were they sampled randomly? Was curriculum learning used?
• Was any reward normalization technique used during training?
• How do you handle this bug (carla-simulator/carla#13) in your training? We have an issue where a car can get stuck in an episode against a static object since the collision is not registered. The experiment only terminates during the timeout.
• Were the agent's trained with 10 million environment steps or 100 million environment steps (10 million steps for each of the 10 environments)?

Thanks for your help!
Sean, Sergio and Seung-Eun (University of Toronto)

Hello,
I try to run the server:

D:\auto_dev\CARLA\CARLA_0.8.2>CarlaUE4.exe /Game/Maps/Town01 -carla-server -benc
hmark -fps=10 -windowed -ResX=800 -ResY=600 -world-port=2000

But, client can't reach it:

D:\auto_dev\CARLA\CARLA_0.8.2\PythonClient>python run_RL.py --city-name Town01 -
-port 2000
D:\Program Files\Python\Python36\lib\site-packages\h5py\__init__.py:36: FutureWa
rning: Conversion of the second argument of issubdtype from `float` to `np.float
ing` is deprecated. In future, it will be treated as `np.float64 == np.dtype(flo
at).type`.
  from ._conv import register_converters as _register_converters
ERROR: (localhost:2000) failed to read data: timed out
ERROR: (localhost:2000) failed to read data: [WinError 10054] An existing connec
tion was forcibly closed by the remote host

Hi, could someone tell me how to train A3C on the CARLA simulator?
I believe I should use A3CTrainer in a3c.py, but I cannot find where this class is used to train the model.
Also, I am not sure how the reward function is defined in A3C. Is it manually defined in somewhere in the codes or is it pre-defined by the CARLA simulator?

Thank you!

Command:

python3 run_RL.py --city-name Town01 --corl-2017

Error:

Traceback (most recent call last):
  File "run_RL.py", line 77, in <module>
    model_file='agent/trained_model/9600000.h5', n_actions=9, frameskip=1)
  File "/home/user/projects/carla-0.8.2/PythonClient/reinforcement-learning/agent/runnable_model.py", line 26, in __init__
    self.setup_model(self.n_actions, self.n_meas, self.args)
  File "/home/user/projects/carla-0.8.2/PythonClient/reinforcement-learning/agent/runnable_model.py", line 64, in setup_model
    self.model = run_train_test.get_model(n_actions, n_meas, args)
  File "/home/user/projects/carla-0.8.2/PythonClient/reinforcement-learning/agent/asyncrl/run_train_test.py", line 101, in get_model
    n_input_channels=args.n_images_to_accum, nonlinearity_str=args.nonlinearity, weight_init_str=args.weight_init, bias_init=args.bias_init)
  File "/home/user/projects/carla-0.8.2/PythonClient/reinforcement-learning/agent/asyncrl/run_train_test.py", line 65, in __init__
    self.dqn_net = dqn_head.NatureDQNHead(n_input_channels=n_input_channels, nonlinearity_str=nonlinearity_str, bias=bias_init)
  File "/home/user/projects/carla-0.8.2/PythonClient/reinforcement-learning/agent/asyncrl/dqn_head.py", line 18, in __init__
    L.Convolution2D(n_input_channels, 32, 8, stride=4, bias=bias),
  File "/home/user/.local/lib/python3.5/site-packages/chainer/links/connection/convolution_2d.py", line 124, in __init__
    ('dilate', 1), ('groups', 1))
  File "/home/user/.local/lib/python3.5/site-packages/chainer/utils/argument.py", line 18, in parse_kwargs
    raise TypeError(message)
TypeError: __init__() got unexpected keyword argument(s) 'bias'

with chainer==4.3.1

Fix:
Change all L.Convolution2D params in reinforcement-learning/agent/asyncrl/dqn_head.py:

L.Convolution2D(n_input_channels, 32, 8, stride=4, bias=bias),

to (base -> initial_bias)

L.Convolution2D(n_input_channels, 32, 8, stride=4, initial_bias=bias),

Reference: http://docs.chainer.org/en/stable/reference/generated/chainer.links.Convolution2D.html

hi,
Your work is just so amazing!! And I want to ask two questions: can I use this https://github.com/muupan/async-rl to train the A3C model for RL? By the way, when will the training code for Imitation Learning be released? Looking forward to your reply~Thanks
Best wishes

After executing the code and Carla starts, it gives me an error... this is the full log
Traceback (most recent call last): File "run_RL.py", line 89, in <module> args.host, args.port) File "/media/bignrz/World/carla simulator/CARLA_0.8.2/PythonClient/carla/driving_benchmark/driving_benchmark.py", line 294, in run_driving_benchmark benchmark_summary = benchmark.benchmark_agent(experiment_suite, agent, client) File "/media/bignrz/World/carla simulator/CARLA_0.8.2/PythonClient/carla/driving_benchmark/driving_benchmark.py", line 129, in benchmark_agent + '.' + str(end_index)) File "/media/bignrz/World/carla simulator/CARLA_0.8.2/PythonClient/carla/driving_benchmark/driving_benchmark.py", line 227, in _run_navigation_episode control = agent.run_step(measurements, sensor_data, directions, target) File "/media/bignrz/World/projects/carla RL envs/reinforcement-learning/agent/runnable_model.py", line 35, in run_step action_idx = self.actor.act(obs_preprocessed=obs_preprocessed) File "/media/bignrz/World/projects/carla RL envs/reinforcement-learning/agent/asyncrl/a3c.py", line 49, in act action = pout.action_indices[0] File "/home/bignrz/.local/lib/python3.6/site-packages/cached_property.py", line 35, in __get__ value = obj.__dict__[self.func.__name__] = self.func(obj) File "/media/bignrz/World/projects/carla RL envs/reinforcement-learning/agent/asyncrl/policy_output.py", line 51, in action_indices return _sample_discrete_actions(self.probs.data) File "/media/bignrz/World/projects/carla RL envs/reinforcement-learning/agent/asyncrl/policy_output.py", line 27, in _sample_discrete_actions histogram = np.random.multinomial(1, batch_probs[i]) File "mtrand.pyx", line 4199, in numpy.random.mtrand.RandomState.multinomial File "_common.pyx", line 324, in numpy.random._common.check_array_constraint ValueError: pvals < 0, pvals > 1 or pvals contains NaNs
any help please.

When I run python run_RL.py on windows10, an error occurs:
ModuleNotFoundError: No module named 'carla.driving_benchmark'
Does anyone know how to solve it ?