Make sure environments in SocialRobot work with tf_agents about alf HOT 5 CLOSED

1. SocialRobot should run in the same thread that it is created because of its inner implementation (gazebo::sensors) . There are three options to solve this to work with tf_agents: 1) move the creation from main thread to BatchedPyEnvironment 2) using ParallelPyEnvironment 3) wrap gazebo interface in one thread . Since gazebo can not spawn multi instances by multi-threads 1) and 3) is not necessery

2. tf_agents support parallel environments using multi-threads (BatchedPyEnvironment) or mutil-process (ParallelPyEnvironment)

3. example to work with tf_agents
   train_eval.py
   train_eval.patch diff with tf_agents/agents/ddpg/examples/v2/train_eval.py

from alf.

👍
Is it possible to make some wrap for socialrobot environment so that we don't need to change tf_agents code to run with social robot environment？ It will be cool if that's possible.

from alf.

Is it possible to make some wrap for socialrobot environment so that we don't need to change tf_agents code to run with social robot environment？ It will be cool if that's possible.

All examples in tf_agents (tf_agents/agents/*/examples/v*/*) creates at least two environments tf_env (created in self process by BatchedPyEnvironment or created in new process by ParallelPyEnvironment for collect data) and eval_tf_env (one instance created in self process for test ) .
Since many singletons implementation in gazebo and multi instances in one process is not supported, partial examples that created tf_env and eval_tf_env in the same process can not run without changing code . Besides, different port should be specified explicitly or generate randomly for each gazebo instance .

from alf.

A possible implementation would be to wrap socialbot using ProcessPyEnvironment in socialbot_suite. For port, we can automatically do increment from a given starting port (from environment). So it can be used in the same way as other environment.

from alf.

A possible implementation would be to wrap socialbot using ProcessPyEnvironment in socialbot_suite. For port, we can automatically do increment from a given starting port (from environment). So it can be used in the same way as other environment.

with commit update wrap with process and config.gin :

import alf.environments.suite_socialbot

train_eval.env_name = "SocialBot-Pr2Gripper-v0"
train_eval.env_load_fn = @suite_socialbot.load
train_eval.num_parallel_environments = 2

suite_socialbot.load.wrap_with_process = False # recommend set to False when num_parallel_environments > 2

we can run trainning as

python -m tf_agents.agents.ddpg.examples.v2.train_eval --gin_file=config.gin --root_dir=~/tmp/ddpg/socialbot/Pr2Gripper

from alf.