toddhester / rl-texplore-ros-pkg Goto Github PK

The package is not catkinized which makes it unusable for ROS Hydro. 

Original issue reported on code.google.com by [email protected] on 6 Feb 2014 at 11:26

Hi,

I have a quick question concerning the re-use of a model. The scenario i am learning from takes a while to execute the actions from the rl_agent hence I would like to be able to somehow save the learned experience and re-use it later on.
From what i understand, this can be done using rl_msgs/RLEnvSeedExperience.msg message where before starting a new episode, I feed the past experience to the rl_agent which then takes it into consideration for planning in the next episode, is this correct?
Now concerning the SeedExperience message itself.

Message that contains a seed experience to initialize the model
float32[] from_state
int32     action
float32[] to_state
float32   reward
bool      terminal

I do not understand why the from_state and to_state are vectors? Is this because the rl_agent also accept experience from multi dimensional state spaces? In my case, the state space has 1 dimension. Would it mean that from_state and to_state would be one dimensional vectors for each seed?

I would appreciate if anyone could help.

Thanks,

Thibault

Hi,

I am testing a simple example: the car velocity control with random set point. I have build all the code with no problems in ROS jade. I am running on a powerful MSI computer with 8 cores and 8GB of RAM (also 8GB of Swap Memory).

For the agent, I am running: rosrun rl_agent agent --agent texplore --lambda 0.6 --model m5tree --gamma 0.95 --planner parallel-uct --prints true --actrate 10

For the env, I am running: rosrun rl_env env --env carrandom --lag --stocstic --prints

As you can see, in this image:

The memory occupation is continuously increasing. When it fills up the memory, the swap memory starts to be filled. After swap is full, Ubuntu directly kills the process and I can't keep on simulating.

Is this a known issue? Any idea? (Any "push_back" or "new" statement in the code which I should check?)

Thanks in advance.

Ran 'catkin_make' under top-level dir, then cd into src/ and tried running 'make' and got this error:

$ make

[  0%] Built target _rl_msgs_generate_messages_check_deps_RLStateReward
[  0%] Built target std_msgs_generate_messages_lisp
[  0%] Built target _rl_msgs_generate_messages_check_deps_RLAction
[  0%] Built target _rl_msgs_generate_messages_check_deps_RLEnvDescription
[  0%] Built target _rl_msgs_generate_messages_check_deps_RLEnvSeedExperience
[  0%] Built target _rl_msgs_generate_messages_check_deps_RLExperimentInfo
[  5%] Built target rl_msgs_generate_messages_lisp
[  5%] Built target std_msgs_generate_messages_py
[ 12%] Built target rl_msgs_generate_messages_py
[ 12%] Built target std_msgs_generate_messages_nodejs
[ 17%] Built target rl_msgs_generate_messages_nodejs
[ 17%] Built target std_msgs_generate_messages_eus
[ 24%] Built target rl_msgs_generate_messages_eus
[ 24%] Built target std_msgs_generate_messages_cpp
[ 29%] Built target rl_msgs_generate_messages_cpp
[ 29%] Built target rl_msgs_generate_messages
Scanning dependencies of target agentlib
[ 30%] Building CXX object rl_agent/CMakeFiles/agentlib.dir/src/Agent/DiscretizationAgent.cc.o
In file included from /home/dustin/code_projects/rl-texplore-ros-pkg/src/rl_agent/include/rl_agent/DiscretizationAgent.hh:4,
                 from /home/dustin/code_projects/rl-texplore-ros-pkg/src/rl_agent/src/Agent/DiscretizationAgent.cc:1:
/home/dustin/code_projects/rl-texplore-ros-pkg/src/rl_common/include/rl_common/Random.h:1129:23: error: ‘constexpr’ needed for in-class initialization of static data member ‘const float Random::_F’ of non-integral type [-fpermissive]
 1129 |    static const float _F    = 1. / _M;
      |                       ^~
make[2]: *** [rl_agent/CMakeFiles/agentlib.dir/build.make:63: rl_agent/CMakeFiles/agentlib.dir/src/Agent/DiscretizationAgent.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1443: rl_agent/CMakeFiles/agentlib.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

Solution:

change line 1129 of rl-texplore-ros-pkg/src/rl_common/Random.h to remove 'static' to be:

const float _F = 1. / _M;

After this change, running make worked just fine.

I met this error after compiled the rl_agent and rl_env code ,I could neither run 'rosrun rl_agent agent --agent qlearning' nor 'rosrun rl_env --env taxi', and get the same error 'couldn't find executable name xxx'