tkn-tub / veins-gym Goto Github PK

After setting up a custom environment, I was testing whether my observation_space and action_space were properly defined. I was able to call:

• env.observation_space and get the properly defined observation_space

• env.observation_space.sample() and get a well-working sample

Though when calling env.observation_space.shape, I got "None" as a return which doesn't make sense considering I could sample it properly.

When I run:
obs_space = env.observation_space
action_space = env.action_space
print("The observation space: {}".format(obs_space))
print("The action space: {}".format(action_space))

I got:
The observation space: None
The action space: None

Because of this error, I could not use these codes for example:

state_shape = env.observation_space.shape
or
state_shape= env.observation_space.n

I got en error:
AttributeError: 'NoneType' object has no attribute 'shape'

In order to set my DQN, I need this.

What am I doing wrong?

Regards,
Serap

Sorry for bothering you again, Mr Bose.

When I was using veinsgym, I changed that run.sh in a way to run my scenario directly, a compiled file in the root of the project. It worked fine there.

But now I created another project with src and simulations (one of the defaults of the omnet). So there exists a file also called run with the following content in simulations dir (it's name is "run" but I will write runX to make easier to understand in the figure below):

#!/bin/sh
cd `dirname $0` ../src/aibasedoffloadings -n .:../src $*
# for shared lib, use: opp_run -l ../src/aibasedoffloadings -n .:../src $*

This is done by omnet. When I copy my veinsgym scenario in the simulations folder, it will have its name, say, rltask and inside of it I put the run.sh and the .ini file (alongside with the .ned files).

root
 | -out
 | - src
 | - simulations
      | - rltask
           | - run.sh (exec ../runX "$@")
 |    | - runX

If I try to use the default run from omnet referencing it in run.sh, the python scripts starts but keeps hanging on the listening part. However everything is alright, as they are the same files that worked before with the rltask when I was running directly from the root in the other open project. So I already know it should work, but if I try to run using the default run script of omnet, it hangs.

I believe you have a reason to have created that particular run you shared along both serpentine and dcc, so maybe you know why this is happening.

I would like also to say that, in this project I am Project Referencing INET and VEINS, they aren't libs in a subdirectory inside the project, but opened and built projects on a different dir. I am saying this because, in both serpentine and dcc, the script "run" Mr Bose wrote considers a lib dir with veins inside, for instance. I am not quite sure how to modify the script to take into account these differences as INET and VEINS are referenced and outside project dir. I tried to use the default run because of this.

(In my other project, run.sh would have exec ../rltask "$@", and rltask-[x86_84/le] compiled file was copied in the root dir of the project, and by that I know it is not in the python script or in the omnet .ned, .cc,.h files I used, as it is working fine).

Either way Mr Bose could help me, I will be glad.
I hope it is something simple.

One would like to start separated process directly from omnet++ and use with veins gym.
That is possible, however, at line 224 when an episode is done or when done is true on general , the code tries to wait() a process that does not exist if you never launched one from RL agent. Program crashes.
So, that call could be changed to:

if self.veins:
    self.veins.wait()

Changed to that in my source code and it worked as intended with no apparent side effect.

Hi,
when trying to use Veins-Gym with Gym version 0.24.1, I get the following error message:

AssertionError: Observation space (None) does not inherit from gym.spaces.Space

This occurs when calling

env = gym.make("veins-v1")

right after registering Veins-Gym as an environment using:

gym.register(
	id="veins-v1",
	entry_point="veins_gym:VeinsEnv",
	kwargs={
		"scenario_dir": "scenario",
		"print_veins_stdout": True
	},
)

This issue does not occur with Gym version 0.24.0. The Gym changelog of version 0.24.1 mentions a change to the environment checker that might be the reason for this problem:

Replaced the environment checker introduced in V24, such that the environment checker will not call step and reset during make. This new version is a wrapper that will observe the data that step and reset returns on their first call and check the data against the environment checker.

As far as I understood it, Veins-Gym initializes the observation and action spaces after communicating with the Veins simulator for the first time, which happens when calling the reset() function. This method might no longer work for newer Gym versions.

It is possible to work around this problem by disabling the environment checker in the gym.make() command like this:

env = gym.make("veins-v1", disable_env_checker=True)

Best regards,
Daniel

I've installed Veins-Gym and have run the sample program for the serpentine model, but now I'm looking to use Veins-Gym with a different project (which includes several .cc files that collect data I'd like to use within the reinforcement learning) and run the simulation during the reinforcement learning process.

I haven't found any detailed instructions on how to do so in the serpentine project or in the Veins-Gym files; there's a Jupyter notebook file within the serpentine project that goes through it, but when it talks about the scenario there's a line saying "See TODO: add link to other tutorial here to find out how to write your own." I was wondering if there's supposed to be a link with instructions there or anywhere else that I've missed or if anyone would be able to give an explanation on how to integrate Veins-Gym with another project and use current simulation data from that project's files during the reinforcement learning. Thank you!

Hello,
I have a problem that when I do the training, that after like a thousand episode or a random number, it generates an error while doing env.reset(), even though it works for alot of episodes, how can I find the reason? The error is shown below. Please let me know how can I fix this. Thanks
`RROR:root:Veins instance with PID 3784104 timed out after 15.00 seconds

TimeoutError Traceback (most recent call last)
in
69
70 while(i!=n_episodes):
---> 71 observation = env.reset()
72 done = False
73 score = 0

~/.local/lib/python3.8/site-packages/gym/wrappers/order_enforcing.py in reset(self, **kwargs)
16 def reset(self, **kwargs):
17 self._has_reset = True
---> 18 return self.env.reset(**kwargs)

~/.local/lib/python3.8/site-packages/veins_gym/init.py in reset(self)
273 self._veins_shutdown_handler = veins_shutdown_handler
274
--> 275 initial_request = self._parse_request(self._recv_request())[0]
276 logging.info("Received first request from Veins, ready to run.")
277 return initial_request

~/.local/lib/python3.8/site-packages/veins_gym/init.py in _parse_request(self, data)
351 self.socket.send(init_msg.SerializeToString())
352 # request next request (actual request with content)
--> 353 real_data = self._recv_request()
354 real_request = veinsgym_pb2.Request()
355 real_request.ParseFromString(real_data)

~/.local/lib/python3.8/site-packages/veins_gym/init.py in _recv_request(self)
331 self._timeout,
332 )
--> 333 raise TimeoutError(
334 f"Veins instance did not send a request within {self._timeout}"
335 " seconds"

TimeoutError: Veins instance did not send a request within 15.0 seconds

`

Thank for you job, very nice.
Did it support simulation of C-V2X (LTE or NR)

I would like to share my experience of failed installation by snakemake -jall

Environment

ubuntu 22 LTS
protoc 3.20.0，3.20.1，3.21.1

pip show protobuf 
Version: 3.20.1

The first error in snakemake -jall :

/usr/bin/ld: /usr/local/lib/libprotobuf.a(gzip_stream.cc.o): undefined reference to symbol 'inflateEnd'

What I did:

1. copy files in protoc-3.20.1-linux-x86_64.zip to the sub folder in /usr/(sub folders)
2. protobuf-3.21.1/python$ pip install .

protobuf-3.21.1/cmake$ cmake .
protobuf-3.21.1/cmake$ make -j12
protobuf-3.21.1/cmake$ sudo make install

4. See .drone.yml and sudo apt-get --allow-releaseinfo-change update && sudo apt-get install -y python3-pip python3-wheel protobuf-compiler libprotobuf-dev libzmq3-dev

The second error in snakemake -jall :

#error This file was generated by a newer version of protoc which is

Correct installation method

Because 2 versions of protoc are installed, one of which is installed through the .drone.yml statement. So Firstly sudo apt remove libprotobuf-dev protobuf-compiler-grpc
Download protobuf-all-21.1.tar.gz
unzip
Open protobuf-3.21.2/src/README.md
We can see in document like this:

     ./configure
     make -j$(nproc) # $(nproc) ensures it uses all cores for compilation
     make check
     sudo make install
     sudo ldconfig # refresh shared library cache.

Delete serpentine-env-master folder and re-extract serpentine-env-master.zip
Maybe very important but easily overlooked is the last sentence "sudo ldconfig". Calm down and read the README is very important

Hello, I am using env.reset() whenever an episode finish, so basically its not a clean shutdown from omnett. I got this output after around 20k episode, is there any specific reason you might know for that?
Thanks in advance

WARNING:root:Veins process 972485 did not shut down gracefully, sending kill.
ERROR:root:Veins process 972485 could not even be killed!

AssertionError Traceback (most recent call last)
in
67 #for i in range(n_episodes):
68 while(i!=n_episodes):
---> 69 observation = env.reset()
70 done = False
71 score = 0

~/.local/lib/python3.8/site-packages/gym/wrappers/order_enforcing.py in reset(self, **kwargs)
16 def reset(self, **kwargs):
17 self._has_reset = True
---> 18 return self.env.reset(**kwargs)

~/.local/lib/python3.8/site-packages/veins_gym/init.py in reset(self)
233 Waits until first request from veins experiment has been received.
234 """
--> 235 self.close()
236 self.socket = self.context.socket(zmq.REP)
237 if self.port is None:

~/.local/lib/python3.8/site-packages/veins_gym/init.py in close(self)
295 if self.veins:
296 # TODO: send shutdown message (which needs to be implemted in veins code)
--> 297 shutdown_veins(self.veins)
298 self.veins = None
299

~/.local/lib/python3.8/site-packages/veins_gym/init.py in shutdown_veins(process, gracetime_s)
126 "Veins process %d could not even be killed!", process.pid
127 )
--> 128 assert (
129 process.poll() and process.returncode is not None
130 ), "Veins could not be killed."

AssertionError: Veins could not be killed.

Hi,
In order to use veins-gym framework with federated learning, I was thinking about how to make it run with multiple instances of openAI environments. I going to start to work on this but, since it is not the major part of my thesis, I may try to find the easiest working solution for now.

What I need is to have different, separated working instances of agents, and therefore, it seems that I need different GymConnection modules working in parallel each of them connected to a different port.

I was also thinking how I could separate the configuration part from the learning part itself, as, for now, the tool receives a certain message that means configuration.

If you have any suggestion regarding this, I will be very glad to think about how I could make it.
Thank you very much.

After setting up the project to use veinsgym. I am getting the bellow error. Do you know what could be the issue? I have checked and everything looks ok. the Define_Module is there, the makefile, I am using CUSTOM the one provided in the folder. I see the obj are ok.

I used OMNet++ 5.6, and 5.7 same error.

A runtime error occurred:

Class "GymConnection" not found -- perhaps its code was not linked in, or the class wasn't registered with Register_Class(), or in the case of modules and channels, with Define_Module()/Define_Channel() -- in module (omnetpp::cModule) SerpentineScenario2 (id=1), during network setup

Launch a debugger with the following command?

nemiver --attach=10758 &

When I change the observation space, I am getting this error:
"AssertionError: The reward returned by step() must be a float" via env.checker

But before, the program was running well. And also, my reward is described as float, too.
What should be done?
Regards,

Hello, Firstly, Thanks for this great work!
Secondly, from what I have seen during my study to veins-gym, is that the shutdown command will always be associated with zero reward and a random sample from the observation space.is I would like to suggest if you can add a reward that can be sent from the simulation with the terminal state, since alot of environments have some reward for reaching the terminal state.
I over come this issue by the following code:

if (accident == true){
			reward_last = -15;
			std::array<double, 6> observation_last= {0, 0, 0, 0, 0, 0};
		        const auto request = serializeObservation(observation_last, reward_last);
		        auto response = gymCon->communicate(request);
		        veinsgym::proto::Request request2;
			request2.set_id(1);
			*(request2.mutable_shutdown()) = {};
			auto response2 = gymCon->communicate(request2);
			}

My problem now in some cases, it send the first request, but then it gets idle in the notebook without sending the shutdown, and I cant figure out why, maybe you have any idea?
In the picture below, you can see the output from my environment, the reward = -15 is sent, but then it doesnt shutdown!

Thanks for your help!

I have a problem that env.reset() is not running.

I have my action and observation space defined as the following:

However, whenever I run the Python file, I get the following error:

It says "Unknown space type", so I checked the line that causes the error:

The space I set seems correct, and little has been changed from the serpentine example.
I checked by printing out the values in the following lines:

Any suggestions?