AttributeError: 'DeviceArray' object has no attribute 'observation' about jax-rl HOT 21 CLOSED

Can you please give some details about:

• What command you're running
• Any local changes you've made
• The machine you're running on (OS, CPU manufacturer, GPU if applicable)

from jax-rl.

I am trying to implement custom env with your code. I now get:

AttributeError Traceback (most recent call last)
/tmp/ipykernel_46/149212032.py in
49 train_loop = TRAIN_LOOPS[args.policy]
50
---> 51 train_loop(args, policy, replay_buffer, env)

/tmp/ipykernel_46/1511689711.py in base_train_loop(args, policy, replay_buffer, env)
1 def base_train_loop(args: dict, policy, replay_buffer: ReplayBuffer, env):
----> 2 evaluations = [eval_policy(env, policy, args.domain_name, args.task_name, args.seed)]
3
4 timestep = env.reset()
5 episode_reward = 0

/tmp/ipykernel_46/3982767400.py in eval_policy(env, policy, domain_name, task_name, seed, eval_episodes)
7 for _ in range(eval_episodes):
8 timestep = eval_env.reset()
----> 9 while not timestep.last():
10 action = policy.select_action(flat_obs(timestep.observation))
11 timestep = eval_env.step(action)

AttributeError: 'DeviceArray' object has no attribute 'last'

Why would resetting the env return the total timestep?
timestep = eval_env.reset()
This should return observation?

from jax-rl.

In addition, how can I use env/custom env without dm_control suite which requires mojoco engine?

from jax-rl.

I am trying to implement custom env with your code. I now get:

If possible, try changing your environment to adhere to the same format as the DeepMind Control Suite. You can find their base Environment class here:
https://github.com/deepmind/dm_control/blob/4cf89ef1a0d99f7e9f70b3f5ad0bf2a685cbd935/dm_control/rl/control.py#L28
And the timestep object should be a NamedTuple as defined here: https://github.com/deepmind/dm_env/blob/abee135a07cc8e684173586dc8a20e696bbd40fb/dm_env/_environment.py#L25

Hope this helps!

from jax-rl.

In addition, how can I use env/custom env without dm_control suite which requires mojoco engine?

I think my above answer should help with adding a custom env. MuJoCo is now free and open source if that was your main concern. The README of this repo has details on how to install.

from jax-rl.

Mojoco is a pain to install on linux and I don't really need it. Is there a workaround?

from jax-rl.

For a custom env as long as your timestep object has the same format as the class I linked above it should work but I haven't tested it

from jax-rl.

Is it possible to modify your code to not using dm_control at all and use the original gym env?

from jax-rl.

As long as you have your own environment that you can initialize in place of the calls to suite.load you won't have to make any other changes. In terms of the code I linked above it doesn't depend on other libraries that much so you should be able to just copy those classes or the general structure of them and wrap your environment in it.

There's also this package but it's the wrong direction from what you're looking for. Might still be useful to consult for the differences.

from jax-rl.

I am not sure that I understood your last post. My custom env is a gym.env and I can instantiate it. What other changes to my or your code do I need?

from jax-rl.

I made a PR adding gym support. Please let me know if this works for you, you'll need to use the new file main_gym.py instead of main_dm_control.py: #13 I'll work on cleaning it up and merging it this week.

from jax-rl.

from jax-rl.

You can pull the branch from that PR, gym-support and test it now.

from jax-rl.

from jax-rl.

I need to clean the branch up before merging it. Did you run make init first? Also the best way to get the new branch locally would not be to clone it but to:

git checkout master
git pull
git checkout -b gym-support origin/gym-support

from jax-rl.

from jax-rl.

It builds a new poetry environment and downloads all dependencies as well as initializes jax-rl as a local python package

from jax-rl.

from jax-rl.

By copying the code into the notebook? I don't think I can help you with that part of the process

from jax-rl.

from jax-rl.

from jax-rl.