Comments (6)
Hello Fabrice,
JAX's functional semantics make it a bit tricky to implement the gym API where the Env itself is supposed to maintain state, and still work for end-to-end training.
But the good news is that you can easily create gym Environments by calling create_gym_env
in place of create_env
, which will wrap the environment with a gym.Env
. See:
https://github.com/google/brax/blob/main/brax/envs/wrappers.py
from brax.
Thanks @erikfrey for the quick response.
Interesting. I'd suggest creating a simple example notebook/script where this gets used.
Other questions:
- Does this
GymWrapper
support rendering? If not, are there plans for adding support for it (which seems to already be present for the other envs?) - What is the right way of "scaling up"/vectorizing this
GymWrapper
? (I assume you wouldn't want users to pass it to one of thegym.vector.VectorEnv
subclasses, for instance?) - How well would you expect this
GymWrapper
to "scale" compared to the jax-only version?
from brax.
Right now we only support visualization via the three.js webview. Actual rendering - as in a python function that returns an image buffer - is on our roadmap! But don't have an ETA yet.
re: vectorizing, we have only used to gym wrapper for CPU training on multi-process envs, so no batching dimension in the gym env. That said, VectorEnv
class looks good. We could extend the create_gym_env
to return VectorEnv
if you specify a batch dimension. Feel free to submit a PR if that would be helpful to you.
performance: as long as you're careful to avoid copying data between CPU and accelerator (for example, by ensuring your learner is jitted too, and that the actions you pass to step
are on device), then there should be no runtime performance penalty to using a gym wrapper, just a very small jit time cost for a bit of extra function tracing.
from brax.
OpenAI/StableBaseline VectorEnv
automatically calls reset
on a worker when the episode finished.
In brax
however I guess that because how vmap
works one needs to check if a given episode in the batch is done and treat this on the agent side (e.g.: ignore data from finished episodes in the batch) since there is no way to reset a single episode in the batch, right?
Thank you!
from brax.
Hi @floringogianu - yes, we leave it up to the user to decide how to reset. For the case of training, we actually do reset individual episodes within a batch. We support this in the training env wrapper, see here:
https://github.com/google/brax/blob/main/brax/training/env.py#L57
from brax.
Thank you for the quick reply and for pointing to that example.
from brax.
Related Issues (20)
- softbody simulation HOT 1
- Ant (positional) initialization often inevitably leads to unhealthy state / early termination with a few timesteps HOT 1
- Load v1 configs in v2 for spring/positional backends HOT 1
- XlaRuntimeError when create barkour_v0_joystick env HOT 1
- Training Quadruped needs to be more stable HOT 1
- Apply external force to the agent HOT 2
- Discrepancy in Observation Dimension for Humanoid Environment HOT 2
- How do I load a pre-trained model? HOT 1
- Inconsistency in solved return for Humanoid environment HOT 1
- Manually Set Initial Pose of Agent in the Reset Function HOT 1
- Mesh-Mesh Collision Problems HOT 1
- gradient on trajectory vs sum of gradients on 1 step transitions. HOT 1
- Reacher Task and AutoResetWrapper HOT 1
- Programatic Action Specifications HOT 1
- Subsume part of System inside State; EDIT: Or add Options to reset HOT 11
- MjxEnv no where to be found HOT 2
- Broken links on Github's brax's main page HOT 2
- [MJX] Model loading errors in `brax.io`, for features supported by MJX HOT 1
- Rendering an image with brax.io.image HOT 6
- Dump to json fails if mj model contains no meshes HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from brax.