Comments (4)
Hi @AntonBock
To test the policy in simulation or real-world you only need to compose the observation/state space with information from the environment (sensors, etc.) and apply the actions taken by the policy to the environment (controllers)...
So, it all depends on the environment you have...
Could you please provide more information about your simulated environment and the real one? Are you planning to use a middleware to control the objective in your environment, for example, ROS?
from skrl.
Hi again,
Thanks for the quick response!
We use ROS to control a Franka Panda arm.
Using your ppo_Franka_Cabinet.py example, we have trained a Franka robot in Isaac Gym.
How could we modify the franka_cabinet example to use observations we get from ROS, instead of wrapping Gym and getting the observations from there?
from skrl.
Hi @AntonBock
I think the following questions are relevant for testing in real-world:
- Are you able to get, using ROS all the data required to build the observation space (according to the
FrankaCabinet
task it isself.cfg["env"]["numObservations"] = 23
)? How do you pretend to control the robot: setting the joints directly using a ROS topic or with MoveIt? - How important is it to return the instantaneous reward at each time step in the real-world test environment? Are you able to compute it using the information from the real world?
- Is your real-world system capable of working (retrieving sensor information and controlling the robot using ROS) at the frequency at which the agent was trained in the simulation (
dt: 0.0166 # 1/60 seconds
)?
Well... I think creating a separate environment, for testing in real-world, that uses ROS to build the observation space and control the robot, is the best solution... Currently, I am creating a testing environment (in the real world) using ROS... I think we can discuss its implementation here the day after tomorrow...
from skrl.
Hi @Toni-SM
Those are some of the same considerations we have made, and we should be able to get all the required information and rewards at the correct frequency.
We look forward to hearing about your ROS environment tomorrow
from skrl.
Related Issues (20)
- [Error] PPO on a single environment HOT 1
- ModuleNotFoundError: No module named 'skrl.envs.wrappers' HOT 1
- Getting blank stderr while training with OIGE HOT 1
- No module named omni.isaac.contrib_envs and omni.isaac.orbit_envs HOT 1
- Mean rewards are not calculated properly
- Random action only samples from the first action space dimension
- dm_control wrapper HOT 1
- Failing to wrap Isaac Gym Preview 4 Environments if No Wrapper Type is Given HOT 1
- Documentation -> WrapperTag: "isaaclab" is wrong, write "isaac-orbit" instead HOT 2
- Wall clock time in Isaac Gym benchmarks? HOT 1
- How to implement the curriculum learning using the existing data
- [Question] Is there support for MARL? HOT 1
- [Question] TD3_DEFAULT_CONFIG HOT 3
- "pygame.error:display Surface quit" in train.eval under render_mode="human" HOT 4
- Error Running Orbit Example HOT 3
- A little bug on environment wrapper
- [Bug] Indexing issue in memory sampling function HOT 2
- Environment reset function HOT 4
- I notice that MAPPO is supportted in 0.11.0 HOT 1
- PPO discrete action for gym.Env HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from skrl.