Comments (1)
Thanks for your interest. To answer your questions:
My question is how these variables are transformed into their final form. I noticed in the code above that you've defined some variables to scale these positional variables, such as _n_discrete_x_bins, _n_discrete_y_bins, etc.
Our model predicts indices of discrete bins, which are subsequently converted to continuous values in the range of [0, 1]. These are then translated to the coordinates in the simulated workspace.
Suppose I try the VIMA algorithm on a physical robot using my camera. How would my actions be transformed from the form below to the final grasping position? Also, the values below don't appear to be in pixel coordinates.
Since the model outputs coordinates relative to camera frame, we need to transform to world frame to get grasping positions.
from vima.
Related Issues (20)
- [Evaluation] How to get the result reported in the paper
- [Test] The test result is not consistent with that reported in paper. HOT 11
- Question of the object encoding and training details HOT 1
- Request for Training Code and Clarification on Parallelization & Batch Size HOT 1
- Some questions about the input observation HOT 1
- Request for Guidance on Implementing imitation_loss HOT 3
- Question: how to specific state or restore the state of the environment?
- RuntimeError: Failed to reset environment after 10 retries HOT 3
- Exception: Model "t5-base" on the Hub doesn't have a tokenizer HOT 4
- The inference process of the VIMA strongly relies on the images in the prompts and objects in the environment HOT 1
- Franka in Isaac-sim HOT 1
- The order of object token is always same
- ImportError: cannot import name 'checkpoint' from 'transformers.models.t5.modeling_t5' HOT 15
- Asking for better explanation HOT 2
- Action dim not match HOT 2
- How to eval the baseline? HOT 1
- Implementation details of the loss scaling algorithm HOT 1
- Invalid GLX version: major 1, minor 2 HOT 1
- Metrics related to evaluating the models.
- How is the current inference frequency? Did someone try it on real world experiment?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vima.