Comments (2)
yunfanjiang
commented on August 17, 2024
Thank you for your interest in our work! T5-large model uses an embedding dim of 1024, instead of 768 that T5-base uses. Therefore, loading the provided VIMA ckpts will raise size mismatch error.
That being said, there are still several workarounds. For example, you can avoid loading weight for t5_prompt_encoder. But pay attention to this adapter layer which reconciles different embedding sizes. Let me know if you have further questions.
from vima.
yunfanjiang
commented on August 17, 2024
I'm closing this issue. Feel free to reopen if you encounter further questions.
from vima.
Related Issues (20)
- [Evaluation] How to get the result reported in the paper
- [Test] The test result is not consistent with that reported in paper. HOT 11
- Question of the object encoding and training details HOT 1
- Request for Training Code and Clarification on Parallelization & Batch Size HOT 1
- Some questions about the input observation HOT 1
- Request for Guidance on Implementing imitation_loss HOT 3
- Question: how to specific state or restore the state of the environment?
- RuntimeError: Failed to reset environment after 10 retries HOT 3
- Exception: Model "t5-base" on the Hub doesn't have a tokenizer HOT 4
- The inference process of the VIMA strongly relies on the images in the prompts and objects in the environment HOT 1
- Franka in Isaac-sim HOT 1
- The order of object token is always same
- ImportError: cannot import name 'checkpoint' from 'transformers.models.t5.modeling_t5' HOT 15
- Asking for better explanation HOT 2
- Action dim not match HOT 2
- How to eval the baseline？ HOT 1
- Implementation details of the loss scaling algorithm HOT 1
- Invalid GLX version: major 1, minor 2 HOT 1
- Metrics related to evaluating the models.
- How is the current inference frequency? Did someone try it on real world experiment?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vima.