Finetuning ImageBind with LoRA about imagebind HOT 7 OPEN

Great, thank you so much! I'll keep you posted depending on how far I get.

from imagebind.

Can I know how you run the model to generate images?

from imagebind.

^^^ upvoting WilTay1's question. also wondering if you know how to train ImageBind without using LoRA?

from imagebind.

^^^ upvoting WilTay1's question. also wondering if you know how to train ImageBind without using LoRA?

@ChloeL19 you can train the model without LoRA using ImageBind-LoRA. Simply remove the --lora argument when calling train.py and set --full_model_checkpointing. I don't have the resources to fine-tune it but it should work in theory. Try it out on the toy dataset provided in the repo, and later you can try to implement dataloaders for the original datasets

from imagebind.

Can I know how you run the model to generate images?

late, but there are many repositories that just replace CLIP for ImageBind
A good example is https://github.com/Birch-san/imagebind-guided-diffusion

from imagebind.

Nice work. Thanks
Could you a little explain why you used LoRA. Did the paper use it?

from imagebind.

There is a repo that uses LoRA to bind modalities to language.

from imagebind.