cubiq / diffusers_ipadapter Goto Github PK

Hi and thank you for your amazing job.
I'm aware that with comfyui, it's possible to use multiple IPAdapter instances. However, I'm looking for insights on how to achieve this with diffusers and StableDiffusionPipelines.

1. Can I use two IPAdapter instances within a single StableDiffusionPipelines pipeline in the diffusers library?
2. If possible, how can I achieve this?
   thank you

I can't find the attribute named get_prompt_embeds in ip_adapter.py, where is it

I came across this video: https://www.youtube.com/watch?v=HtmIC6fqsMQ and this one: https://www.youtube.com/watch?v=7m9ZZFU3HWo and the results look great. IIUC, they're using multiple images with a single IP Adapter instance. So I went hunting for how to do this with A1111, but it looks like the most popular extension ( https://github.com/Mikubill/sd-webui-controlnet ) doesn't currently allow for multiple images.

I came across your comment here and am wondering if there's some way to integrate this with that A1111 extension?

I've not looked "under the hood" on IP adapter, this repo, or the controlnet extension, so I'm sorry if I've got some misunderstandings here! I'm very much toward the "end user" end of the spectrum.

Hey great repo! I came across your comfyui workflow using attention mask (or multiple masks for different regions). Wondering if this would be implemented for diffusers as well? If not, would you mind providing some hints on how I can connect attention masks to ip adapters? Thanks!

Hello, I'm super glad you implmented ip-adapter for diffuser. However, i'm missing the Attention masking. Is there any way to add this to the diffuser version?

Hi which IP Adapter models are we supposed to use for SDXL with this ?

I've tried all three SDXL models at
https://huggingface.co/h94/IP-Adapter/tree/main/sdxl_models
and they've all failed with...

[/content/Diffusers_IPAdapter/ip_adapter/ip_adapter.py](https://cn6dw4sgihk-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab_20231019-090936_RC00_574892596#) in __init__(self, pipe, ipadapter_ckpt_path, image_encoder_path, device, dtype, resample)
     35 
     36         # detect features
---> 37         self.is_plus = "latents" in ipadapter_model["image_proj"]
     38         self.output_cross_attention_dim = ipadapter_model["ip_adapter"]["1.to_k_ip.weight"].shape[1]
     39         self.is_sdxl = self.output_cross_attention_dim == 2048

KeyError: 'image_proj'

Can we get support for Instant style