Does this work with IPAdapter node? about comfyui-layerdiffuse HOT 5 OPEN

I can confirm that using attn injection is going to yield worse results. The reason behind is that IPAdapter also injects attn blocks, so there might be conflict on attn weights trying to adapt to 2 tasks at the same time.

from comfyui-layerdiffuse.

Tested that it works for IP-Adapter. Just make sure your input to IPAdapter should ideally have simple background or transparent background so that there are not too much noise information from the background getting captured.

ld_ipadapter.json

from comfyui-layerdiffuse.

Got it. Thank you!

from comfyui-layerdiffuse.

Actually, just to follow your example, has the attention injection ever work for you? If I choose to use attention injection, the rendering seems broken to me, which is very different than the author mentioned here:

https://github.com/layerdiffusion/sd-forge-layerdiffuse
However, in practice, I find the layer_xl_transparent_attn.safetensors will lead to better results. This layer_xl_transparent_conv.safetensors is still included for some special use cases that needs special prompt understanding. Also, this model may introduce a strong style influence to the base model.

from comfyui-layerdiffuse.

It does seem like using Conv will impact the rendering quality. I'm wondering if there's any better solution to this.

from comfyui-layerdiffuse.