Question: Inpainting model for SDXL about generative-models HOT 20 OPEN

+1

from generative-models.

@landmann If you are referring to small changes, than it is most likely due to the encoding/decoding step of the pipeline. We bring the image in a latent space (containing less information than the original image) and after the inpainting we decode it back to an actual image but in this process we are losing some information (the encoder is lossy as mentioned by the authors). This is expected behavior.

Workaround can be found here: huggingface/diffusers#3514 (comment)

from generative-models.

Is there any in-paint configs for SDXL 1.0 (base)?

from generative-models.

A yaml file for SDXL-inpainting is desperately needed.

from generative-models.

I will much appreciate any news about inpainting too!

from generative-models.

+1

from generative-models.

Waiting for the inpainting model for SDXL, too!

from generative-models.

it will be v-prediction and zero terminal SNR, most likely. i think that will come with 1.1.

from generative-models.

when to release the inpainting model？ there will be much use of the inpainting model

from generative-models.

Thanks @bghira! Is there a timeline for this? Also, is the SDXL-base model v-prediction too or is that also in the 1.1 release?

from generative-models.

SDXL can already be used for inpainting, see: https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/stable_diffusion_xl#inpainting

To me it seems like they are using the base model for inpainting.

Will there be a fine tuned version like stable-diffusion-2-inpainting?

from generative-models.

Anyone notice inpainting with a mask alters the image the mask says it should not be altering? Is this expected?

from generative-models.

Haha good ol ctrl+c ctrl+v - love it.

from generative-models.

Is it possible to finetune SDXL 1.0 base model for inpainting?

I tried to do that but I get a error from UNET that it expects the input to have 4 channels but it got 9 channels (4 - input image, 4 - masked image, 1 - mask (B/W))

Am I doing anything wrong or should I wait for inpainting model release?

from generative-models.

if you're feeling adventurous you can modify the pix2pix finetuner from Diffusers project.

it already uses 8 channel input.

from generative-models.

+1

from generative-models.

+1 im waiting for inpainting model of SDXL

from generative-models.

SDXL INPAINTING MODEL has been out!

from generative-models.

yes pls !

from generative-models.

new inpaint model
https://huggingface.co/wangqyqq/sd_xl_base_1.0_inpainting_0.1.safetensors/blob/main/sd_xl_base_1.0_inpainting_0.1.safetensors

its also HALF
so 16GB RTX nvidia doesnt work !

from generative-models.