Details about training part about editanything HOT 6 OPEN

1.The txt and json files can be refer to https://github.com/lllyasviel/ControlNet
2.The latest verision is trained with laion dataset, so the training code is a little bit differet, we will release it soon.
3.More training details will be added.

from editanything.

1.The txt and json files can be refer to https://github.com/lllyasviel/ControlNet 2.The latest verision is trained with laion dataset, so the training code is a little bit differet, we will release it soon. 3.More training details will be added.

Thanks for your reply,

Since in this project, mask image is taken as an extra source input to the model, so will that change the training part, such as the json file?

Also, if I want to add one more image as a guidance such as clothing item, is it possible to incorporate this in the training and inference framework?

from editanything.

@ukaukaaaa @gasvn Im also interested on this.

A training guide to train JSON COCO (with annotation and segmented masks) would be super helpful!

from editanything.

1.The txt and json files can be refer to https://github.com/lllyasviel/ControlNet 2.The latest verision is trained with laion dataset, so the training code is a little bit differet, we will release it soon. 3.More training details will be added.

Thanks for your reply,

Since in this project, mask image is taken as an extra source input to the model, so will that change the training part, such as the json file?

Also, if I want to add one more image as a guidance such as clothing item, is it possible to incorporate this in the training and inference framework?

Sorry I missed your comments. I am not sure about the exact form of how to use "one more image" as the guidance. Maybe the biggest problem here is the training data. Also, you can check our latest version, where I achieve a training free drag item from one image into another based on reference scheme. But of course, there are some failure cases as no training is evolved. With training data, I think the current pipeline would be improved.

from editanything.

@ukaukaaaa @gasvn Im also interested on this.

A training guide to train JSON COCO (with annotation and segmented masks) would be super helpful!

The training data is the same to the basic controlnet. As long as you can construct a dataloader that produce the data with jpg = item['jpg']
txt = item['txt']
hint = item['hint']， you can use this training code.

from editanything.

from editanything.