nftblackmagic / diffusion-tryon-trainer Goto Github PK

Diffusion-Tryon-Trainer

License: Other

Shell 0.50% Python 91.88% C++ 2.70% Cuda 4.80% Dockerfile 0.07% Makefile 0.02% C 0.04%

diffusion-tryon-trainer's Introduction

This is a unoffical training code for OOTDiffusion

This repository contains the training code for the OOTDiffusion project. We extend our gratitude to the contributions of OOTDiffusion and have built upon this foundation by utilizing Huggingface's Diffusors library to implement training on the VTON dataset for virtual try-on. Our project aims to enhance the accuracy and realism of virtual try-ons through cutting-edge diffusion model technology, providing users with a more authentic try-on experience.

Some results

Install requirements

Python env

conda env create -f environment.yaml
conda activate groot

Data Preparation

VITON-HD

Download VITON-HD dataset
Download pre-warped cloth image/mask from Google Driver or Baidu Cloud and put it under your VITON-HD dataset
Download cloth captions from train Google Driver test Google Driver

After these, the folder structure should look like this (the unpaired-cloth* only included in test directory):

├── VITON-HD
|   ├── test_pairs.txt
|   ├── train_pairs.txt
│   ├── [train | test]
|   |   ├── image
│   │   │   ├── [000006_00.jpg | 000008_00.jpg | ...]
│   │   ├── cloth
│   │   │   ├── [000006_00.jpg | 000008_00.jpg | ...]
│   │   ├── cloth-mask
│   │   │   ├── [000006_00.jpg | 000008_00.jpg | ...]
│   │   ├── cloth-warp
│   │   │   ├── [000006_00.jpg | 000008_00.jpg | ...]
│   │   ├── cloth-warp-mask
│   │   │   ├── [000006_00.jpg | 000008_00.jpg | ...]
│   │   ├── unpaired-cloth-warp
│   │   │   ├── [000006_00.jpg | 000008_00.jpg | ...]
│   │   ├── unpaired-cloth-warp-mask
│   │   │   ├── [000006_00.jpg | 000008_00.jpg | ...]
|   |   ├── cloth_caption
│   │   │   ├── [000006_00.jpg | 000008_00.jpg | ...]

How to run training

bash train_ootd.sh

Inference over CP dataset

Weight preparation

Train the weights or download a pretrained weight from Huggingface The weights need to be put under checkpoints dir.

Run inference

sh inference_test_dataset.sh

Acknowledgements

Our unet code is directly from OOTDiffusion. We also thank DCI-VTON-Virtual-Try-On, our dataset module depends on it.

This code is only for study and research.

Contributors

_{nftblackmagic}
💻

_{Yu (Brian) Yao}
💻

_Stevada
💻

_{Dingkang Wang}
💻

_xiaoweilu
💻

diffusion-tryon-trainer's People

Contributors

Stargazers

Watchers

diffusion-tryon-trainer's Issues

loss does not converge

hello，this is a very good job, but when I train it, the loss function dose not converge. I use the command just like the train_ootd.sh,but comment the 8bit_adam and mixed_precision='fp16' ,and the batch size is 1. Are your training settings the same as in train_ootd.sh?

Training fail

Dea project owner,
I had a bug when training.

ValueError: Input and output must have the same number of spatial dimensions, but got input with spatial dimensions of [512] and output size of (64, 64). Please provide input tensor in (N, C, d1, d2, ...,dK) format and output size in (o1, o2, ...,oK) format.

Note: I tested code with val dataset, which is same train dataset

Contributors

@all-contributors please add @nftblackmagic for code

Training Issue

Dear project owner,
I traning with batchsize = 2 in rtx a5000 ada (32gb vram), but model only use 13Gb vram and loads very much ram (~120Gb Ram)

image size is 512*512? Why

Thanks for your training code.
About the image size, why not set 512*384?

about inference

python run/run_ootd_cp_dataset.py
--gpu_id=0
--model_type=hd
--image_scale=2.0
--num_steps=20
--num_samples=1
--batch_size=8
--resolution=512
--dataroot="./data/VITON-HD"
--test_data_list='test_pairs.txt'
--model_root="runwayml/stable-diffusion-v1-5"
--unet_root="./checkpoints/"
--vit_dir="openai/clip-vit-large-patch14"

--unet_root="./checkpoints/" ? is not use save train model?

how to get warp cloth?

i see the OOTDiffusion, they do not use warp cloth to train or test , so why use this part of dataset and how to get this data? (for ex . some warp model? )

How to use your trained weights?

Thanks for your great work. I have a question to ask as follows.

I download the weights you uploaded to the huggingface,

and it is in this format

.
├── unet_garm
│   ├── config.json
│   └── diffusion_pytorch_model.safetensors
└── unet_vton
    ├── config.json
    └── diffusion_pytorch_model.safetensors

while in the Diffusion-Tryon-Trainer/run/run_ootd_cp_dataset.py, it shows that the model_root in this format like /workspace/OOTDiffusion/checkpoints/ootd/ootd_hd/checkpoint-36000

How to use the unet_garm and unet_garm, and what's the differences between them?

Training for Dresscode Dataset

Hi! Thanks for your impressive job. Do you have plans to publish code for dresscode dataset(upperbody,lowerbody,dress)?

nftblackmagic / diffusion-tryon-trainer Goto Github PK

diffusion-tryon-trainer's Introduction

This is a unoffical training code for OOTDiffusion

Some results

Install requirements

Python env

Data Preparation

VITON-HD

How to run training

Inference over CP dataset

Weight preparation

Run inference

Acknowledgements

Contributors

diffusion-tryon-trainer's People

Contributors

Stargazers

Watchers

Forkers

diffusion-tryon-trainer's Issues

Recommend Projects

Recommend Topics

Recommend Org