sergeywong / cp-vton Goto Github PK
View Code? Open in Web Editor NEWReimplemented code for "Toward Characteristic-Preserving Image-based Virtual Try-On Network"
License: MIT License
Reimplemented code for "Toward Characteristic-Preserving Image-based Virtual Try-On Network"
License: MIT License
I found the same images in both the train and test split.
For example,
/train/image/005749_0.jpg == /test/image/000010_0.jpg
/train/image/003574_0.jpg == /test/image/000001_0.jpg
复现实验后,觉得该方向技术和产生商业价值还有些距离。
Sorry if this is a dumb question, but if I have a brand new dataset of different clothes, how do I generate the masks for them? I am seeing that the matlab file is very customized for viton data. Is there a different way in python which allows us to generate the masks of the clothes?
I found error
Traceback (most recent call last):
File "train.py", line 191, in <module>
main()
File "train.py", line 182, in main
train_tom(opt, train_loader, model, board)
File "train.py", line 100, in train_tom
criterionVGG = VGGLoss()
File "/work/CP-VTON/cp-vton-master/networks.py", line 388, in __init__
self.vgg = Vgg19()
File "/work/CP-VTON/cp-vton-master/networks.py", line 356, in __init__
vgg_pretrained_features = models.vgg19(pretrained=True).features
TypeError: vgg19() got an unexpected keyword argument 'pretrained'
on torchvision 0.1.6
I was executing convert_data.m. Its creating cloth-mask images with white background images. Kindly help with this issue.
Hi, I was trying to run the Try-on-module test, and I got the following error:
File "test.py", line 163, in <module>
main()
File "test.py", line 156, in main
test_tom(opt, train_loader, model, board)
File "test.py", line 89, in test_tom
model.cuda()
...
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 258, in <lambda>
return self._apply(lambda t: t.cuda(device))
RuntimeError: Cannot initialize CUDA without ATen_cuda library.
I think this error accord because I am using a VM with no GPU support, and I installed the CPU version of Pytorch.
Is there a way to run your project without GPU support?
Are there any pre-trained models available for this ?
How can I change the tensorboard (in train.py/test.py) to show warped cloth overlayed on the person without making the warped cloth or the person transparent?
Hi @sergeywong,
As you can see below, there is minimal transformation and warping of the clothing. The cloth is not resized to fit the model either.
Other examples with similar issue:
What could be causing the issue here?
Hi @sergeywong,
I'm trying to train GMM on images of size 512, 384 - double of what has been used in cp-vton training. What changes would I need to make in networks.py?
I've tried a couple of things but they haven't shown promise:
Can you please let me know how can I train on images of size (512,384)?
Thanks.
When running the model on my own data and making the change as in #7 (permuting and unpermuting size dimension for im
and im_h
) I get the following error:
Traceback (most recent call last):
File "test.py", line 163, in <module>
main()
File "test.py", line 151, in main
test_gmm(opt, train_loader, model, board)
File "test.py", line 69, in test_gmm
grid, theta = model(agnostic, c)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(*input, **kwargs)
File "/mnt/efs/experiments/lucas_gans/cp-vton/networks.py", line 416, in forward
featureA = self.extractionA(inputA)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(*input, **kwargs)
File "/mnt/efs/experiments/lucas_gans/cp-vton/networks.py", line 73, in forward
return self.model(x)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(*input, **kwargs)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(*input, **kwargs)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 320, in forward
self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size [64, 22, 4, 4], expected input[4, 24, 256, 192] to have 22 channels, but got 24 channels instead
This was run after successfully running the model in the first instance. The model currently looks as follows:
Sequential(
(0): Conv2d(22, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(1): ReLU(inplace)
(2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(3): Conv2d(64, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(4): ReLU(inplace)
(5): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(6): Conv2d(128, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(7): ReLU(inplace)
(8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(9): Conv2d(256, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
(10): ReLU(inplace)
(11): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(13): ReLU(inplace)
(14): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(16): ReLU(inplace)
)
The weight of size [64, 22, 4, 4]
seems predefined by:
self.extractionA = FeatureExtraction(22, ngf=64, n_layers=3, norm_layer=nn.
self.extractionB = FeatureExtraction(3, ngf=64, n_layers=3, norm_layer=nn.BatchNorm2d)
The input expected input[4, 24, 256, 192] to have 22 channels
comes from the data size, which for some reason is 2 channels larger?
in train.py
RuntimeError: The expanded size of the tensor (1) must match the existing size (3) at non-singleton dimension 0
in test.py:
RuntimeError: The expanded size of the tensor (1) must match the existing size (3) at non-singleton dimension 0
RuntimeError: The size of tensor a (270) must match the size of tensor b (3) at non-singleton dimension 2
Hi,
I am getting a tensor dimension error while running test.py as below:
python test.py --name gmm_traintest_new --stage GMM --workers 4 --datamode test --data_list test_pairs.txt --checkpoint checkpoints/gmm_train_new/gmm_final.pth
The error seems to arise from cp_dataset.py module in line 91. Somehow the size/shape of the image read is (3, 360, 270) where the image is of 360x270 pix. This throws the following error:
I've fixed this issue by using permute on the object im in cp_dataset.py after the image has been read and transformed. Although it fixes the error shown in the screengrab above, I still have some issues with the implementation. Seems like the input size of image MUST be 256x192?
In paper, the author contends that
Body shape: a 1-channel feature map of a blurred binary mask that roughly
covering different parts of human body.
the body shape doesn't include the head, but clearly your codes are not for it.
parse_shape = (parse_array > 0).astype(np.float32)
Hey, I get these results after I've added Total Variation loss (https://discuss.pytorch.org/t/yet-another-post-on-custom-loss-functions/14552).
What can cause this problem?
Solution:
Change x = x.view(x.size(0), -1)
to
x = x.reshape(x.size(0), -1)
The code U = torch.mul(dist_squared,torch.log(dist_squared))
means r^2logr^2, but the tps RBF is r^2logr, is this an error?
When I train the 2 stages, I find that the loss changes in a small oscillation from beginning to end even if it falls in a general. I think it is unrelated to the fix of learning rate for I just use the origin learing rate in the code and the loss keep falling in the overall trend. Why is the loss always jumping?
0
Hello Authors,
I want to use your code in my project and I haven't found a license in the repo. I would like to know the license. Is it like VITON which is Apache V2 licensed?
Thanks in Advance
Hi @sergeywong,
While AffineGridGen is defined in the networks module, it is not used anywhere for affine warping. How can we incorporate this into TpsGridGen (TPS)?
Where would we need to make the changes in init and forward?
I'm trying to preprocess the data. I used Look-into-Person for human segmentation. As the model requires the images in "image-parse" to be of 1-channel, I saved the segmented images as grayscale images i.e. shape is (256, 192). But the problem is pixel values in my segmented images are different from the one that you get from this repo (data_download.py). It seems the author's preprocessed images in "image-parse" contains pixels with values between 0-19 only, while mine is in range of 0-255. There is some kind of mapping involved. How to map the pixel values in 0-19 range? Can anyone help me with this?
can you please explain these lines.
Before the trainning, you should generate warp-mask & warp-cloth, using the test process of GMM with --datamode train. Then move these files or make soft links under the directory data/train. An example training command is
When convert_data.m file is used to prepare data from VITON, the Image parse generated by it using mat file from segment folder is tilted by 90°.
Hi, there is a way to restart the training from a specific checkpoint?
I'm trying this command but it starts always from step 0
python train.py
--name gmm_train_new
--stage GMM
--workers 0
--save_count 5000
--checkpoint "checkpoints/gmm_train_new/step_065000.pth"
Because of dataset license, VITON Dataset is not available anymore.
There is no pre-trained model in CP-VTON... Isn't there any way to run your code?
Thx...
How to generate segment maps? I have no idea about this Question,could you help me?
Hi, I use your datasets to test your model. But I get this result.I don't know why and how to solve it?
Thank you.
I can't seem to find them anywhere but I can see that people have mentioned them before and have used them too. Could you please point me to the right direction? Thank you!
Hi,
I am trying to run CP-VTON, but I couldn't find the download link for the pretrained models. Could you please provide the download links or guide me on how to obtain the pretrained models?
Thank you!
Xxx
Thx for your job and sharing. It's really helpful to my research. Thanks again.
I have trained and tested your model successfully, and have a litte question about loss.
When I train in satge II, the mask loss almost have not change from the beigining to the end.
So, I check the calculation abouot mask loss, and find that the "loss_mask = criterionMask(m_composite, cm)" in line 133 of train.py is differernt from the equation (4) in paper.
Is there any detail about this I can konw?
Thank you a lot.
@sergeywong
hello! The dataset I am using in cp-vton is viton_resize, but the viton data is not available because "The dataset is no longer publicly available due to copyright issues. For thoese who have already downloaded the dataset, please note that using or distributing it is illegal!" and I understand that it is not available for the same reason. I was wondering if it is okay to publish a paper using the viton_resize dataset?
Hii @sergeywong ,
when I tested this model for custom images it gives an error, I have also implemented all the possible solution that mentioned in Issues like :
Here are all possible outcomes that I applied when implementing this model. Please help me to solve this problem.
Hi,
I have downloaded all the data with data_download.py
, but...
When I try to train CP-VTON GMM stage, I get this error immediately:
RuntimeError: output with shape [1, 256, 192] doesn't match the broadcast shape [3, 256, 192]
See the full log below. Any help much appreciated! (I get the same error if I run cp_dataset.py alone.)
Thanks,
Mike
C:\Users\Mike\AppData\Local\Programs\Python\Python36\python.exe M:/VITON/cp-vton-master/train.py --name gmm_train_new --stage GMM --save_count 5000 --shuffle
Namespace(batch_size=4, checkpoint='', checkpoint_dir='checkpoints', data_list='train_pairs.txt', datamode='train', dataroot='data', decay_step=100000, display_count=20, fine_height=256, fine_width=192, gpu_ids='', grid_size=5, keep_step=100000, lr=0.0001, name='gmm_train_new', radius=5, save_count=5000, shuffle=True, stage='GMM', tensorboard_dir='tensorboard', workers=1)
Start to train stage: GMM, named: gmm_train_new!
initialization method [normal]
initialization method [normal]
Traceback (most recent call last):
File "M:/VITON/cp-vton-master/train.py", line 191, in <module>
main()
File "M:/VITON/cp-vton-master/train.py", line 176, in main
train_gmm(opt, train_loader, model, board)
File "M:/VITON/cp-vton-master/train.py", line 58, in train_gmm
inputs = train_loader.next_batch()
File "M:\VITON\cp-vton-master\cp_dataset.py", line 166, in next_batch
batch = self.data_iter.__next__()
File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __next__
return self._process_data(data)
File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\dataloader.py", line 846, in _process_data
data.reraise()
File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\_utils.py", line 369, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\_utils\worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "M:\VITON\cp-vton-master\cp_dataset.py", line 86, in __getitem__
shape = self.transform(parse_shape) # [-1,1]
File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torchvision\transforms\transforms.py", line 61, in __call__
img = t(img)
File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torchvision\transforms\transforms.py", line 166, in __call__
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torchvision\transforms\functional.py", line 217, in normalize
tensor.sub_(mean[:, None, None]).div_(std[:, None, None])
RuntimeError: output with shape [1, 256, 192] doesn't match the broadcast shape [3, 256, 192]
Process finished with exit code 1
Dear Mr @sergeywong ,
As i far as i understand, your network reimplements GMM from this paper: https://arxiv.org/pdf/1703.05593.pdf . In the original GMM's paper the architecture was:
Stage 1:
ImageA -> featureExtraction -> fA
ImageB -> featureExtraction -> fB
then matching(fA, fB) -> AffineRegresstion(CAB) -> ThetaAff , then warp to imageA to become ImageWarpA.
Stage 2:
Repeat over featureExtraction ImageWarpA , then matching(fImageWarpA, fB) -> TBS Regression -> ThetaTPS.
Which matching is correlation layers plus L2-Norm.
However, in your network you implemented like this (please see the attached)
Can you please explain why it that, or just point out where i am missing you in the paper?
Thank you for your time.
Cheers!
Namespace(batch_size=4, checkpoint='', checkpoint_dir='checkpoints', data_list='train_pairs.txt', datamode='train', dataroot='data', decay_step=100000, display_count=20, fine_height=256, fine_width=192, gpu_ids='', grid_size=5, keep_step=100000, lr=0.0001, name='gmm_train_new', radius=5, save_count=50, shuffle=True, stage='GMM', tensorboard_dir='tensorboard', workers=4)
Start to train stage: GMM, named: gmm_train_new!
initialization method [normal]
initialization method [normal]
Traceback (most recent call last):
File "train.py", line 191, in
main()
File "train.py", line 176, in main
train_gmm(opt, train_loader, model, board)
File "train.py", line 70, in train_gmm
grid, theta = model(agnostic, c)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/content/drive/My Drive/cp-vton/networks.py", line 422, in forward
theta = self.regression(correlation)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/content/drive/My Drive/cp-vton/networks.py", line 124, in forward
x = x.view(x.size(0), -1)
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.