sergeywong / cp-vton Goto Github PK

I found the same images in both the train and test split.
For example,
/train/image/005749_0.jpg == /test/image/000010_0.jpg
/train/image/003574_0.jpg == /test/image/000001_0.jpg

复现实验后，觉得该方向技术和产生商业价值还有些距离。

github: https://github.com/cinastanbean/cp-vton

Sorry if this is a dumb question, but if I have a brand new dataset of different clothes, how do I generate the masks for them? I am seeing that the matlab file is very customized for viton data. Is there a different way in python which allows us to generate the masks of the clothes?

I found error

Traceback (most recent call last):
  File "train.py", line 191, in <module>
    main()
  File "train.py", line 182, in main
    train_tom(opt, train_loader, model, board)
  File "train.py", line 100, in train_tom
    criterionVGG = VGGLoss()
  File "/work/CP-VTON/cp-vton-master/networks.py", line 388, in __init__
    self.vgg = Vgg19()
  File "/work/CP-VTON/cp-vton-master/networks.py", line 356, in __init__
    vgg_pretrained_features = models.vgg19(pretrained=True).features
TypeError: vgg19() got an unexpected keyword argument 'pretrained'

on torchvision 0.1.6

I was executing convert_data.m. Its creating cloth-mask images with white background images. Kindly help with this issue.

Hi, I was trying to run the Try-on-module test, and I got the following error:

  File "test.py", line 163, in <module>
    main()
  File "test.py", line 156, in main
    test_tom(opt, train_loader, model, board)
  File "test.py", line 89, in test_tom
    model.cuda()
 ...
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 258, in <lambda>
    return self._apply(lambda t: t.cuda(device))
RuntimeError: Cannot initialize CUDA without ATen_cuda library.

I think this error accord because I am using a VM with no GPU support, and I installed the CPU version of Pytorch.

Is there a way to run your project without GPU support?

Are there any pre-trained models available for this ?

How can I change the tensorboard (in train.py/test.py) to show warped cloth overlayed on the person without making the warped cloth or the person transparent?

Hi @sergeywong,

As you can see below, there is minimal transformation and warping of the clothing. The cloth is not resized to fit the model either.

Other examples with similar issue:

What could be causing the issue here?

Hi @sergeywong,

I'm trying to train GMM on images of size 512, 384 - double of what has been used in cp-vton training. What changes would I need to make in networks.py?

I've tried a couple of things but they haven't shown promise:

• changed the number of layers in feature extraction from 3 to 4 (lines 408, 409 in networks.py)
• changed the number of inputs in the linear layer in FeatureRegression to 6486 (line 115 in networks.py)

Can you please let me know how can I train on images of size (512,384)?

Thanks.

When running the model on my own data and making the change as in #7 (permuting and unpermuting size dimension for im and im_h) I get the following error:

Traceback (most recent call last):
  File "test.py", line 163, in <module>
    main()
  File "test.py", line 151, in main
    test_gmm(opt, train_loader, model, board)
  File "test.py", line 69, in test_gmm
    grid, theta = model(agnostic, c)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/mnt/efs/experiments/lucas_gans/cp-vton/networks.py", line 416, in forward
    featureA = self.extractionA(inputA)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/mnt/efs/experiments/lucas_gans/cp-vton/networks.py", line 73, in forward
    return self.model(x)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
    input = module(input)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 320, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size [64, 22, 4, 4], expected input[4, 24, 256, 192] to have 22 channels, but got 24 channels instead

This was run after successfully running the model in the first instance. The model currently looks as follows:

Sequential(
  (0): Conv2d(22, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (1): ReLU(inplace)
  (2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (3): Conv2d(64, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (4): ReLU(inplace)
  (5): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (6): Conv2d(128, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (7): ReLU(inplace)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): Conv2d(256, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (10): ReLU(inplace)
  (11): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (13): ReLU(inplace)
  (14): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (16): ReLU(inplace)
)

The weight of size [64, 22, 4, 4] seems predefined by:

self.extractionA = FeatureExtraction(22, ngf=64, n_layers=3, norm_layer=nn.
self.extractionB = FeatureExtraction(3, ngf=64, n_layers=3, norm_layer=nn.BatchNorm2d)

The input expected input[4, 24, 256, 192] to have 22 channels comes from the data size, which for some reason is 2 channels larger?

in train.py
RuntimeError: The expanded size of the tensor (1) must match the existing size (3) at non-singleton dimension 0

in test.py:
RuntimeError: The expanded size of the tensor (1) must match the existing size (3) at non-singleton dimension 0

RuntimeError: The size of tensor a (270) must match the size of tensor b (3) at non-singleton dimension 2

Hi,

I am getting a tensor dimension error while running test.py as below:
python test.py --name gmm_traintest_new --stage GMM --workers 4 --datamode test --data_list test_pairs.txt --checkpoint checkpoints/gmm_train_new/gmm_final.pth

The error seems to arise from cp_dataset.py module in line 91. Somehow the size/shape of the image read is (3, 360, 270) where the image is of 360x270 pix. This throws the following error:

I've fixed this issue by using permute on the object im in cp_dataset.py after the image has been read and transformed. Although it fixes the error shown in the screengrab above, I still have some issues with the implementation. Seems like the input size of image MUST be 256x192?

https://github.com/maizer2/cp-vton/blob/master/inference.py

In paper, the author contends that

Body shape: a 1-channel feature map of a blurred binary mask that roughly
covering different parts of human body.

the body shape doesn't include the head, but clearly your codes are not for it.

parse_shape = (parse_array > 0).astype(np.float32)

Hey, I get these results after I've added Total Variation loss (https://discuss.pytorch.org/t/yet-another-post-on-custom-loss-functions/14552).

What can cause this problem?

Solution:

Change x = x.view(x.size(0), -1)

to

x = x.reshape(x.size(0), -1)

The code U = torch.mul(dist_squared,torch.log(dist_squared)) means r^2logr^2, but the tps RBF is r^2logr, is this an error?

When I train the 2 stages, I find that the loss changes in a small oscillation from beginning to end even if it falls in a general. I think it is unrelated to the fix of learning rate for I just use the origin learing rate in the code and the loss keep falling in the overall trend. Why is the loss always jumping?

0

Hello Authors,
I want to use your code in my project and I haven't found a license in the repo. I would like to know the license. Is it like VITON which is Apache V2 licensed?
Thanks in Advance

Hi @sergeywong,

While AffineGridGen is defined in the networks module, it is not used anywhere for affine warping. How can we incorporate this into TpsGridGen (TPS)?

Where would we need to make the changes in init and forward?

I'm trying to preprocess the data. I used Look-into-Person for human segmentation. As the model requires the images in "image-parse" to be of 1-channel, I saved the segmented images as grayscale images i.e. shape is (256, 192). But the problem is pixel values in my segmented images are different from the one that you get from this repo (data_download.py). It seems the author's preprocessed images in "image-parse" contains pixels with values between 0-19 only, while mine is in range of 0-255. There is some kind of mapping involved. How to map the pixel values in 0-19 range? Can anyone help me with this?

can you please explain these lines.
Before the trainning, you should generate warp-mask & warp-cloth, using the test process of GMM with --datamode train. Then move these files or make soft links under the directory data/train. An example training command is

When convert_data.m file is used to prepare data from VITON, the Image parse generated by it using mat file from segment folder is tilted by 90°.

Hi, there is a way to restart the training from a specific checkpoint?
I'm trying this command but it starts always from step 0
python train.py
--name gmm_train_new
--stage GMM
--workers 0
--save_count 5000
--checkpoint "checkpoints/gmm_train_new/step_065000.pth"

Because of dataset license, VITON Dataset is not available anymore.
There is no pre-trained model in CP-VTON... Isn't there any way to run your code?
Thx...

How to generate segment maps? I have no idea about this Question，could you help me?

Hi, I use your datasets to test your model. But I get this result.I don't know why and how to solve it？
Thank you.

I can't seem to find them anywhere but I can see that people have mentioned them before and have used them too. Could you please point me to the right direction? Thank you!

Hi,

I am trying to run CP-VTON, but I couldn't find the download link for the pretrained models. Could you please provide the download links or guide me on how to obtain the pretrained models?

Thank you!

Xxx

Thx for your job and sharing. It's really helpful to my research. Thanks again.
I have trained and tested your model successfully, and have a litte question about loss.
When I train in satge II, the mask loss almost have not change from the beigining to the end.
So, I check the calculation abouot mask loss, and find that the "loss_mask = criterionMask(m_composite, cm)" in line 133 of train.py is differernt from the equation (4) in paper.
Is there any detail about this I can konw?
Thank you a lot.
@sergeywong

hello! The dataset I am using in cp-vton is viton_resize, but the viton data is not available because "The dataset is no longer publicly available due to copyright issues. For thoese who have already downloaded the dataset, please note that using or distributing it is illegal!" and I understand that it is not available for the same reason. I was wondering if it is okay to publish a paper using the viton_resize dataset?

Hii @sergeywong ,
when I tested this model for custom images it gives an error, I have also implemented all the possible solution that mentioned in Issues like :

1. I used im_h.permute in cp_dataset.py
2. Installed different torchvision version(0.2.1).
3. converted rgb to grey image for tensor resize.
4. used different normalize in cp_dataset.py
   self.transform = transform = transforms.Compose([transforms.ToTensor(),transforms.Normalize((0.5,), (0.5,))])
5. manually resize in (1,256,192) but still facing tensor size issue.

Here are all possible outcomes that I applied when implementing this model. Please help me to solve this problem.

Hi,

I have downloaded all the data with data_download.py, but...

When I try to train CP-VTON GMM stage, I get this error immediately:

RuntimeError: output with shape [1, 256, 192] doesn't match the broadcast shape [3, 256, 192]

See the full log below. Any help much appreciated! (I get the same error if I run cp_dataset.py alone.)

Thanks,

Mike

C:\Users\Mike\AppData\Local\Programs\Python\Python36\python.exe M:/VITON/cp-vton-master/train.py --name gmm_train_new --stage GMM --save_count 5000 --shuffle
Namespace(batch_size=4, checkpoint='', checkpoint_dir='checkpoints', data_list='train_pairs.txt', datamode='train', dataroot='data', decay_step=100000, display_count=20, fine_height=256, fine_width=192, gpu_ids='', grid_size=5, keep_step=100000, lr=0.0001, name='gmm_train_new', radius=5, save_count=5000, shuffle=True, stage='GMM', tensorboard_dir='tensorboard', workers=1)
Start to train stage: GMM, named: gmm_train_new!
initialization method [normal]
initialization method [normal]
Traceback (most recent call last):
  File "M:/VITON/cp-vton-master/train.py", line 191, in <module>
    main()
  File "M:/VITON/cp-vton-master/train.py", line 176, in main
    train_gmm(opt, train_loader, model, board)
  File "M:/VITON/cp-vton-master/train.py", line 58, in train_gmm
    inputs = train_loader.next_batch()
  File "M:\VITON\cp-vton-master\cp_dataset.py", line 166, in next_batch
    batch = self.data_iter.__next__()
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __next__
    return self._process_data(data)
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\dataloader.py", line 846, in _process_data
    data.reraise()
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\_utils.py", line 369, in reraise
    raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\_utils\worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "M:\VITON\cp-vton-master\cp_dataset.py", line 86, in __getitem__
    shape = self.transform(parse_shape) # [-1,1]
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torchvision\transforms\transforms.py", line 61, in __call__
    img = t(img)
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torchvision\transforms\transforms.py", line 166, in __call__
    return F.normalize(tensor, self.mean, self.std, self.inplace)
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python36\lib\site-packages\torchvision\transforms\functional.py", line 217, in normalize
    tensor.sub_(mean[:, None, None]).div_(std[:, None, None])
RuntimeError: output with shape [1, 256, 192] doesn't match the broadcast shape [3, 256, 192]


Process finished with exit code 1

Dear Mr @sergeywong ,

As i far as i understand, your network reimplements GMM from this paper: https://arxiv.org/pdf/1703.05593.pdf . In the original GMM's paper the architecture was:
Stage 1:
ImageA -> featureExtraction -> fA
ImageB -> featureExtraction -> fB

then matching(fA, fB) -> AffineRegresstion(CAB) -> ThetaAff , then warp to imageA to become ImageWarpA.

Stage 2:
Repeat over featureExtraction ImageWarpA , then matching(fImageWarpA, fB) -> TBS Regression -> ThetaTPS.
Which matching is correlation layers plus L2-Norm.

However, in your network you implemented like this (please see the attached)

Can you please explain why it that, or just point out where i am missing you in the paper?

Thank you for your time.
Cheers!

Namespace(batch_size=4, checkpoint='', checkpoint_dir='checkpoints', data_list='train_pairs.txt', datamode='train', dataroot='data', decay_step=100000, display_count=20, fine_height=256, fine_width=192, gpu_ids='', grid_size=5, keep_step=100000, lr=0.0001, name='gmm_train_new', radius=5, save_count=50, shuffle=True, stage='GMM', tensorboard_dir='tensorboard', workers=4)
Start to train stage: GMM, named: gmm_train_new!
initialization method [normal]
initialization method [normal]
Traceback (most recent call last):
File "train.py", line 191, in
main()
File "train.py", line 176, in main
train_gmm(opt, train_loader, model, board)
File "train.py", line 70, in train_gmm
grid, theta = model(agnostic, c)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/content/drive/My Drive/cp-vton/networks.py", line 422, in forward
theta = self.regression(correlation)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/content/drive/My Drive/cp-vton/networks.py", line 124, in forward
x = x.view(x.size(0), -1)
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

Running on the same dataset, i am getting this error. Any insights would be great. Thank you.