sangwoomo / instagan Goto Github PK

i try to do test train.py by jeans2skirt_ccp dataset.
but, mask image can't change tensor or, normalize
this error
Traceback (most recent call last):
File "train.py", line 24, in
for i, data in enumerate(dataset):
File "/home/s-yasui/src/instagan/data/init.py", line 72, in iter
for i, data in enumerate(self.dataloader):
File "/home/s-yasui/anaconda3/envs/ai/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 819, in next
return self._process_data(data)
File "/home/s-yasui/anaconda3/envs/ai/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data
data.reraise()
File "/home/s-yasui/anaconda3/envs/ai/lib/python3.7/site-packages/torch/_utils.py", line 385, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/s-yasui/anaconda3/envs/ai/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/s-yasui/anaconda3/envs/ai/lib/python3.7/site-packages/torch/utils/data/utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/s-yasui/anaconda3/envs/ai/lib/python3.7/site-packages/torch/utils/data/utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/s-yasui/src/instagan/data/unaligned_seg_dataset.py", line 71, in getitem
A_segs = self.read_segs(A_seg_path, seed)
File "/home/s-yasui/src/instagan/data/unaligned_seg_dataset.py", line 42, in read_segs
seg = self.fixed_transform(seg, seed)
File "/home/s-yasui/src/instagan/data/unaligned_seg_dataset.py", line 34, in fixed_transform
return self.transform(image)
File "/home/s-yasui/anaconda3/envs/ai/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 61, in call
img = t(img)
File "/home/s-yasui/anaconda3/envs/ai/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 166, in call
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "/home/s-yasui/anaconda3/envs/ai/lib/python3.7/site-packages/torchvision/transforms/functional.py", line 217, in normalize
tensor.sub(mean[:, None, None]).div(std[:, None, None])
RuntimeError: output with shape [1, 300, 200] doesn't match the broadcast shape [3, 300, 200]

please help , thank you

i have some problems，When I started training the model。
RuntimeError: output with shape [1, 300, 200] doesn't match the broadcast shape [3, 300, 200]
What configuration information should I modify??

Hi,
I'm getting this runtime error during training on line 71 of unaligned_seg_dataset.py

RuntimeError: output with shape [1, 200, 200] doesn't match the broadcast shape [3, 200, 200]

It looks like there is a transform being applied to the segmentation mask that doesn't match the correct dimensions. Segmentation masks are 8-bit 1 channel gray image, while training images are 24-bit 3 channel RGB.

Nothing

If we set opt.batch_size > 1, feat = torch.cat([enc_img, enc_segs_sum], dim=1) raise RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1

hello,when i run the python ./datasets/generate_ccp_dataset.py, i got some errors
trainA: 99%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 989/1004 [00:31<00:00, 19.93it/s]Traceback (most recent call last): File "./datasets/generate_ccp_dataset.py", line 92, in <module> main() File "./datasets/generate_ccp_dataset.py", line 12, in main generate_ccp_dataset(args) File "./datasets/generate_ccp_dataset.py", line 36, in generate_ccp_dataset generate_ccp_dataset_train(args, 'A', args.cat1) File "./datasets/generate_ccp_dataset.py", line 52, in generate_ccp_dataset_train ann = sio.loadmat(str(args.pix_ann_root / '{}.mat'.format(ann_id)))['groundtruth'] File "/usr/local/lib/python3.6/dist-packages/scipy/io/matlab/mio.py", line 208, in loadmat matfile_dict = MR.get_variables(variable_names) File "/usr/local/lib/python3.6/dist-packages/scipy/io/matlab/mio5.py", line 272, in get_variables hdr, next_position = self.read_var_header() File "/usr/local/lib/python3.6/dist-packages/scipy/io/matlab/mio5.py", line 217, in read_var_header mdtype, byte_count = self._file_reader.read_full_tag() File "mio5_utils.pyx", line 548, in scipy.io.matlab.mio5_utils.VarReader5.read_full_tag File "mio5_utils.pyx", line 556, in scipy.io.matlab.mio5_utils.VarReader5.cread_full_tag File "streams.pyx", line 86, in scipy.io.matlab.streams.GenericStream.read_into OSError: could not read bytes trainA: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍| 999/1004 [00:31<00:00, 32.07it/s]
is there anything mistake about the datasets? thanku!

GPU memory for High resolution:
In the paper you present qualitative results in high resolution images, I wonder how can I size the GPU needed for 1024x768 images? Could you please share the one you used for higher resolution in the paper ?

i did an experiments on hair style transfer, stright hair to curl hair. but the fake seg B seems to be all zeros, so it don't work to transfer hair style .
could you please tell me instagan is able to work on the experiment,
and maybe what is wrong about the experiment

thanks a lot !!

Hi，I find there have a error train image in jeans2skirt_ccp/trainB/0480.png, and its seg lable is the skin.

Can we perform multiple objects transformation in single training?
For example Upperwear & Lowerwear.
If yes, what kind of dataset is expected for the training? Is some modification required in the code?

hello , i have have a problem when google colab disconnect . i want to continue to train the last model
how to put argument for continue train

Hi, I had read the issue #7 and issue #8 and I want to transfer a single photo from google, since there is no mask for that photo, I need to train the pix2pix model with the MHP dataset first. Then I should use the pix2pix model to produce a mask for that photo. Finally, I put both that photo and its mask into the pants2skirt_mhp model. Am I right?

Hi, I want to train a pix2pix model, and run: python train.py --dataroot ./datasets/photos2pants_ccp --name photos2pants_ccp_pix2pix --model pix2pix

but the result:
----------------- Options ---------------
batch_size: 1
beta1: 0.5
checkpoints_dir: ./checkpoints
continue_train: False
dataroot: ./datasets/photos2pants_ccp [default: None]
dataset_mode: aligned
direction: AtoB
display_env: main
display_freq: 400
display_id: 1
display_ncols: 6
display_port: 8097
display_server: http://localhost
display_winsize: 256
epoch: latest
epoch_count: 1
fineSizeH: 200
fineSizeW: 200
gpu_ids: 0
init_gain: 0.02
init_type: normal
input_nc: 3
isTrain: True [default: None]
lambda_L1: 100.0
loadSizeH: 220
loadSizeW: 220
load_iter: 0 [default: 0]
lr: 0.0002
lr_decay_iters: 50
lr_policy: lambda
max_dataset_size: inf
model: pix2pix [default: insta_gan]
n_layers_D: 3
name: photos2pants_ccp_pix2pix [default: experiment_name]
ndf: 64
netD: set
netG: unet_256
ngf: 64
niter: 100
niter_decay: 100
no_dropout: False
no_flip: False
no_html: False
no_lsgan: True
norm: batch
num_threads: 4
output_nc: 3
phase: train
pool_size: 0
print_freq: 100
resize_or_crop: resize_and_crop
save_by_iter: False
save_epoch_freq: 5
save_latest_freq: 5000
serial_batches: False
suffix:
update_html_freq: 1000
verbose: False
----------------- End -------------------
dataset [AlignedDataset] was created
#training images = 604
initialize network with normal
initialize network with normal
model [Pix2PixModel] was created
---------- Networks initialized -------------
[Network G] Total number of parameters : 54.414 M
[Network D] Total number of parameters : 9.760 M

WARNING:root:Setting up a new session...
create web directory ./checkpoints\photos2pants_ccp_pix2pix\web...
Traceback (most recent call last):
File "train.py", line 24, in
for i, data in enumerate(dataset):
File "D:\MINE\STUDY in SYSU\SmartLLV\Papers\instagan-master\data_init_.py", line 72, in iter
for i, data in enumerate(self.dataloader):
File "D:\devcool\Anaconda2\envs\python36\lib\site-packages\torch\utils\data\dataloader.py", line 286, in next
return self._process_next_batch(batch)
File "D:\devcool\Anaconda2\envs\python36\lib\site-packages\torch\utils\data\dataloader.py", line 307, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
AttributeError: Traceback (most recent call last):
File "D:\devcool\Anaconda2\envs\python36\lib\site-packages\torch\utils\data\dataloader.py", line 57, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "D:\devcool\Anaconda2\envs\python36\lib\site-packages\torch\utils\data\dataloader.py", line 57, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "D:\MINE\STUDY in SYSU\SmartLLV\Papers\instagan-master\data\aligned_dataset.py", line 26, in getitem
assert(self.opt.loadSize >= self.opt.fineSize)
AttributeError: 'Namespace' object has no attribute 'loadSize'

/generate_coco_dataset.py Motorcycle -> Bicycle

loading annotations into memory...
Done (t=11.53s)
creating index...
index created!
trainA: 0%| | 44/118287 [00:03<2:00:02, 16.42it/s]Traceback (most recent call last):
File "/home/caocao/instagan-master/datasets/generate_coco_dataset.py", line 96, in
main()
File "/home/caocao/instagan-master/datasets/generate_coco_dataset.py", line 15, in main
generate_coco_dataset(args)
File "/home/caocao/instagan-master/datasets/generate_coco_dataset.py", line 35, in generate_coco_dataset
generate_coco_dataset_sub(args, 'train', 'A', args.cat1)
File "/home/caocao/instagan-master/datasets/generate_coco_dataset.py", line 77, in generate_coco_dataset_sub
seg_union[seg_union > 1] = 1 # remove duplicated parts
TypeError: 'int' object does not support item assignment
trainA: 0%| | 46/118287 [00:06<4:25:51, 7.41it/s]

I have a question when I read your paper. In my opinion, the far right of the graph Figure 2 maybe 'Y' or not "Y", just like the way in Cyclegan. But it is 'X' or not 'X' here. Can you help me, please?

Hello,

When I start training the model on the existing dataset downloaded as per the instructions. I get an error mismatch output shape.

RuntimeError: output with shape [1, 240, 160] doesn't match the broadcast shape [3, 240, 160]

I think this error has to do with the output layer. Please let me know which file should I change to make this working.
Thank you.

Hi! Is it possible to run Instagan in Google Colab? I've got a problem of running visdom server in colab. Seems that it's not allowed by Google. What is a possible solution? Much thanks in advance!

Is it possible to have varying epoch ratio training schedule for generator and discriminator with the current version of the code?
Like if I want to have a training schedule where the generator is trained for 1 epoch and then the discriminator is trained for 5 epochs and so on.
Can I have a 5:1 epoch ratio training schedule for discriminator and generator?

Hi. I've seen your research very interesting.
I am trying to train a model on github that I am interested in.

My gpu enviroment is:

After doing dataset generatation as described in git

sudo CUDA_VISIBLE_DEVICES=0 python3 train.py --dataroot ./datasets/jeans2skirt_ccp --model insta_gan --name jeans2skirt_ccp_instagan --loadSizeH 330 --loadSizeW 220 --fineSizeH 300 --fineSizeW 200 --niter 400 --niter_decay 200

When I execute the above, I get the following error.

I know it is a Cuda version issue, so if you run it in various versions, it will appear as well.
Thank you for your understanding. What I wrote in Korean

Getting the above when trying to run the instaGan model. I found a similar closed issue. Where you asked to print the seg_paths variable and check for segmentation pairs. I have segmentation images for all input images both in train_A and train_B, but it still not working.

As the code assumes that every image has at least one corresponding mask, how can you get the Results on YouTube-searched videos (pants->skirt) and the Results on Google-searched images (pants->skirt)?

If I want to train the InstaGAN model with a larger batch_size, you have provided batch_size option which can be modified. But what is actually happening is, it only works for batch_size = number of gpu's and gives a size of tensor mismatch error otherwise.

Is your code only suited for this or something else needs to be modified?

fake_A_seg_list = self.fake_A_seg_list + [self.fake_A_seg_sng]  # not detach
for i in range(self.ins_iter - idx - 1):
	fake_A_seg_list.append(empty)

self.fake_A_seg_mul = torch.cat(fake_A_seg_list, dim=1)
self.fake_A_mul = torch.cat([self.fake_A_img_sng, self.fake_A_seg_mul], dim=1)

Hi, your code is perfect. But I have some questions.
According your code, I think that self.fake_A_mul is to be GAN_loss in the global in your paper, is this right?
More than, what's the point of "fake_A_seg_list.append(empty)", because in "NLayerSetDiscriminator", empty will be abandoned, so it confuses me, please help me to understand the code?
And, in your paper, you use pix2pix model to predict the segmentation, how can I reproduce this or when you release this? If it can be done in just input the image, mask can be predict by the model, transfer can also be done by the model, it's unbelievable!
Thank you, sir!

Hello, I'm following the instruction but faced the error while training.
I ran the script on Macbook Pro.
Do you have any idea to fix this issue?

$ python train.py --dataroot ./datasets/jeans2skirt_ccp --model insta_gan --name jeans2skirt_ccp_instagan --loadSizeH 330 --loadSizeW 220 --fineSizeH 300 --fineSizeW 200 --niter 400 --niter_decay 200 --gpu_ids -1
----------------- Options ---------------
               batch_size: 1                             
                    beta1: 0.5                           
          checkpoints_dir: ./checkpoints                 
           continue_train: False                         
                 dataroot: ./datasets/jeans2skirt_ccp    	[default: None]
             dataset_mode: unaligned_seg                 
                direction: AtoB                          
              display_env: main                          
             display_freq: 400                           
               display_id: 1                             
            display_ncols: 6                             
             display_port: 8097                          
           display_server: http://localhost              
          display_winsize: 256                           
                    epoch: latest                        
              epoch_count: 1                             
                fineSizeH: 300                           	[default: 200]
                fineSizeW: 200                           
                  gpu_ids: -1                            	[default: 0]
                init_gain: 0.02                          
                init_type: normal                        
                 input_nc: 3                             
                  ins_max: 4                             
                  ins_per: 2                             
                  isTrain: True                          	[default: None]
                 lambda_A: 10.0                          
                 lambda_B: 10.0                          
               lambda_ctx: 1.0                           
               lambda_idt: 1.0                           
                loadSizeH: 330                           	[default: 220]
                loadSizeW: 220                           
                load_iter: 0                             	[default: 0]
                       lr: 0.0002                        
           lr_decay_iters: 50                            
                lr_policy: lambda                        
         max_dataset_size: inf                           
                    model: insta_gan                     
               n_layers_D: 3                             
                     name: jeans2skirt_ccp_instagan      	[default: experiment_name]
                      ndf: 64                            
                     netD: set                           
                     netG: set                           
                      ngf: 64                            
                    niter: 400                           	[default: 100]
              niter_decay: 200                           	[default: 100]
               no_dropout: True                          
                  no_flip: False                         
                  no_html: False                         
                 no_lsgan: False                         
                     norm: instance                      
              num_threads: 4                             
                output_nc: 3                             
                    phase: train                         
                pool_size: 50                            
               print_freq: 100                           
           resize_or_crop: resize_and_crop               
             save_by_iter: False                         
          save_epoch_freq: 5                             
         save_latest_freq: 5000                          
           serial_batches: False                         
                set_order: decreasing                    
                   suffix:                               
         update_html_freq: 1000                          
                  verbose: False                         
----------------- End -------------------
dataset [UnalignedSegDataset] was created
#training images = 167
initialize network with normal
initialize network with normal
initialize network with normal
initialize network with normal
model [InstaGANModel] was created
---------- Networks initialized -------------
[Network G_A] Total number of parameters : 26.815 M
[Network G_B] Total number of parameters : 26.815 M
[Network D_A] Total number of parameters : 9.754 M
[Network D_B] Total number of parameters : 9.754 M
-----------------------------------------------
Setting up a new session...
create web directory ./checkpoints/jeans2skirt_ccp_instagan/web...
Traceback (most recent call last):
  File "train.py", line 24, in <module>
    for i, data in enumerate(dataset):
  File "/Users/tomo/Documents/GitHub/instagan/data/__init__.py", line 72, in __iter__
    for i, data in enumerate(self.dataloader):
  File "/Users/tomo/.pyenv/versions/3.7.4/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 819, in __next__
    return self._process_data(data)
  File "/Users/tomo/.pyenv/versions/3.7.4/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data
    data.reraise()
  File "/Users/tomo/.pyenv/versions/3.7.4/lib/python3.7/site-packages/torch/_utils.py", line 385, in reraise
    raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/Users/tomo/.pyenv/versions/3.7.4/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/Users/tomo/.pyenv/versions/3.7.4/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/Users/tomo/.pyenv/versions/3.7.4/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/Users/tomo/Documents/GitHub/instagan/data/unaligned_seg_dataset.py", line 71, in __getitem__
    A_segs = self.read_segs(A_seg_path, seed)
  File "/Users/tomo/Documents/GitHub/instagan/data/unaligned_seg_dataset.py", line 42, in read_segs
    seg = self.fixed_transform(seg, seed)
  File "/Users/tomo/Documents/GitHub/instagan/data/unaligned_seg_dataset.py", line 34, in fixed_transform
    return self.transform(image)
  File "/Users/tomo/.pyenv/versions/3.7.4/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 61, in __call__
    img = t(img)
  File "/Users/tomo/.pyenv/versions/3.7.4/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 166, in __call__
    return F.normalize(tensor, self.mean, self.std, self.inplace)
  File "/Users/tomo/.pyenv/versions/3.7.4/lib/python3.7/site-packages/torchvision/transforms/functional.py", line 217, in normalize
    tensor.sub_(mean[:, None, None]).div_(std[:, None, None])
RuntimeError: output with shape [1, 300, 200] doesn't match the broadcast shape [3, 300, 200]

I tried the other one, pants2skirt_mhp_instagan but almost same result.
RuntimeError: output with shape [1, 240, 160] doesn't match the broadcast shape [3, 240, 160]
I'm sorry if this is not the right place to ask.
Thanks in advance.

Dear authors,

First off, thank you very much for sharing your code!

In section B of your appendix you state that you have trained with a batch size of 4 on 4 GPU in parallel. Similarly, I tried to train you model with a batch size of 4 and also used flags --netD=set and --netG=set in order to benefit from your proposed sequential mini-batch method. However, this results in an error within ResnetSetGenerator's forward pass (*), which I did not encounter when using a batch size of 1. This makes me wonder: is your flag batch_size=4 (for multi-GPU training) compatible with the proposed sequential mini-batch training (paper section 2.3)? If not, then according to the information in section B of your appendix this means that your reported results are NOT obtained via the sequential mini-batch training (paper section 2.3)? If yes, do you recognize the issue (*) that I'm running into? Thank you very much for any help!

(*) more precisely, the error is as follows:

>>> feat = torch.cat([enc_img, enc_segs_sum], dim=1)
Traceback (most recent call last):
  File "/snap/pycharm-community/163/helpers/pydev/_pydevd_bundle/pydevd_exec2.py", line 3, in Exec
    exec(exp, global_vars, local_vars)
  File "<input>", line 1, in <module>
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 1 and 4 in dimension 0 at /pytorch/aten/src/THC/generic/THCTensorMath.cu:71

The above error occurs in forward(self, inp) of ResnetSetGenerator and, while debugging at the code snippet

        enc_img = self.encoder_img(img)  
        enc_segs = list() 
        for i in range(segs.size(1)): 
            if mean[i] > 0: 
                seg = segs[:, i, :, :].unsqueeze(1)
                enc_segs.append(self.encoder_seg(seg))

I obtained the following information

>>> segs.shape # batch size is 4
torch.Size([4, 2, 256, 256])
>>> len(enc_segs) # i.e. we pass two instance masks
2
>>> enc_segs[0].shape
torch.Size([4, 256, 64, 64])
>>> enc_segs[1].shape
torch.Size([4, 256, 64, 64])
>>> torch.cat(enc_segs).shape # now the batch size is 4*2, seems like batch size and number of passed instances got mixed now?
torch.Size([8, 256, 64, 64])

Then subsequently calling

enc_segs_sum = torch.sum(enc_segs, dim=0, keepdim=True)

marginalizes as well over the batch entries and we obtain a dimension mismatch when calling

feat = torch.cat([enc_img, enc_segs_sum], dim=1)

because the image batch size (i.e. 4) and the segmentation batch size (i.e. now 1) at axis 0 no longer match. Can you spot if I'm doing something wrong here or is batch size > 1 indeed incompatible with the GPU-memory efficient training method proposed in section 2.3?

Edit 1: okay so eventually this is related to this issue, as I am currently coding on a single GPU. Let me try replicating my problem on a different machine with multiple GPU.

Edit 2: This solves the issue. So the problem resulted indeed from me trying to train with batch size > 1 on a single GPU, the symptoms were just a bit confusing.

Hello，thanks for contributing such a good project.
I noticed parameter of ‘gpu_ids’ could be set multi GPU (ie 0,1,2). But argument of ‘batch_size’ default 1, multi GPU does not work in this case.
So, should I set batch_size =4 if set ‘gpu_ids’ is '0,1,2,3'？

Excuse me! Thanks for your excellent work. I want to download the pre-train model in your paper. Can you give it to me?

Hi, thanks for the project! I'm trying to replicate results, but have a problem following your instructions.

0. I've cloned the repo.
1. I've downloaded jeans2skirt dataset.
2. Trying to start learning

python3 train.py --dataroot ./datasets/jeans2skirt_ccp --model insta_gan --name jeans2skirt_ccp_instagan --loadSizeH 330 --loadSizeW 220 --fineSizeH 300 --fineSizeW 200 --niter 400 --niter_decay 200

I get the following error:

Traceback (most recent call last):
  File "train.py", line 24, in <module>
    for i, data in enumerate(dataset):
  File "/home/ubuntu/pr/data/__init__.py", line 72, in __iter__
    for i, data in enumerate(self.dataloader):
  File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 637, in __next__
    return self._process_next_batch(batch)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 658, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
RuntimeError: Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 138, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 138, in <listcomp>
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/home/ubuntu/pr/data/unaligned_seg_dataset.py", line 71, in __getitem__
    A_segs = self.read_segs(A_seg_path, seed)
  File "/home/ubuntu/pr/data/unaligned_seg_dataset.py", line 42, in read_segs
    seg = self.fixed_transform(seg, seed)
  File "/home/ubuntu/pr/data/unaligned_seg_dataset.py", line 34, in fixed_transform
    return self.transform(image)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 60, in __call__
    img = t(img)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 163, in __call__
    return F.normalize(tensor, self.mean, self.std, self.inplace)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/torchvision/transforms/functional.py", line 208, in normalize
    tensor.sub_(mean[:, None, None]).div_(std[:, None, None])
RuntimeError: output with shape [1, 300, 200] doesn't match the broadcast shape [3, 300, 200]```

I suppose it is because mask images have only 1 channel. Can you help?

Hello, I really appreciate the awesome instaGAN you guys put forward. However, when I read your paper, I found something confusing about the losses: the L_cyc and L_idt cannot prevent the G to make shortcuts like learning a identity mapping(x==y', y==x'), so you use L_ctx to prevent this shortcut. But if G still learn the identity mapping, it does not hurt L_ctx and it will still be zero.

For some unknown reason I can't understand, the processed images look like this, which is the real_A image of the first epoch

Any debugging clue?

I am just training my dataset with this instagan model. my training dataset has 14221 images and at the first trial, it has error as show in figure when iter value is 4200. Second trial, now it stopped when iter value is 8900. It has increased a lot than the first trial. I am just adding my dataset folder under the directory /instagan/datasets.

Hi，I want to train the pix2pix model to extract pants or skirts from photos of CCP dataset，and I get the matrix of pixel-level annotation of every photo，the value of matrix are the label number（0-58，as there are 59 kind of annotations， and I think there are 2 ways to train：
（1）set every pixel values of pants or skirts to 255，and the other pixel values to 0，then I can get the mask of pants or skirts;
（2）use the values of matrix to train，so the pix2pix model can learn to label the different parts of the photo. Finally use the method(1) to get the masks for insta-gan to train;

If use method(1), I think the trainning result can be very bad;
If use method(2), the original value of annotation matrix are numbers from 0 to 58, 23 means jacket and 24 means pants,etc. As the small difference of the value lead to big difference of meaning， have you done some preprocessing to the matrix value such as multiply 5？
Or maybe you have better idea to train the pix2pix model？

First, thank you for good quailty paper.
I read this paper with admire to you

But I have a question.
Where can I find functions that f_GX, f_GA (Figure 2, in red box)

Thanks.

I run the follow command: python ./datasets/generate_ccp_dataset.py --save_root ./datasets/jeans2skirt_ccp --cat1 jeans --cat2 skirt
but there is noting in ./datasets/jeans2skirt/valA_seg and valB_segs.

E:\Users\Raytine\Anaconda3\python.exe F:/humanpose/instagan-master/datasets/generate_mhp_dataset.py
trainA: 0%| | 0/4000 [00:00<?, ?it/s]Traceback (most recent call last):
File "F:/humanpose/instagan-master/datasets/generate_mhp_dataset.py", line 122, in
main()
File "F:/humanpose/instagan-master/datasets/generate_mhp_dataset.py", line 17, in main
generate_ccp_dataset(args)
File "F:/humanpose/instagan-master/datasets/generate_mhp_dataset.py", line 40, in generate_ccp_dataset
generate_mhp_dataset(args, 'train', 'A', get_cat_id(args.cat1))
File "F:/humanpose/instagan-master/datasets/generate_mhp_dataset.py", line 63, in generate_mhp_dataset
if not args.no_skip_horizontal:
AttributeError: 'Namespace' object has no attribute 'no_skip_horizontal'

Hello, can i change the batch_size? eg: batch_size=8,16.... but encounter a problem----- RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 1 and 8 in dimension 0 at c:\a\w\1\s\windows\pytorch\aten\src\thc\generic/THCTensorMath.cu:83
How to solve this problem?
Looking forward to your reply,Thank you !!!

Hi,is this possible to generate fake human images with full body with the human mask ?

@sangwoomo thank you for this nice repository!

When I was running the test-script (python test.py --dataroot ./datasets/shp2gir_coco --model insta_gan --name shp2gir_coco_instagan --loadSizeH 200 --loadSizeW 200 --fineSizeH 200 --fineSizeW 200 --ins_per 2 --ins_max 20 --phase sample --epoch 200), I was facing the broadcasting error, as mentioned before:
https://gitmemory.com/issue/sangwoomo/instagan/19/483668690

RuntimeError: output with shape [1, 200, 200] doesn't match the broadcast shape [3, 200, 200]

@mrvlasyuk Do you know how I can solve the problem? As it worked at some point according to your statement.

Hi, Any help would be appreciated. This error also is happening when i run on the jeans2skirt_ccp dataset.

WARNING:root:Setting up a new session...
create web directory ./checkpoints/deep_nude_results/web...
Traceback (most recent call last):
File "train.py", line 24, in
for i, data in enumerate(dataset):
File "/Users/WillMReynolds/Desktop/GAN_STUDY/instaGAN/data/init.py", line 72, in iter
for i, data in enumerate(self.dataloader):
File "/Users/WillMReynolds/anaconda3/envs/tfdeeplearning/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 582, in next
return self._process_next_batch(batch)
File "/Users/WillMReynolds/anaconda3/envs/tfdeeplearning/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 608, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
RuntimeError: Traceback (most recent call last):
File "/Users/WillMReynolds/anaconda3/envs/tfdeeplearning/lib/python3.5/site-packages/torch/utils/data/_utils/worker.py", line 99, in worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/Users/WillMReynolds/anaconda3/envs/tfdeeplearning/lib/python3.5/site-packages/torch/utils/data/utils/worker.py", line 99, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "/Users/WillMReynolds/Desktop/GAN_STUDY/instaGAN/data/unaligned_seg_dataset.py", line 71, in getitem
A_segs = self.read_segs(A_seg_path, seed)
File "/Users/WillMReynolds/Desktop/GAN_STUDY/instaGAN/data/unaligned_seg_dataset.py", line 42, in read_segs
seg = self.fixed_transform(seg, seed)
File "/Users/WillMReynolds/Desktop/GAN_STUDY/instaGAN/data/unaligned_seg_dataset.py", line 34, in fixed_transform
return self.transform(image)
File "/Users/WillMReynolds/anaconda3/envs/tfdeeplearning/lib/python3.5/site-packages/torchvision/transforms/transforms.py", line 61, in call
img = t(img)
File "/Users/WillMReynolds/anaconda3/envs/tfdeeplearning/lib/python3.5/site-packages/torchvision/transforms/transforms.py", line 164, in call
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "/Users/WillMReynolds/anaconda3/envs/tfdeeplearning/lib/python3.5/site-packages/torchvision/transforms/functional.py", line 208, in normalize
tensor.sub(mean[:, None, None]).div(std[:, None, None])
RuntimeError: output with shape [1, 240, 160] doesn't match the broadcast shape [3, 240, 160]

I am trying to train the InstaGAN on a smaller dataset of around 40 images from both domains. But I am getting the following error :
File "/network/home/bansalsa/instagan/data/unaligned_seg_dataset.py", line 49, in read_segs
segs.append(-torch.ones(segs[0].size()))
IndexError: list index out of range

I have read the already available issues and double-checked that there is a mask for every image in the trainA and trainB folders but still I'm getting the same error. Also, the images are readable via PIL, I have checked that too.

This is the screenshot to the output that I'm getting :

What could be a reason for getting this error?

I trained the pix2pix model to predict the pants mask and the skirts mask, and the result of pix2pxi model were satisfied, then I trained the instagan model, but the result seems bad. So how did you train your instagan model? Have you combined the skirts and skin mask together, and train the instagan to learn to transfer pants to skirts and the skin of legs?

thans fou you good job
I use a 1080ti gpu,and it out of memory,how can I train with my gpu,is there any idea? thankyou very much

Hi！I use your pre-trained model and follow the README.md，here follows the output:
---------- Networks initialized -------------
[Network G_A] Total number of parameters : 26.815 M
[Network G_B] Total number of parameters : 26.815 M

Traceback (most recent call last):
File "test.py", line 29, in
for i, data in enumerate(dataset):
File "D:\MINE\Papers\instagan-master\data_init_.py", line 72, in iter
for i, data in enumerate(self.dataloader):
File "D:\devcool\Anaconda2\envs\python36\lib\site-packages\torch\utils\data\dataloader.py", line 286, in next
return self._process_next_batch(batch)
File "D:\devcool\Anaconda2\envs\python36\lib\site-packages\torch\utils\data\dataloader.py", line 307, in process_next_batch
raise batch.exc_type(batch.exc_msg)
RuntimeError: Traceback (most recent call last):
File "D:\devcool\Anaconda2\envs\python36\lib\site-packages\torch\utils\data\dataloader.py", line 57, in worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "D:\devcool\Anaconda2\envs\python36\lib\site-packages\torch\utils\data\dataloader.py", line 57, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "D:\MINE\STUDY in SYSU\SmartLLV\Papers\instagan-master\data\unaligned_seg_dataset.py", line 71, in getitem
A_segs = self.read_segs(A_seg_path, seed)
File "D:\MINE\STUDY in SYSU\SmartLLV\Papers\instagan-master\data\unaligned_seg_dataset.py", line 42, in read_segs
seg = self.fixed_transform(seg, seed)
File "D:\MINE\STUDY in SYSU\SmartLLV\Papers\instagan-master\data\unaligned_seg_dataset.py", line 34, in fixed_transform
return self.transform(image)
File "D:\devcool\Anaconda2\envs\python36\lib\site-packages\torchvision\transforms\transforms.py", line 60, in call
img = t(img)
File "D:\devcool\Anaconda2\envs\python36\lib\site-packages\torchvision\transforms\transforms.py", line 163, in call
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "D:\devcool\Anaconda2\envs\python36\lib\site-packages\torchvision\transforms\functional.py", line 208, in normalize
tensor.sub(mean[:, None, None]).div(std[:, None, None])
RuntimeError: The expanded size of the tensor (1) must match the existing size (3) at non-singleton dimension 0

Could you elaborate on what ImagePool.query does in a few words? thanks!

Hi，I find MHP and COCO generate seg label for test/val dataset, but CPP have not.
generate_ccp_dataset_val() in generate_ccp_dataset.py do not save seg label.
Why is that？
Who to test the model "python test.py --dataroot ./datasets/jeans2skirt_ccp --model insta_gan --name jeans2skirt_ccp_instagan --loadSizeH 300 --loadSizeW 200 --fineSizeH 300 --fineSizeW 200"?

Traceback (most recent call last):
File "test.py", line 10, in
opt = TestOptions().parse()
File "/content/options/base_options.py", line 103, in parse
opt = self.gather_options()
File "/content/options/base_options.py", line 68, in gather_options
model_option_setter = models.get_option_setter(model_name)
File "/content/models/init.py", line 30, in get_option_setter
model_class = find_model_using_name(model_name)
File "/content/models/init.py", line 10, in find_model_using_name
modellib = importlib.import_module(model_filename)
File "/usr/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'models.instagan_model'

Hi,

Do you think your approach would present an advantage over CycleGan when using a dataset of single objects on white background? For example
singe apple on white background -> single orange on white background

i finished training the model and ran the test.py. according to your code in test_model.py, i guess that you use image only as the input for the generator. but the generator requests both segmentation masks and images. so i'm quiet confused about it
def set_input(self, input): # we need to use single_dataset mode self.real_A = input['A'].to(self.device) self.image_paths = input['A_paths']