ha0tang / selectiongan Goto Github PK

[CVPR 2019 Oral] Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translation

Home Page: http://disi.unitn.it/~hao.tang/project/SelectionGAN.html

Shell 1.39% Python 66.24% Lua 31.72% MATLAB 0.65%

computer-vision computer-graphics gans generative-adversarial-network deep-learning image-generation image-translation image-manipulation image-to-image-translation pytorch

selectiongan's Introduction

👯 We are looking self-motivated researcher to join/visit our Group.

Hao Tang

[Homepage] [Google Scholar] [Twitter]

I am currently a postdoctoral researcher at Computer Vision Lab, ETH Zurich, Switzerland.

⚡ News

We released the code of XingVTON and CIT for virtual try-on, the code of TransDA for source-free domain adaptation using Transformer, the code of IEPGAN for 3D pose transfer, the code of TransDepth for monocular depth prediction using Transformer, the code GLANet for unpaired image-to-image translation, the code MHFormer for 3D human pose estimation.

🌱 My Repositories

3D-Aware Image/Video Generation

3D-SGAN (ECCV 2022)

3D Human Pose Estimation

MHFormer (CVPR 2022)

Text-to-Image Synthesis

DF-GAN (CVPR 2022 Oral)
PPE (CVPR 2022)

3D Objection Generation

CGT (AAAI 2022)
IEPGAN (ICCV 2021)
AniFormer (BMVC 2021)

Monocular Depth Prediction

TransDepth (ICCV 2021)
StructuredAttention (CVPR 2018 Spotlight)

Face Anonymisation

AnonyGAN (ICIAP 2021)

Person Image Generation

XingGAN (ECCV 2020)
BiGraphGAN (BMVC 2020 Oral)
C2GAN (ACM MM 2019 Oral)
GestureGAN (ACM MM 2018 Oral & Best Paper Candidate)

Scene Image Generation

LGGAN (CVPR 2020)
DAGAN (ACM MM 2020)
DPGAN (TIP 2021)
SelectionGAN (CVPR 2019 Oral)
CrossMLP (BMVC 2021 Oral)
EdgeGAN
PanoGAN (TMM 2022)

Unsupervised Image Translation

GLANet
AttentionGAN (IJCNN 2019 Oral）
GazeAnimation (ACM MM 2020)
AsymmetricGAN (ACCV 2018 Oral)

Deep Dictionary Learning

DDLCN (WACV 2019 Oral)

Virtual Try-On

HCANet
CIT

Hand Gesture Recognition

HandGestureRecognition (Neurocomputing 2019)

Source-Free Domain Adaptation

TransDA

selectiongan's People

Contributors

Stargazers

Watchers

selectiongan's Issues

question

how to run the evaluation code ?does it need some pars?

self.padding, self.dilation, self.groups) RuntimeError: Calculated padded input size per channel: (3 x 3). Kernel size: (4 x 4). Kernel size can't be greater than actual input size

self.padding, self.dilation, self.groups)
RuntimeError: Calculated padded input size per channel: (3 x 3). Kernel size: (4 x 4). Kernel size can't be greater than actual input size

Can't download the dataset by scripts

Hello. I have used sh download_selectiongan_dataset.sh to download the dataset but the URLs seem to brokedown with ERROR 404: Not Found.

the code is different from the paper

in the paper, page 5, the Fm is said to be provided into 3 line, where a matrix multiplication operation was perform between 2 of them to create channel attention map A, but i can't see that in the paper. is that a new change? or it's a more effective way?

question

Execute code command --continue_train --which_epoch 35 --epoch_count 36,it comes an error.It can not found _net_D.pth.coule you tell me how to solve it?

a little requset

Would you please share me some papers about uncertainty map? I want to learn more about it. Thank you.

sharpness difference

Hello, could you provide the sharpness difference evaluation code? Thanks

Facial Expression Generation

Hi,

Sorry may you please guide me how I can download the facial expression data in your work

g2a checkpoint Issue

The images generated by provided pre-train checkpoints are not comparable to the samples shown in the paper. Following are the results for 64x64 and 256x256 respectively. And the script for testing is python test.py --dataroot /project/dzhang4/vast_data/dayton_5 --name dayton_g2a_256_pretrained --model selectiongan --which_model_netG unet_256 --which_direction BtoA --dataset_mode aligned --norm batch --gpu_ids 0 --batchSize 4 --loadSize 286 --fineSize 256 --no_flip --eval

inception score

Hello, I trained a model on the CVUSA dataset, but the IS value and KL result are slightly worse during the evaluation. The following are my running commands. Are there any problems that I have not noticed?
python train.py --dataroot ./datasets/cvusa/
--name cvusa_selectiongan
--model selectiongan
--which_model_netG unet_256
--which_direction AtoB
--dataset_mode aligned
--norm batch
--gpu_ids 0,1
--batchSize 4
--loadSize 286
--fineSize 256
--no_flip
--display_id 1
--lambda_L1 100
--lambda_L1_seg 1

Running code

Where should we run this code .In collab and kaggle notebook , we are running out of space , to download data sets ,with given command for pose transfer (person transfer).

Can't download pretrained models

Hello. I've been trying to download your pretrained models but neither the Google Drive, nor the Baidu link works. Could you please check this? Thanks.

Please include script to install requirements

Hello,
Kindly include a requirement to install the dependencies needed for installing the repository
Thanks

Arbitrary semantic map for dayton generation

Hi,
I am wondering if you have tried to use arbitrary semantic maps to generate different ground-view images on dayton? And how is the result?

How to evaluate your model, where is the code of metrics

Excuse me. Could you tell me how to get the scores of these metrics in your paper?
Such as: SSIM, IS, PSNR, DS, FID scores

Thanks!

a question about the equation(3) in your tpami submission

what does the paramater “j” mean？

Checkers on generated images

Hi,

I have tried to duplicate the result based on your dayton_a2g_256_pretrained model. However, there are many checkers in the generated images which is not clearly shown in Fig.4 of you paper. I followed all the hyperparameters as you mentioned in readme file

I am wondering if you encountered the same issue in your experiments or do I miss anything?

The attached is one of the screenshots. Thank you in advance!

hellow!

Hi! when I run the train,py ,I meet a error :ModuleNotFoundError: No module named 'models.pix2pix_model' . Could you help me ?

CNDNN_ERROR ?

Hello Sir,

Using my-datasets, I tried to train your code.
But I met CUDNN-ERROR.

...
/pytorch/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:312: operator(): block: [374,0,0], thread: [62,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed.
/pytorch/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:312: operator(): block: [374,0,0], thread: [63,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed.
Traceback (most recent call last):
  File "/data1/TESTBOARD/additional_networks/generation/SelectionGAN_Ha0Tang/semantic_synthesis/train.py", line 40, in <module>
    trainer.run_generator_one_step(data_i)
  File "/data1/TESTBOARD/additional_networks/generation/SelectionGAN_Ha0Tang/semantic_synthesis/trainers/pix2pix_trainer.py", line 35, in run_generator_one_step
    g_losses, generated = self.pix2pix_model(data, mode='generator')
  File "/home/itsme/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/itsme/anaconda3/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 166, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/home/itsme/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data1/TESTBOARD/additional_networks/generation/SelectionGAN_Ha0Tang/semantic_synthesis/models/pix2pix_model.py", line 46, in forward
    input_semantics, real_image)
  File "/data1/TESTBOARD/additional_networks/generation/SelectionGAN_Ha0Tang/semantic_synthesis/models/pix2pix_model.py", line 136, in compute_generator_loss
    input_semantics, real_image, compute_kld_loss=self.opt.use_vae)
  File "/data1/TESTBOARD/additional_networks/generation/SelectionGAN_Ha0Tang/semantic_synthesis/models/pix2pix_model.py", line 198, in generate_fake
    fake_image = self.netG(input_semantics, z=z)
  File "/home/itsme/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data1/TESTBOARD/additional_networks/generation/SelectionGAN_Ha0Tang/semantic_synthesis/models/networks/generator.py", line 90, in forward
    x = self.fc(x)
  File "/home/itsme/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/itsme/anaconda3/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 443, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/itsme/anaconda3/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 440, in _conv_forward
    self.padding, self.dilation, self.groups)
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED
terminate called after throwing an instance of 'c10::CUDAError'
  what():  CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Exception raised from create_event_internal at /pytorch/c10/cuda/CUDACachingAllocator.cpp:1055 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f44b3256a22 in /home/itsme/anaconda3/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: <unknown function> + 0x10aa3 (0x7f44b34b7aa3 in /home/itsme/anaconda3/lib/python3.7/site-packages/torch/lib/libc10_cuda.so)
frame #2: c10::cuda::CUDACachingAllocator::raw_delete(void*) + 0x1a7 (0x7f44b34b9147 in /home/itsme/anaconda3/lib/python3.7/site-packages/torch/lib/libc10_cuda.so)
frame #3: c10::TensorImpl::release_resources() + 0x54 (0x7f44b32405a4 in /home/itsme/anaconda3/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #4: <unknown function> + 0xa2f382 (0x7f4558065382 in /home/itsme/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #5: <unknown function> + 0xa2f421 (0x7f4558065421 in /home/itsme/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
<omitting python frames>
frame #21: __libc_start_main + 0xe7 (0x7f455add0b97 in /lib/x86_64-linux-gnu/libc.so.6)

How to solve it??

Thanks,
Edward Cho.