xiefan-guo / ctsdg Goto Github PK
View Code? Open in Web Editor NEW[ICCV 2021] Image Inpainting via Conditional Texture and Structure Dual Generation
License: MIT License
[ICCV 2021] Image Inpainting via Conditional Texture and Structure Dual Generation
License: MIT License
Dear author, please delete this question for me, thank you~
criteria/loss.py
loss_adversarial = criterion(real_pred, real_target) + criterion(fake_pred, fake_target) +
criterion(real_pred_edge, edge) + criterion(fake_pred_edge, edge)
I think the former three "criterion(real_pred, real_target), criterion(fake_pred, fake_target), criterion(real_pred_edge, edge)" are right, but the last one "criterion(fake_pred_edge, edge)", I think the second parameter is the "non-truth edge" instead of "the ground_truth edge".
I need your help to help me to understand the "criterion(fake_pred_edge, edge)".
作者你好,我在阅读您的代码的CFA模块RAL部分中有些地方没有看懂 比如代码中‘rate’的设置,以及‘background_kernel_size’设置为2*rate,而且这部分在论文中也没有提到 想问下这部分在哪有详细的叙述或者是参考了哪篇论文的实现
What is the size range of this training dataset? 30%? 40%? still have
Hello, author, I think your work is amazing, but when I started training, he made a mistake, I can't solve it temporarily, can you help me? thank you!
Traceback (most recent call last):
File "E:/code/CTSDG-main/train.py", line 102, in
train(opts, image_data_loader, generator, discriminator, extractor, generator_optim, discriminator_optim, is_cuda)
File "E:\code\CTSDG-main\trainer.py", line 45, in train
output, projected_image, projected_edge = generator(input_image, torch.cat((input_edge, input_gray_image), dim=1), mask)
File "D:\Anaconda3\envs\torch18\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "E:\code\CTSDG-main\models\generator\generator.py", line 248, in forward
dc_texture = torch.cat((dc_texture, ec_textures[ec_texture_skip]), dim=1)
RuntimeError: Sizes of tensors must match except in dimension 2. Got 3 and 2 (The offending index is 0)
您好,我想请问一下关于注意力模块CFA的问题。在计算受损区域与已知区域的余弦相似性时,是如何区分受损区域与已知区域的?期待您的回答,万分感谢!
Could you please provide the best checkpoint, thank you.
你好,最近我用你所提供的预训练模型和我自己训练的模型进行评估,但是他们的结果都比论文中给出的指标好太多了,我能直接引用论文中的指标吗?
Hello, when will the pre-trained models be available?
作者你好,我跑你代码的时候,发现不管是训练还是测试模型的时候,mask_root都只能用你的那个irregular_mask数据集才不会报错,而当我用我自己的mask数据集时,就会出现
而当我把训练代码
image_in_channels=3改为9时,又会报出这个错误
@xiefan-guo 请问这个问题该怎么解决。
Thank you for your excellent work, I would really like to reproduce your model as a baseline for comparison, at your convenience can you tell us the number of training and fine-tuning iterations for each dataset?Thanks !
projected_image = self.texture_feature_projection(dc_texture)
projected_edge = self.structure_feature_projection(dc_structure)
我想请问一下,这部分代码是运行哪里的?
I'm a beginner student and very interested in your paper. I don't understand one thing. How did the input of Structure Decoder in the generator, Corrupted Edge come from? There does not seem to be such a image in the training data set.
你好,我发现在你的论文中,无论是测试还是训练过程损坏的edge都是作为模型的输入的一部分。并且在你发布的训练和测试代码中,损坏的edge是通过将完整的edge点乘masks获得的,其中完整的edge是从完整的图像中获得的。
但是,在实际或测试中,我们并没有完整的图像,因此我们无法获得完整的edge。而且您并没有像 Edge Connect 那样预训练一个用于修复损坏的edge的模型,这也是您在论文中声称的贡献。
然而,从损坏图像中提取的edge并不等于从完整图像中提取出来的完整edge点乘mask,前者在损坏区域的附近会具有更少的信息。
我认为你对你的模型的测试方式有问题:通过将完整边缘乘以掩码替换从损坏的图像中提取边缘,在损坏区域的边界处存在信息泄漏。我相信这种泄漏将有利于修复性能。请问能解释一下吗?
我有复现和改进过这个模型,如果对CTSDG或者inpainting感兴趣的可以联系我,联系方式,wechat: lovedaixiaobaby
作者你好,就是我在用这个代码的时候,我用我自己的数据集,为什么一直会报通道数不匹配的问题,这个问题该怎么解决呢?
@xiefan-guo Hey. Thanks for releasing the codebase.
I tested your pretrained model on celeba, however, the results are not as expected. Even the identity of the person belongs to the training set, still it does not generalize well. What could be the reason?
Thanks for your research.
Dear authors, thank you for making this great work public.
I did not find quantitative results for celeba in your paper, such as ssim and psnr.And I want to know the results on celeba in your experiment.
I saw you post a pre-trained model about celeba. Can I calculate the indicator based on this model?
Hope to hear from you, thank you very much!
你好,最近我用您提供的预训练模型,还有我自己训练的模型在巴黎街景上进行了测试,我不知道是不是评价指标的版本问题,测得的结果要比论文中给出来的好太多,所以,您能提供一下您的测试指标代码嘛,以便更好的进行比较!!!急求!!!谢谢!
Hi,
Thank you for your great works. I couldn't find a section for quantitative results. Would you provide me with the pipeline with the quantitative results? And are you using irregular masks for different masks or do you use your own masks?
Are they isolated for testing or they are included in the training time too?
python test.py --pre_trained ./checkpoints/places2.pt --image_root ./images/ --mask_root ./masks/ --result_root ./result/ --number_eval 1
Sorry to bother you in this way.
When I tried to test this project following the instruction in markdown, this error shows up.
Cuda is available
OrderedDict([('pre_trained', './checkpoints/places2.pt'),
('image_root', './images/'),
('mask_root', './masks/'),
('num_workers', 4),
('batch_size', 1),
('load_size', (256, 256)),
('sigma', 2.0),
('mode', 'test'),
('result_root', './result/'),
('number_eval', 1)])
start test...
0%| | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
File "test.py", line 64, in <module>
output, __, __ = generator(input_image, torch.cat((input_edge, input_gray_image), dim=1), mask)
File "/root/anaconda3/envs/CTSDG/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(*input, **kwargs)
File "/root/CTSDG/models/generator/generator.py", line 97, in forward
ec_textures['ec_t_1'], ec_textures['ec_t_masks_1'] = self.ec_texture_1(ec_textures['ec_t_0'], ec_textures['ec_t_masks_0'])
File "/root/anaconda3/envs/CTSDG/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(*input, **kwargs)
File "/root/CTSDG/models/generator/pconv.py", line 103, in forward
images, masks = self.conv(images, masks)
File "/root/anaconda3/envs/CTSDG/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(*input, **kwargs)
File "/root/CTSDG/models/generator/pconv.py", line 50, in forward
padding=self.padding, dilation=self.dilation, groups=1)
RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[1, 9, 256, 256] to have 3 channels, but got 9 channels instead
And I have no clue how to fix it.
您好,我想问一下这个由原始完整图像处理成为Corrupted Edge和Gray-scale Image这部分的内容具体体现在程序里边的哪一部分呢?
作者您好,我在使用您的程序进行测试时发现,当我对多张图像进行修复时,result文件夹中修复后图像的顺序与我输入的待修复图像和掩码图像不能一一对应,请问应该如何调整才能使批量修复时获得有序的结果?期待您的回复!
Hi, I found that in your paper, the corrupted edge is also included as part of the model input during both testing and training process. And in the training and testing code you released, the corrupted edge is obtained by dot-multiplying the complete edge by the mask, where the complete edge is obtained from the complete image.
However, in practice or testing, we don't have complete images, so we are not able to get complete edges. And you are not pre-training a model for recontructing corrupted edges like Edge Connect does, which is the contribution you claimed in your paper.
As we know, the edge extracted from a corrupted image is not equal to the complete edge multiplied by the mask, the former will have less information in the vicinity of the damaged area.
I think there is something wrong with the way you are testing: replace extracting edges from corrupted images with multiplying the full edges by the mask, there is a leakage of information at the boundaries of corrupted regions. And I believe this leakage will help the improvement of inpainting performance. Can you explain it?
为什么在palace2数据集上训练的效果很差呀 psnr在15w次迭代的时候就到28点多,之后再怎么训练哪怕到50w次也还是这么多,不变了
Here I can see you add a comment.
Maybe you have some special reason for that, cause other size will not generate nice result.
And that will be helpful for me.
Code?
Hello, is there a code to divide mask pictures according to mask ratio? Thank you.
First of all, thank you very much for your outstanding contribution.May I ask about the ablation experiment of the two-stream network architecture in the article, in which the single-stream network with two branches simultaneously modeling the image structure and texture is accomplished?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.