highwaywu / imageinpainting Goto Github PK

Hello, I would like to ask about the issue of attention networks. In the attention.py, what are the generated and known values of the input network in line 20? Are the image features of unknown and known regions?

Hello,

Thank you for such a great job and for releasing your code.
I want to train your network using a custom dataset. When I looked at the options.py file, the batch_size parameter is set to 1 and has not changed anywhere in other training files.
Do you find empirically that batch_size 1 works best with your network? Please suggest this parameter as my dataset contains almost 15000 images and it will be very slow to train with batch_size 1.
Thanks.

Hello, is there any code about the optimizer LBP? I would like to reproduce this work of yours, thank you very much! Because when I look at the first stage results, the results are always unsatisfactory without restoring the basic texture, I would like to look at the first stage results

Hello, I encountered this error while running the code. I downloaded the code and ran it directly. I have installed python 1.6.0, cuda 10.2, cuDNN7.6 and python 3.7. What is the situation, please?

Train/Val with 4/4
Epoch: 1
/LBP/model.py:178: UserWarning: masked_fill_ received a mask with dtype torch.uint8, this behavior is now deprecated,please use a mask with dtype torch.bool instead. (Triggered internally at /pytorch/aten/src/ATen/native/TensorAdvancedIndexing.cpp:570.)
I_i.narrow(1, 0, 1).masked_fill_(self.mask_tmp, 0.)
/LBP/model.py:179: UserWarning: masked_fill_ received a mask with dtype torch.uint8, this behavior is now deprecated,please use a mask with dtype torch.bool instead. (Triggered internally at /pytorch/aten/src/ATen/native/TensorAdvancedIndexing.cpp:570.)
I_i.narrow(1, 1, 1).masked_fill_(self.mask_tmp, 0.)
/LBP/model.py:180: UserWarning: masked_fill_ received a mask with dtype torch.uint8, this behavior is now deprecated,please use a mask with dtype torch.bool instead. (Triggered internally at /pytorch/aten/src/ATen/native/TensorAdvancedIndexing.cpp:570.)
I_i.narrow(1, 2, 1).masked_fill_(self.mask_tmp, 0.)
/LBP/model.py:181: UserWarning: masked_fill_ received a mask with dtype torch.uint8, this behavior is now deprecated,please use a mask with dtype torch.bool instead. (Triggered internally at /pytorch/aten/src/ATen/native/TensorAdvancedIndexing.cpp:570.)
self.L_i.narrow(1, 0, 1).masked_fill_(self.mask_tmp, 0.)
main.py:132: FutureWarning: multichannel is a deprecated argument name for structural_similarity. It will be removed in version 1.0. Please use channel_axis instead.
ssim.append(compare_ssim(a[i], b[i], win_size=11, data_range=255.0, multichannel=True))
Val (4/4) G:17.1737, S:0.8090, P:22.16, M:0.0603
Traceback (most recent call last):
File "main.py", line 221, in
train()
File "main.py", line 185, in train
model.save_networks('Model_weights')
File "/LBP/model.py", line 74, in save_networks
net.cuda(self.opt.device)
File "/root/miniconda3/envs/myconda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 458, in cuda
return self._apply(lambda t: t.cuda(device))
File "/root/miniconda3/envs/myconda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 354, in _apply
module._apply(fn)
File "/root/miniconda3/envs/myconda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 354, in _apply
module._apply(fn)
File "/root/miniconda3/envs/myconda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 376, in _apply
param_applied = fn(param)
File "/root/miniconda3/envs/myconda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 458, in
return self._apply(lambda t: t.cuda(device))
RuntimeError: Invalid device, must be cuda device

It takes me 20 days to train 1000 rounds with 1080TI, what about you?

Hello, if I want to get the results of the first phase of LBP inpainting how should I get it?

Hello, I'm getting this error, using my masks and images:

Traceback (most recent call last): P:31.99, M:0.0248
  File "main.py", line 226, in <module>
    test()
  File "main.py", line 209, in test
    I_g, I_o, val_loss_G = model.optimize_parameters(val=True)
  File "/app/model.py", line 269, in optimize_parameters
    self.forward()
  File "/app/model.py", line 184, in forward
    _, self.I_FEA = self.netG(self.I_g, self.L_g, self.mask)
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/app/networks.py", line 211, in forward
    up31 = self.myattention(up31[:, :256, :, :], up31[:, 256:, :, :], mask)
  File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/app/attention.py", line 13, in forward
    return MyAttentionFunction.apply(generated, known, mask, self.device)
  File "/app/attention.py", line 45, in forward
    weight1, indexes1 = torch.topk(cosine1, k, dim=2)
RuntimeError: selected index k out of range

Also is using torch==1.5.1 okay? As I can't figure out how to install tf==1.8.0 and torch==1.6.0 - because they require different cuda versions.

Thanks!

Hello, I would like to inquire about the specific training details of your article, as I did not find any specific comparison results in the Paris street view dataset during the replication process, and the replication results in other datasets are not as good as the article. Is it random selection and amplification of each proportion of masks in the test, and training with the dataset? If it is a better training method, please let us know.

RuntimeError: invalid argument 5: k not in range for dimension at /pytorch/aten/src/THC/generic/THCTensorTopK.cu:26

This error is attributed to the attention network where the topk function is being used. I have identified the output of F.interpolate(mask.clone(), (32, 32)).view(1, -1) has very few values that are equal to 1. Sometimes, none of the value is 1 and sometimes only one of the value is 1. This raises an issue in topk function.
Please let me know how can I resolve this error.

Thank you.

When I put the image and mask as the above picture's root,it just gives the inpainted result,but where should I get the masked image generated from the origin image and mask?

Hello,

Thank you for such a great job and for releasing your code.
I want to train your network using a custom dataset. When I looked at the options.py file, the batch_size parameter is set to 1 and has not changed anywhere in other training files.

batchSize=2，Traceback (most recent call last):
File "E:\LBP_idea\gabor+dense\gabor-test-dense\main.py", line 234, in
train()
File "E:\LBP_idea\gabor+dense\gabor-test-dense\main.py", line 151, in train
I_g, I_o, loss_G = model.optimize_parameters()
File "E:\LBP_idea\gabor+dense\gabor-test-dense\model.py", line 291, in optimize_parameters
self.forward()
File "E:\LBP_idea\gabor+dense\gabor-test-dense\model.py", line 186, in forward
_, self.I_FEA = self.netG(self.I_g, self.L_g, self.mask)
File "D:\Anaconda\envs\pytorch_3.8\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "E:\LBP_idea\gabor+dense\gabor-test-dense\networks.py", line 265, in forward
up31 = self.myattention(up31[:, :256, :, :], up31[:, 256:, :, :], mask)
File "D:\Anaconda\envs\pytorch_3.8\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "E:\LBP_idea\gabor+dense\gabor-test-dense\attention.py", line 13, in forward
return MyAttentionFunction.apply(generated, known, mask, self.device)
File "D:\Anaconda\envs\pytorch_3.8\lib\site-packages\torch\autograd\function.py", line 506, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
File "E:\LBP_idea\gabor+dense\gabor-test-dense\attention.py", line 27, in forward
patches = patches_all[ctx.flag == 1]
IndexError: The shape of the mask [1, 2048] at index 0 does not match the shape of the indexed tensor [2, 1024, 256] at index 0

Process finished with exit code 1

Hi, I really appreciate your work. May I know when you will update the code?

While going through the code, I found that netLBP's parameters are not getting updated as the optimizerLBP.step() function is not called anywhere in model.py file.

https://github.com/HighwayWu/ImageInpainting/blob/master/model.py#L268
I believe that the above link indicates the function that optimizes parameters of netG, netD, netD2 but it does not update netLBP. Can you please point me to the correct location where the optimizer for LBP is calling the step function? Thank you.

In the paper, your description is "In the loss functions, the parameters trading off different terms are set to be λm = 0.01, λr = 10, λa = 0.2, λp = 1 and λs = 10. Note that all the parameters are fixed when performing the subsequent experiments.", but in the code, many weights are different. I would like to ask the author, is it according to the description in the paper that all stages are set the same, or is it according to the code that the weights of the two generators are different?Just like reconstruction loss, the weight in the paper is 10, but opt.lamda_A in the code defaults to 100.

Hi, I want to know how i can use my own image for model testing.
Because when I put simply a 256x256 image it gives an error.
File "main.py", line 85, in load_lbp
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.error: OpenCV(4.1.2) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

pytorch1.6.0 is not compatible with tenssorflow1.8.0