backseason / poolnet Goto Github PK

你好，我看你们的代码有resent 和vgg做backbone 的，我可以改用其他，例如googlenet，这样代码会是否可以跑通？
如果可以跑通，那backbone 更换过程，需要更改和注意哪些地方

我在训练时采用ＤＵＴＳ－ＴＲ测试使用ＤＵＴＳ－ＴＥ，但是发现ｓａｌ　ｌｏｓｓ值在几千至一万，为何值这么大，但是测试时发现效果还不错。　为何不收敛却效果很好。并且在ｓｏｌｖｅｒ中的ｂｃｅ２ｄ（）函数是准备干嘛用的，和您使用的损失函数有何区别，谢谢！

I evaluated the test results using usual method.
I find that MAE score is closed to your conclusion while the value of MaxF is far from your's
So,Could you provider the evaluation code?
Thank you

您好，我看到您在code当中对resnet50和vgg16的dataset做了相同的处理，请问您提供的预训练模型是基于什么的比赛的么？我看网上流传的resnet50和vgg net 对数据预处理的方式不一样

Thanks for your project. I have read your code about PoolNet, i want to know why there is no Batch Normalization in your decoder ,eg.

您说“VGG-16是直接使用的HED中提供的Caffe版本的预训练参数，ResNet-50也同样使用的Caffe版本的预训练参数；两种参数都不是基于什么比赛的，只是输入图像格式是BGR, [0-255]。”，然而我看代码中是减去一个均值的，这其中原因是什么呢？

Why do I check your vgg16 results and your article is not correct (such as ECSSD I measured 0.081421 and yours is 0.044, PASCAL I measured 0.119801 and yours is 0.072), is this too bad? More, of course, I tested your resnet50 is OK (such as ECSSD I measured 0.038383 and your article is 0.038, PASCAL I measured 0.075219 and your article is 0.065), can you explain?

I just feel curious about the joint training result and loaded the joint-poolnet model you provided.

I tested on the HUK-IS dataset and the output is not the same as yours, the edges are not correctly predicted.

Seems like the pre-train model is not working.

may get your advices on this?

There are two saliency maps missing on DUT-OMRON: sun_akxddynsopjifavt.png and sun_barwtmnxkuxyttlj.png.

Dear author,

Thanks for your great work !

How can I make my own database and train on it?

Thank you !

您好，我想请问下数据训练时采用batchsize=1是因为，训练图片不一致导致的吗，这一块会不会影响收敛，我这边20以后收敛不理想

Hi,

Thanks for your codes. I find that three image pairs (ILSVRC2014_train_00023530, n04442312_17818, n01532829_13482) in your DUTS.zip are not matched as your train_pair.lst.

您好，首先感谢您的工作，看上去确实是一个很有效的模型。

关于这里使用的边缘检测数据集，我有些疑惑。

您在文中提到：

我查阅对应的文献：Yun Liu, Ming-Ming Cheng, Xiaowei Hu, Kai Wang, and Xiang Bai. Richer convolutional features for edge detection. In CVPR, 2017，文章中也提到：

请问文章实际使用的边缘检测数据集是BSDS500和PASCAL VOC Context数据集的混合数据集么？

但是我疑惑的缘由是，这里的Pascal VOC Context的数据集来自Pascal VOC 2012，而实际测试用的Pascal-S数据集也来自Pascal VOC。

https://ccvl.jhu.edu/datasets/

在这里使用来训练，是否有些不妥？虽然真值不同，但是最终的效果的提升会不会由此导致？

因为最终的结果中，Pascal-S的结果在使用边缘数据之后，提升非常大。

另外，由于偶然间的发现，ECSSD中的数据部分与BSDS500相同。这样的话会不会也在一定程度上有所提升？

Thank you for your contribution.I have try to read the code,but I use to read code in Keras.Could you provide the code of network in Keras?Thank you !

据我所知，HKU-IS数据集包含的图片数量是4447张。

谢谢。

Hello, thank you for sharing the code. I need to do a comparison experiment and want to compare with your algorithm, but I didn't find your model training results. Can you provide the test results on six data sets? If you can provide it, please share the download address of Baidu Web Disk. Looking forward to your reply, I wish you a happy life.

请问能否提供下是实验结果图saliencymap

您好，请问测试结果保留在哪里？评价的指标等数据。万分感谢。
The number of parameters: 69557642
Loading pre-trained model from results/run-1/models/final.pth...
Speed: 31.047007 FPS
Test Done!
Testing on e,p,d,h,s,t datasets done.

请问显示这个就是测试结束了吗？

Hi. I am trying joint_main.py for training using spyder, but can't find train_edge_root & train_edge_list file. Thank you for your help.

HI, guy. Thanks for your project. I read this code, but i didn't find any code for the Joint Training for the Edge. Could you please tell me, which line represent the joint training, and how the 2 loss merge?

It's only a very minor issue, but in the datasets section of the README, the links to the MSRA-B/HKU-IS and DUTS datasets are swapped. That is, MSRA-B and HKU-IS navigates to DUTS.zip, and DUTS navigates to msrab_hkuis.zip.

Hi,

I notice that whatever train or test your model, you chose net.eval() mode to run the whole procedure. Does that mean you use the Batchnorm layer only with its init running mean and var? Does this operation make your model have better performance through your experiments?

Thanks for your reading and I look forward to hearing from you.

Can you provide the pre-computed saliency maps, both vgg and resnet version?

I'm looking for the implementation of the Global Guidance Module.
Is it released in the source/pretrained model already?

您好！
我仔细读了你们的实验部分，我有一点疑问，就是你们的FAM模块和图1中的F模块，实验效果是否和这两者的个数和位置有关？

Hi, thanks for your great work.
I find that your run-2.zip does not include the trained model, i.e., run-2/models/final.pth. Could you please offer that? I want to test it on other datasets. Thank you!

I am testing with my own data with the pretrianed model, using python main.py --mode='test' --model='results/run-1/models/final.pth' --test_fold='results/run-1-sal-e' --sal_mode='e' --batch_size=1 --no-cuda
Here is what i got

Loading pre-trained model from results/run-1/models/final.pth...
Traceback (most recent call last):
  File "main.py", line 97, in <module>
    main(config)
  File "main.py", line 46, in main
    test = Solver(None, test_loader, config)
  File "...\repos\PoolNet\solver.py", line 31, in __init__
    self.net.load_state_dict(torch.load(self.config.model, map_location='cpu'))
  File "...\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 769, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for PoolNet:
        Unexpected key(s) in state_dict: "block.0.block_in1.0.weight", "block.0.block_in2.0.weight", "block.0.block_mid1.0.0.weight", "block.0.block_mid1.0.1.weight", "block.0.block_mid2.0.0.weight", "block.0.block_mid2.0.1.weight", "block.0.block_out.0.weight", "block.1.block_in1.0.weight", "block.1.block_in2.0.weight", "block.1.block_mid1.0.0.weight", "block.1.block_mid1.0.1.weight", "block.1.block_mid2.0.0.weight", "block.1.block_mid2.0.1.weight", "block.1.block_out.0.weight", "block.2.block_in1.0.weight", "block.2.block_in2.0.weight", "block.2.block_mid1.0.0.weight", "block.2.block_mid1.0.1.weight", "block.2.block_mid2.0.0.weight", "block.2.block_mid2.0.1.weight", "block.2.block_out.0.weight", "block.3.block_in1.0.weight", "block.3.block_in2.0.weight", "block.3.block_mid1.0.0.weight", "block.3.block_mid1.0.1.weight", "block.3.block_mid2.0.0.weight", "block.3.block_mid2.0.1.weight", "block.3.block_out.0.weight", "fuse.trans.0.weight", "fuse.trans.0.bias", "fuse.trans.1.weight", "fuse.trans.1.bias", "fuse.trans.2.weight", "fuse.trans.2.bias", "fuse.trans.3.weight", "fuse.trans.3.bias", "fuse.fuse.weight", "fuse.fuse.bias", "edgeinfo.trans.0.weight", "edgeinfo.trans.2.weight", "edgeinfo.trans.4.weight", "edgeinfo.trans.6.weight".
        size mismatch for score.score.weight: copying a param with shape torch.Size([1, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([1, 128, 1, 1]).

Do you know what could be the problem?

Hello, I am very interested in your research direction. Thank you for sharing the code. I am a graduate student and my research direction is related to saliency target detection. Your idea is very novel. Can you share the code for PR curve drawing? I want to see the final effec，Looking forward to your reply, I wish you a happy life.

Hi @backseason,

I got some introduction about this work from here https://mp.weixin.qq.com/s?__biz=MzIwMTc4ODE0Mw==&mid=2247497352&idx=1&sn=dff7e64e7d388fbb394d6f7c81a544bc&chksm=96ea2908a19da01edc1697644be521b818e08625e13d20aff91aa350e8a8197fb963e84eaf9e&scene=0&xtrack=1#rd .

The idea is simple yet effective, thanks for your work. My question is, have you tried to implement FAM as a refinement of FPN in those SOTA detection algorithms like yolov3 or retinanet? I think the FAM should be useful in detection also.

GPU: TITAN V 12GB

system: ubuntu 16.04

Pytouch version: Pytorch 0.4.1

cmd:

python  main.py --mode='test'  --model='run_data/run-0/models/final.pth'  --test_fold='./ret/' --sal_mode='test_data'

'run_data/run-0/models/final.pth' is downloaded from the url in README.md
'test_data' is my own test images with shape (300, 400, 3)
The pretrained /dataset/pretrained/resnet50_caffe.pth and run-0/models/final.pth is loaded ok, but error happened when start to test first img.

error:

Traceback (most recent call last):
  File "main.py", line 99, in <module>
    main(config)
  File "main.py", line 49, in main
    test.test()
  File "/home/wepleo/dplearning/commodity_classification/pytorch/PoolNet-master/solver.py", line 70, in test
    preds = self.net(images)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/wepleo/dplearning/commodity_classification/pytorch/PoolNet-master/networks/poolnet_1task.py", line 101, in forward
    conv2merge, infos = self.base(x)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/wepleo/dplearning/commodity_classification/pytorch/PoolNet-master/networks/deeplab_resnet.py", line 197, in forward
    infos.append(self.infos[k](F.interpolate(xls, xs[len(self.infos) - 1 - k].size()[2:], mode='bilinear', align_corners=True)))
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/container.py", line 91, in forward
    input = module(input)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/conv.py", line 301, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: CUDA error: out of memory

There are some classes ConvertLayer,DeepPoolLayer,ScoreLayer and base in networks/PoolNet.py .I want to know these classes correspond to which module refering to paper,for example GGM module,FAM module and Edge Detection module.

Hi.
First of all, I would like to thank you for this great work.
I have finished training the model on MSRA-B dataset, and observed the model works well on the test set.
However, I have a concern about the reliability of this model.
So, I want to evaluate this model. How can I do?
Thank you.

请问1.2的pytorch能否运行

Hi, would you like to tell why you set ceil_mode to True? is it that a trick to get better performance or some other reason?

Thanks for your great work!

I want to directly test your model on my personal dataset but it seems like only pretrained backbone is provided.

Could you provide the trained entire model with ResNet50 backbone ?

Thank you ~

I cannot get the similar results with provided resnet-50 (w/o edge run-0) as repoted in paper.

Thank you for your excellent work,I recently trained the model on other datasets. In addition to the default parameter settings, do you have any suggestions for training tuning?
Looking forward to your reply.

Hi,
I keep getting this memory error every once in a while when I run "joint_main" test with batch size =1. I thought that you might have advice for me to overcome that.

文中对比的方法有Picanet、DGRL、SRM、DSS、DCL等等，
可以提供一下，这些方法测试得到的所有测试集的图片吗？
谢谢您！

hi,thanks for your project.Ｉs it only possible to test with public datasets in your project? If I want to test with my own dataset, what should I do?

Good result, but the big problem of code is written in a very confusing way.
for example, it is not easy to run code for just one image as the input and get one output!!! [ I mean not in the list file ] ...
Please give some guidance about running code with a single image. ( not from the list)

thank

Hi, i am to trying to run it to test some images but keep getting this error when using cpu for inference

Dear author,

Thanks for your great work !

Could you provide the trained model in Baidu cloud disk that I want to directly test on my personal data ? Beacause I can't get the shared model from your given google link.

Thank you !

Hi @backseason @Andrew-Qibin , thanks for your great work! I am trying to retrain the model, but the training loss going with a large fluctuation. Is that because of no BN layer or need a pre-trained backbone? By the way, could you please provide the pre-trained ResNet or VGG model since you modified the network. looking forward to your reply.

Excuse me, is there no visual code in this code?
Similar to the chart in this paper