xuebinqin / basnet Goto Github PK

Hello @Nathanua , thank you so much for your works. Here I still confuse how to evaluate your model on PASCAL-S dataset. As we know the pascal dataset Ground-truth is not binarize yet. It has value not only 0 or 255. How to process it and evaluate your model?
-Thank you-

As you can see, the shoe is not fully detected, the white part of it has been cut off. how can i overcome this ?

尊敬的作者您好，
我认为您在论文中关于损失函数的分析相当的精彩。但是我有几个疑问：1）这些展示图是如何制作出来的？（个人猜测是保存不同训练阶段的模型，处理同一幅图片，然后可视化三个loss）
2）为什么SSIM对边缘的不一致很敏感？根据图片展示确实如此，有理论上的解释吗？（其实本质问题是我想知道您的hybrid loss是理论指导设计还是实验指导设计的。）

Hi
Thank you for the great work on this repository. How do I change the confidence value of the prediction in basnet_test.py ?
Thanks ... :)

代码没有条件跑，弱弱的问一下，labels_v从dataloader中取出好，没有做尺度变换，直接给d0~d7做loss了，这里是如何操作的？没有看明白。

Thank you for your answer, but how should this be used? I didn’t find any specific application in your U_2 code

您好，我训练自己数据集的时候发现在15.6K迭代额时候，d0和d1的输出为nan，导致l0和l1损失为nan
`[epoch: 308/100000, batch: 2456/ 4085, ite: 156707] train loss: 2.115248, tar: 0.097755
l0: 0.090264, l1: 0.090268, l2: 0.094700, l3: 0.108498, l4: 0.157684, l5: 0.269343, l6: 0.561054

[epoch: 308/100000, batch: 2464/ 4085, ite: 156708] train loss: 2.114669, tar: 0.097731
l0: 0.110660, l1: 0.110660, l2: 0.116909, l3: 0.147194, l4: 0.230913, l5: 0.414125, l6: 0.675684

[epoch: 308/100000, batch: 2472/ 4085, ite: 156709] train loss: 2.115880, tar: 0.097773
l0: 0.101519, l1: 0.101512, l2: 0.107206, l3: 0.128373, l4: 0.198813, l5: 0.377140, l6: 0.674387

[epoch: 308/100000, batch: 2480/ 4085, ite: 156710] train loss: 2.116687, tar: 0.097785
l0: 0.092943, l1: 0.092937, l2: 0.097863, l3: 0.117802, l4: 0.182888, l5: 0.299898, l6: 0.505494

[epoch: 308/100000, batch: 2488/ 4085, ite: 156711] train loss: 2.115991, tar: 0.097769
l0: 0.104595, l1: 0.104529, l2: 0.109673, l3: 0.131785, l4: 0.201885, l5: 0.407138, l6: 0.842563

[epoch: 308/100000, batch: 2496/ 4085, ite: 156712] train loss: 2.118025, tar: 0.097791
l0: nan, l1: nan, l2: 2.413359, l3: 2.419617, l4: 2.441422, l5: 2.419549, l6: 2.403301

[epoch: 308/100000, batch: 2504/ 4085, ite: 156713] train loss: nan, tar: nan
l0: nan, l1: nan, l2: 2.489194, l3: 2.498003, l4: 2.527905, l5: 2.497976, l6: 2.474765
`

Hello, i am curious about why the dilation of conv6d_m/conv6d_2 is different from the other stages like conv5d_m/conv5d_2. Is it a trick?

Dear Author:
Thank you for sharing the code. If I want to use your code to train my own data, is there any requirement for the size of the input training image?

I use your pre-trained model on HKU-IS and test on the result which has 0.030 MAE and 0.99 max F_beta!!(save and load the saliency map) Do I use the wrong way to test? Could you share how you test?

Hi,

Thanks for your code!

Recently I need to re-train this network, on another dataset, HKU-IS, the model converges good,
but I don't know when to stop it...

the loss now in your code is 0.099 and 2.83 respecitively...
Because we don't have evaluation during the training...thus I do not know how good is the result now...
can you tell us when to stop the training when you train the BASNet using DUT-TR dataset?

I trained my data set, with a total of 1600 pictures, and trained 9 epoch, 1153 iterations. Batchsize set 12, but the total loss was always around 14-15, and occasionally changed to 13.8-13.9. Is this normal?

Dear author, there are still some missed checks when testing this code? How should I pre-process the data set (e.g. brighten or darken?) to ensure that no inspection is missed (even a few more notable marks are fine)?

Hi,
I'm trying to run the test code but i'm getting the following error:
File "C:/CP/DL_REPOSITORIES/BASNet/basnet_test.py", line 76, in
for i_test, data_test in enumerate(test_salobj_dataloader):
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 345, in next
data = self._next_data()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 841, in _next_data
idx, data = self._get_data()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 808, in _get_data
success, data = self._try_get_data()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 774, in _try_get_data
raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str))
RuntimeError: DataLoader worker (pid(s) 16732) exited unexpectedly

Can you please help?

尊敬的作者：谢谢你的回复，我在测试自己的视频的时候经常出现输入图像尺寸不匹配的问题，但是我在你的U^2NET的代码中并没有看到关于upsample functions函数使用问题，希望你能够更加详细的帮我解答一下，

Am trying to use TensorRT to speed up the inference. I converted my model trained from scratch to ONNX to feed to TensorRT. But TensorRT is throwing this error while parsing the model,

While parsing node number 186 [Gather]:
ERROR: onnx2trt_utils.hpp:277 In function convert_axis:
[8] Assertion failed: axis >= 0 && axis < nbDims
[E] Unable to parse ONNX model file: ../data/int8_api/basnet.onnx

Any pointers on how to address this?

Thanks.

文中对比的方法有Picanet、PAGRN、DGRL、RAS等等，
可以提供一下，这些方法测试得到的所有测试集的图片吗？
谢谢您！

The result on segmentation looks pretty good, but as the result is only a map image, how could I detect the objects in it? for example, detect cats, dogs in it with bounding boxes, I have applied the findcontour function on the resulted map, there are too much contours on a single "area"

Hello, I used your code to add acc above, but there is a problem, loss can go down, acc almost unchanged

Hi @Nathanua , thanks for your great work. Recently I am working on Salient object detection and I was trying to re-train BASNet from scratch to benchmark our own work. However I found I cannot reproduce reported result. I got around 0.078 in MAE on DUTS-TE(Trained on DUTS-TR), and I used the evaluation code you provided here https://github.com/NathanUA/Binary-Segmentation-Evaluation-Tool
Could you please give me some hint or explanation? thanks in advance!

作者您好，我看了一些您的training代码，发现你在训练过程中并没有validation阶段，是不需要吗？那停止训练判断的标准是loss收敛或者上升吗？
其次，您在论文中的可视化结果，是直接模型输出（概率图）-》归一化[0，1]-》保存为RGB（0-255）后直接展示，还是说您在展示前使用了某个阈值（假如阈值是0.5）将模型输出图片（像素值为0到255范围的整数）变为mask（像素值只有0或者255）了？

I want to re -train the basnet.pth model with my custom data set . I use the following code in the bastnet_train.py

model_dir_path = './saved_models/basnet_bsi/basnet.pth'
net = BASNet(3, 1)
net.load_state_dict(torch.load(model_dir_path))
if torch.cuda.is_available():
net.cuda()

I add the following line to load the existing model weights.
net.load_state_dict(torch.load(model_dir_path))

After training with my custom data set the model performance become down compare to the before re-train model . But working good with the similar type of data by which i re-train with.
Please guide me.

Dear Nathan, i've used your BASNet code but i'm confused, why test data and test result are in different orientation? for example, i put horizontal orientation test data & the result is vertical orientation.

thank you.

作者您好，想请教您一下，paper里面loss热图那里（图5）提到的前景和后景的预测概率指的是什么呀，我不是很理解，谢谢。

Hello,Thank you very much for your work ! Did you test the thur15 dataset?

I wonder, how to train only the refinement module? Cause I don't get familiar with Pytorch. Thank you.

My machine is 1080ti, which is running short of gpu memory.

有没有测试代码的repo

https://github.com/NathanUA/BASNet/blob/80ede575fd4d16507bb16db3263488e960e68771/model/BASNet.py#L288

Should be "hd6 = self.relu6d_2(self.bn6d_2(self.conv6d_2(hx)))", right?

作者您好，非常感谢您的分享。
关于我在使用加入SSIM loss进行监督的时候，预测结果会出现间隙或者空洞，这是什么原因导致的呢？

Hello, because I am interesting in both loss and want to do some experiment.
Could both of them directly combined with semantic segmentation? Because this paper is for salient object detection.
Thanks You!

i want to ask about this:

if ite_num % 2000 == 0: # save model every 2000 iterations

        torch.save(net.state_dict(), model_dir + "basnet_bsi_itr_%d_train_%3f_tar_%3f.pth" % (ite_num, running_loss / ite_num4val, running_tar_loss / ite_num4val))
        running_loss = 0.0
        running_tar_loss = 0.0
        net.train()  # resume train
        ite_num4val = 0

why running_loss , tar_loss and ite_num4val is set to 0 after saving the model ? thankss

Hello @Nathanua thank you for the great work and code. I have several problems as follows.

1. I am trying to reproduce the evaluation metric. Do you have a code for evaluating the results?
2. For the precision and recall did you use the same threshold for prediction and ground truth?, so what happens if the threshold is 0 which means all of the pixels will be white and means the true positive will be high since all pixel in prediction and ground truth becomes white?, and also what happen if the threshold is 1?
3. Could you explain the relaxed F-measure?, I am still confused and trying to figuring out is it like these steps?

• Binarize prediction (pred) and ground truth (gt) with threshold = 0.5
• Do an erode operation in both pred and gt, how to do it? what is your kernel size? is it 3x3 as stated in here
• Do XOR operation in eroded and non-eroded version which means the different pixels will be white and the same pixels will be black (it will be like edge only).
• The precision and recall will be computed in predXOR and gtXOR however I am still confused about slack p where it set to be 3 pixels. then the relaxed F measure is similar computation with the original F measure.

-Thank you-

Dear Nathan, i've used your BASNet code but i'm confused.

1. based on my experiment, is it true test data and test result are in different orientation? for example, i put horizontal orientation test data & the result is vertical orientation. sorry if wrong.

2. can you explain what is running_loss, running_tar_loss, ite_num4val and tar ?

thank you before :)

Hello, I want to ask some questions about iou loss and ssim loss.
When I saw the code basnet_train.py, I found that :

ssim_out = 1 - ssim_loss(pred,target)
iou_out = iou_loss(pred,target)

I think that iou loss still need to be subtracted by 1 like that:
iou_out = 1 - iou_loss(pred,target)

But why it do not need to do that?
Thanks you!

大佬666

我想请问一下论文图一中的 boundary 是怎么生成的？在原论文中好像没有相关的描述，谢谢您！

感谢您的工作！
我想请问一下在basnet_test.py文件的最后，对于输出结果进行了一个归一化操作normPRED
def normPRED(d):
ma = torch.max(d)
mi = torch.min(d)
dn = (d - mi) / (ma - mi)
return dn
通常分割将网络输出层sigmoid之后作为最终结果，为什么在这项工程中加入了这种归一化操作，是有什么其他的考虑吗？ 期待着您的回复！

作者您好，很感谢您之前的回复，对我很有帮助。我根据您的readme下载了代码，预训练模型并在本地重新生成了结果，我对比了SOD数据集。本地生成的结果在指标上（MAE，maxF和S-measure）要稍微好于论文（即您所提供的结果），我想（如果我的猜想并不正确的话，请您指正）这可能是pytorch版本（1.1.0 vs 0.4.0）或者系统环境（我在linux服务器上重现结果，并在windows系统上使用sal_toolbox评估模型）。
但是，出现了一个问题，不论是您提供的结果，还是我所重现的结果，在PACSAL-S数据集上测试时，maxF均为NaN（AUC也是）。另注：PASCAL-S的gt为uint8格式（SOD为logical格式），但是即使将格式进行转换后，测试指标仍然为NaN；除了PASCAL-S以外的其他数据集上的指标均是正常的。

Hi. This model of GPU memory is large. Have you tried to optimize it? Thanks

我下载了您给出的预训练模型，并放在了所指定的文件夹内。
但是在运行test文件的时候，还是会从pytorch的网站上下载模型

HI, you mentioned that SSIM loss is used to encourage the original image structure in your paper. And it's reasonable. Recently, I want to add ssim loss to image segmentation to increase the building extraction performance. In my experiment, the ssim training loss is convergence well while the SSIM result between pred segmentation result and GT is lower than before.
I'm really confused. Any answer or suggestion will be thankful!

Hi, again, thanks for your work and paper!

During my search of the model, I found some of them do not have pre-calculated saliency map, like PiCANet, DSS(no DUTSTE), etc.
but in your paper, you include them for PR curve comparison.

Do you mind sharing those pre-calculated saliency maps as well? It would be great for other research people in future if they need to access them.

Really thanks if it is possible to share.

你的iou loss怎么import进来的？

您好，感谢您的文章。想请教一下SSIM损失中关注边缘的主要原因是因为patch还是因为损失本身呢？换句话说如果用其他损失如DiceLoss，也应用到patch级别会有帮助嘛，谢谢！

Hi Xuebin Qin,
Thank you for your great work. However, I have a problem. When I train your model with batch size =1, it has too low performance that is far from batch size=8. How to train with bach size=1 and still have good performance?
Thank you