Comments (2)
zchrissirhcz
commented on July 28, 2024
11
Thanks for your reply. Now I can get correct evaluation result. I'll put my steps here.
1.use RCF's caffemodel to genereate png files
Now I use the downloaded rcf_pretrained_bsds.caffemodel to generated multi-scaled results, saved as *_fuse.png.
2. do nms
I use your provides edge_nms.m script, and find it cannot use edgesNms funtion, thus I move it to Piotr Dollar's edges folder, i.e. /opt/work/piotr/edges, and change this line:
edge = edgesNms(edge, O, 2, 5, 1.01, 8);to
edge = edgesNmsMex(edge, O, 2, 5, 1.01, 8);Then run it to generate nms-ed results.
Note, I've also downloaded Piotr Dollar's toolbox and both these two folders(and children folders) are added to path.
3. use BSDS benchmark scripts for boundary evaluation
I downloaded http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/BSR/BSR_full.tgz, and by editing bench_bsds500.m to these lines:
addpath benchmarks
clear all;close all;clc;
imgDir = '../BSDS500/data/images/test';
gtDir = '../BSDS500/data/groundTruth/test';
%inDir = '../BSDS500/ucm2/test';
%outDir = '../BSDS500/ucm2/test_eval';
inDir = '/opt/data/HED-BSDS/test-fcn-nms';
outDir = '/opt/data/HED-BSDS/RCF-eval';
%mkdir(outDir);
% running all the benchmarks can take several hours.
tic;
%allBench(imgDir, gtDir, inDir, outDir)
boundaryBench(imgDir, gtDir, inDir, outDir);
toc;
plot_eval(outDir);Then I run that script, and after a long time(several hours) waiting, some .txt files are generated, including eval_bdry.txt and eval_bdry_thr.txt.
I thought eval_bdry.txt and eval_bdry_thr.txt can be put under your provided plotting code folder, and I run https://github.com/yun-liu/plot-edge-pr-curves/blob/master/plot_bsds.m.
However, errors just occurs. Notice that there are exactly 7 numbers (7 columns) in *_bdry.txt in plot-edge-pr-curves/edge_eval_bsds, and in BSR/bench/benchmarks/collect_eval_bdry.m, there is:
fname = fullfile(pbDir,'eval_bdry.txt');
fid = fopen(fname,'w');
if fid==-1,
error('Could not open file %s for writing.',fname);
end
fprintf(fid,'%10g %10g %10g %10g %10g %10g %10g %10g\n',bestT,bestR,bestP,bestF,R_max,P_max,F_max,Area_PR);
fclose(fid);i.e. The BSDS boundary benchmark will generate eval_bdry.txt with exactly 8 numbers, instead of 7.
Similarly, my generated eval_bdry_thrs.txt have 4 columns, and your provides *_bdry_thr.txt have 3 columns.
4. removing the first columns in eval_bdry.txt and eval_bdry_thr.txt
This is claimed in https://github.com/yun-liu/plot-edge-pr-curves/blob/master/README.md
5. do plot
modify and run plot-edge-pr-curves/plot_bdsd.m
Now I get a nearly exactly same result:
ODS=0.806 OIS=0.823 AP=0.839 R50=0.914 - RCF (2017)
ODS=0.811 OIS=0.830 AP=0.846 R50=0.913 - RCF-MS (2017)
ODS=0.811 OIS=0.831 AP=0.847 R50=0.916 - MyRCF (2018)
MyRCF_bdry.txt:
0.796388 0.826376 0.811105 0.823153 0.838022 0.830521 0.846884
MyRCF_bdry_thr.txt:
0.916885 0.481511 0.631423
0.913225 0.533613 0.673619
0.911576 0.552105 0.687699
0.909399 0.571562 0.701947
0.908081 0.581843 0.709245
0.906144 0.59488 0.718239
0.904932 0.602573 0.72343
0.903104 0.612941 0.730254
0.901829 0.619326 0.734345
0.89963 0.628464 0.739987
0.897387 0.637358 0.745345
0.89596 0.642711 0.748495
0.893796 0.6503 0.752849
0.89235 0.654891 0.755399
0.890059 0.661843 0.759171
0.888551 0.666303 0.761543
0.886357 0.672549 0.764791
0.884839 0.676687 0.76689
0.882561 0.682873 0.769981
0.881107 0.686943 0.772004
0.878687 0.693304 0.775065
0.876151 0.699512 0.777931
0.874373 0.703607 0.77975
0.871617 0.709732 0.782388
0.869685 0.713748 0.784038
0.866786 0.719888 0.786536
0.864841 0.723922 0.788132
0.8619 0.729702 0.790311
0.859778 0.73352 0.791646
0.856537 0.739282 0.793602
0.853173 0.744997 0.795424
0.851031 0.74876 0.796626
0.847647 0.754523 0.798379
0.845409 0.758379 0.799533
0.842058 0.763776 0.801009
0.839689 0.767495 0.801971
0.836156 0.773108 0.803397
0.833805 0.776892 0.804343
0.830269 0.782603 0.805732
0.827745 0.786396 0.806541
0.823914 0.792099 0.807694
0.819892 0.797895 0.808744
0.817159 0.801629 0.80932
0.812873 0.807348 0.810101
0.809865 0.811157 0.81051
0.80516 0.816768 0.810922
0.801654 0.820633 0.811032
0.796388 0.826376 0.811105
0.792854 0.830137 0.811067
0.787342 0.835859 0.810875
0.781505 0.841581 0.810432
0.777526 0.845342 0.810017
0.771536 0.850996 0.809321
0.767256 0.854771 0.808653
0.760605 0.860382 0.807422
0.755853 0.864141 0.806378
0.748499 0.86977 0.80459
0.743674 0.873558 0.8034
0.736299 0.879468 0.801541
0.731124 0.883238 0.800014
0.722958 0.888869 0.797375
0.714331 0.894332 0.794261
0.70829 0.897902 0.791904
0.698607 0.902929 0.787735
0.692028 0.906222 0.784772
0.681991 0.911481 0.78021
0.674913 0.914805 0.776759
0.66391 0.91985 0.7712
0.656116 0.923474 0.767169
0.643958 0.928576 0.76051
0.630972 0.933331 0.75293
0.621734 0.936501 0.747326
0.607383 0.941441 0.738386
0.597245 0.944526 0.731773
0.581938 0.948922 0.721443
0.570807 0.951668 0.713599
0.553205 0.955427 0.700697
0.540565 0.958061 0.691159
0.51997 0.961771 0.675006
0.505487 0.964148 0.663245
0.482445 0.967519 0.643843
0.458149 0.970794 0.622514
0.441003 0.972765 0.606878
0.413966 0.976042 0.581361
0.394876 0.977813 0.562568
0.364598 0.981287 0.531658
0.34286 0.983228 0.508428
0.30815 0.986119 0.469567
0.283816 0.988241 0.440984
0.244985 0.991295 0.392876
0.20395 0.993577 0.338431
0.176363 0.994982 0.299618
0.134409 0.996868 0.236879
0.106832 0.998216 0.193007
0.0671202 0.999695 0.125794
0.0440549 0.999961 0.0843918
0.0173671 1 0.0341413
0.00601356 1 0.0119552
0.000150339 1 0.000300633
from rcf.
yun-liu
commented on July 28, 2024
@zchrissirhcz I guess that you have used the wrong wrappers. You should use the function of boundaryBench in the BSDS500 benchmark to evaluate the produced edges. Moreover, the edge thinning has been performed in the function of evaluation_bdry_image using bwmorph function in matlab.
from rcf.
Related Issues (20)
- Questions about relaxed deep supervison HOT 6
- Multicue DataSet HOT 3
- GPU版本的sigmoid_cross_entropy_loss_layer HOT 4
- 关于edge_nms.m运行问题 HOT 4
- How to evaluate the NYUD? HOT 2
- pretrain model无法下载 HOT 1
- 定位公差maxDist HOT 2
- PASCAL VOC边缘集是怎么生成的
- NYUV2数据集获取 HOT 2
- 关于整体分割过程的问题
- 求pytorch版本的预训练ResNet模型
- 求pytorch版本的Resnet预训练模型
- Multicue数据集MaxDist HOT 3
- 关于matlab画精确率-召回率图的问题
- 您好，我想问一下，任意尺寸图片输入后会统一调整为多大？ HOT 4
- 数据集 HOT 2
- 输出图片都是黑白的 HOT 4
- 使用pytorch版本跑NYUD数据集时需要更改那些地方的代码呢
- How to plot the curve on Multicue dataset HOT 1
- 关于轮廓细节信息的影响 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rcf.