xhzdeng / crpn Goto Github PK

After run test model on some test images (experiments/scripts/test.sh), got following lines (show 1st 10 lines here):

testimage-001 1.000 69.0 624.3 159.9 624.4
testimage-001 1.000 167.5 624.5 207.0 624.6
testimage-001 1.000 120.6 564.6 172.2 564.7
testimage-001 1.000 64.4 564.7 112.8 564.8
testimage-001 1.000 9.5 624.5 60.9 624.5
testimage-001 1.000 178.0 563.4 224.6 562.2
testimage-001 0.999 348.3 222.6 441.9 222.5
testimage-001 0.999 10.6 443.2 77.4 443.3
testimage-001 0.999 9.6 564.4 56.9 564.4
testimage-001 0.999 9.7 222.7 104.0 222.5
testimage-001 0.999 10.1 504.0 74.1 504.0

How to interpret them? It does not look like a boundbox in standard voc format (i.e. xmin,ymin,xmax,ymax), since y1 and y2 are so close to each other, does not look a valid boundbox.

论文中提到，将ICDAR2015将短边resize到900，ICDAR2013将短边resize到640，这是靠反复实验得到的一个参数吗？或者说有没有什么选择的方法？作者有没有考虑过multi-scale的测试？我换了数据，loss在训练一千次之后，一直到十万次，数值一直是在零点几到二点几之间波动，learning rate已经设置到1*e-5，感觉一直没有收敛，请问作者在之前实验过程中遇到过吗？

[723.29144 542.9097 723.29144 552.94037 723.29144 552.94037
723.29144 542.9097 ]
[270. nan 723.29144 555.44806 270. nan
723.29144 557.9557 ]
[723.29144 560.4634 723.29144 562.97107 723.29144 562.97107
723.29144 560.4634 ]]
Traceback (most recent call last):
File "./tools/train_net.py", line 112, in
max_iters=args.max_iters)
File "/home/zq/crpn/tools/../lib/fast_rcnn/train.py", line 162, in train_net
model_paths = sw.train_model(max_iters)
File "/home/zq/crpn/tools/../lib/fast_rcnn/train.py", line 103, in train_model
self.solver.step(1)
File "/home/zq/crpn/tools/../lib/rpn/labelmap_layer.py", line 66, in forward
x2 = int(round(bbox[2] * spatial_scale))
ValueError: cannot convert float NaN to integer

这样的错误是要删除对应的图片吗,还是修改sort_points中的代码?

为什么bbox_pred 的输出是N x 16,输出的是(x1,y1,x2,y2,x3,y3,x4,y4)x2 吗?还是4 x(x,y,w,h)?

I noticed that you have uploaded a testing model (thanks for that!),
I was just wondering what datasets you trained that model on?
Thanks :)

使用的是提供得测试模型，和data/demo中的图片，最后都没有检测结果，debug发现检测到得bbox置信度都很低(0.1左右)，被nms过滤掉，这可能是什么原因呢 ？

Hi, thanks for sharing this nice work.

I was wondering if the evaluation code (which is modified pascal voc code) is correct for quadrilateral detection. It seems that the below code only uses p1 and p2 but p3 and p4.

Shouldn't all 4 points be used to calculate the intersection over union of two quadrilaterals?

I0423 10:43:13.650262 4174 net.cpp:242] This network produces output loss_rpn_bl
I0423 10:43:13.650265 4174 net.cpp:242] This network produces output loss_rpn_br
I0423 10:43:13.650269 4174 net.cpp:242] This network produces output loss_rpn_tl
I0423 10:43:13.650272 4174 net.cpp:242] This network produces output loss_rpn_tr
I0423 10:43:13.650315 4174 net.cpp:255] Network initialization done.
I0423 10:43:13.650451 4174 solver.cpp:56] Solver scaffolding done.
Loading pretrained model weights from Model/pretrain.caffemodel
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 139667392706304:
#000: H5F.c line 408 in H5Fis_hdf5(): unable open file
major: File accessibilty
minor: Not an HDF5 file
#1: H5Fint.c line 532 in H5F__is_hdf5(): unable to open file
major: Low-level I/O
minor: Unable to initialize object
#2: H5FD.c line 809 in H5FD_open(): open failed
major: Virtual File Layer
minor: Unable to initialize object
#3: H5FDsec2.c line 346 in H5FD_sec2_open(): unable to open file: name = 'Model/pretrain.caffemodel', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
major: File accessibilty
minor: Unable to open file
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 139667392706304:
#000: H5F.c line 586 in H5Fopen(): unable to open file
major: File accessibilty
minor: Unable to open file
#1: H5Fint.c line 1236 in H5F_open(): unable to open file: time = Mon Apr 23 10:43:13 2018
, name = 'Model/pretrain.caffemodel', tent_flags = 0
major: File accessibilty
minor: Unable to open file
#2: H5FD.c line 809 in H5FD_open(): open failed
major: Virtual File Layer
minor: Unable to initialize object
#3: H5FDsec2.c line 346 in H5FD_sec2_open(): unable to open file: name = 'Model/pretrain.caffemodel', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
major: File accessibilty
minor: Unable to open file
F0423 10:43:13.652704 4174 net.cpp:791] Check failed: file_hid >= 0 (-1 vs. 0) Couldn't open Model/pretrain.caffemodel
*** Check failure stack trace: ***
./experiments/scripts/train.sh: line 55: 4174 Aborted (core dumped) ./tools/train_net.py --gpu 0 --solver models/${NET}/solver.pt --weights ${WEIGHTS} --imdb ${TRAIN_IMDB} --iters ${ITERS} --cfg models/${NET}/config.yml ${EXTRA_ARGS}

作者你好,这是什么错误?怎么解决

我用不同的数据集训练，处理数据都是同一的脚本，但是有的数据集训练的时候，会出现rpn loss 为0的情况，有的甚至一直为0，这是为什么，可不可以给我点建议，谢谢大牛
@xhzdeng

论文中的K是什么意思,为什么要用角度除以K

during runtest, occur caffe/layers/log_layer.hpp: No such file or directory

I get this error when test demo.py , running in GTX 1081Ti 12 G

F0416 21:56:18.036826 12284 concat_layer.cpp:42] Check failed: top_shape[j] == bottom[i]->shape(j) (75 vs. 76) All inputs must have the same shape, except at concat_axis.
*** Check failure stack trace: ***
Aborted (core dumped)

下载了您的代码,在运行demo.py的时候出现了这样的问题,怎么解决?

查看了作者读取数据集的函数，发现读取的是四个点八个坐标值，与网上的教程“将ICDAR21015转化为VOC2007”有所区别，那里提及的bndbox是以x,y,w,h,angle的格式制作annotations。我参照作者的代码，将数据集制作为如下格式，不知道与作者用于训练的VOC2007数据集的格式是否有差异？

some infomation,

• ./tools/test_net.py --gpu 0 --def models/vgg16/test.pt --net /home/deep3/work/crpn/models/vgg16/test.caffemodel --imdb voc_2007_test --cfg models/vgg16/config.yml
  Called with args:
  Namespace(caffemodel='/home/deep3/work/crpn/models/vgg16/test.caffemodel', cfg_file='models/vgg16/config.yml', comp_mode=False, gpu_id=0, imdb_name='voc_2007_test', max_per_image=100, prototxt='models/vgg16/test.pt', set_cfgs=None, vis=False, wait=True)

AP for text = 0.0459
Mean AP = 0.0459
Mean REC = 0.1231
Mean PREC = 0.1032

what's wrong?
thanks

Thanks for sharing the paper and repository. Wondering where to download your pretrained model file so that we can try with ICDAR 13 and 15 dataset. A pointer to training dataset will be great too.

Hello, @xhzdeng
I am trying to get F-measure for ICDAR2015 test subset. I change TEST.SCALES to 840 in config.yaml and save detentions to txt from lib/fast_rcnn/test.py just after the line 316 to be able to run ICDAR eval tool. For the provided test.caffemodel I am getting is only 0.8357763975155279.

I've tried scripts in data folder but seems the links are expired.
Then I tried to download dataset from http://host.robots.ox.ac.uk/pascal/VOC/voc2007/, and could not successfully run the code, too.
Please help me.
（也尝试魔改输入数据处理的代码，还是失败了，萌新求助QAQQQ）

CTPN's performance on ICDAR 2013 dataset is F-measure 0.88 (R 0.83, P 0.93), not 0.822 in your paper (table 2 on page 11.) They published test model at https://github.com/tianzhi0549/CTPN (downloadable test model has F 0.86 on ICDAR 2013, which does not include side refinement feature described in the paper.)

CTPN does not handle angled text well, though. Their score on 2015 dataset is quite low.

i am confused with the sentences below
"For each diagonal, select any one corner from last two types and rotate the diagonal until three points (two endpoints and the third one) are collinear, then a quadrilateral proposal determined by those two diagonals will be obtained."
how could three conners determine a quadrilateral proposal? while rotating the diagonal, where is the fixed point?

您好，我按照您论文中说的，在synth text pretrain的model基础上，在icdar13 和icdar15的training dataset上training了20000步，lr是0.0001, 但是结果只有78%， 麻烦问一下这是什么问题呢？是什么参数没设对吗？@xhzdeng

使用SynthText生成训练集，但生成的结果是保存在SynthText.h5文件中的，如果要用生成的训练集来训练crpn，改怎么把生成的.h5转化为crpn需要的VOC2007的数据呢？

Hello @xhzdeng, I am trying to run your code. But when I run
cd $CRPN_ROOT/caffe-fast-rcnn
make -j8 && make pycaffe
I got the following error
Makefile:6: *** Makefile.config not found. See Makefile.config.example.. Stop.

Should I download more files?

Check failed: registry.count(type) == 1 (0 vs. 1) Unknown layer type: Python (known types: AbsVal, Accuracy, ArgMax........
编译成功后运行demo.py, 出现这个问题，网上说是缺少头文件问题，难道是作者少上传文件了吗？

谢谢分享论文和代码，新人小白想问问关于ICDAR2013和ICDAR2015数据库数据局格式处理的代码是否会发布，谢谢。

CXX .build_release/src/caffe/proto/caffe.pb.cc
CXX src/caffe/util/upgrade_proto.cpp
CXX src/caffe/util/db_leveldb.cpp
CXX src/caffe/util/math_functions.cpp
CXX src/caffe/util/insert_splits.cpp
CXX src/caffe/util/signal_handler.cpp
CXX src/caffe/util/im2col.cpp
CXX src/caffe/util/io.cpp
src/caffe/util/math_functions.cpp:1:49: fatal error: boost/math/special_function                                                                             s/next.hpp: No such file or directory
compilation terminated.
src/caffe/util/signal_handler.cpp:1:26: fatal error: boost/bind.hpp: No such fil                                                                             e or directory
compilation terminated.
Makefile:579: recipe for target '.build_release/src/caffe/util/math_functions.o'                                                                              failed
make: *** [.build_release/src/caffe/util/math_functions.o] Error 1
make: *** Waiting for unfinished jobs....
Makefile:579: recipe for target '.build_release/src/caffe/util/signal_handler.o'                                                                              failed
make: *** [.build_release/src/caffe/util/signal_handler.o] Error 1
In file included from src/caffe/util/im2col.cpp:4:0:
./include/caffe/util/math_functions.hpp:7:26: fatal error: glog/logging.h: No su                                                                             ch file or directory
compilation terminated.
Makefile:579: recipe for target '.build_release/src/caffe/util/im2col.o' failed
make: *** [.build_release/src/caffe/util/im2col.o] Error 1
In file included from src/caffe/util/db_leveldb.cpp:2:0:
./include/caffe/util/db_leveldb.hpp:7:24: fatal error: leveldb/db.h: No such fil                                                                             e or directory
compilation terminated.
Makefile:579: recipe for target '.build_release/src/caffe/util/db_leveldb.o' fai                                                                             led
make: *** [.build_release/src/caffe/util/db_leveldb.o] Error 1
In file included from .build_release/src/caffe/proto/caffe.pb.cc:5:0:
.build_release/src/caffe/proto/caffe.pb.h:12:2: error: #error This file was gene                                                                             rated by a newer version of protoc which is
 #error This file was generated by a newer version of protoc which is
  ^
.build_release/src/caffe/proto/caffe.pb.h:13:2: error: #error incompatible with                                                                              your Protocol Buffer headers. Please update
 #error incompatible with your Protocol Buffer headers.  Please update
  ^
.build_release/src/caffe/proto/caffe.pb.h:14:2: error: #error your headers.
 #error your headers.
  ^
In file included from .build_release/src/caffe/proto/caffe.pb.cc:5:0:
.build_release/src/caffe/proto/caffe.pb.h:23:35: fatal error: google/protobuf/ar                                                                             ena.h: No such file or directory
compilation terminated.
Makefile:586: recipe for target '.build_release/src/caffe/proto/caffe.pb.o' fail                                                                             ed
make: *** [.build_release/src/caffe/proto/caffe.pb.o] Error 1
In file included from src/caffe/util/upgrade_proto.cpp:8:0:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file                                                                              or directory
compilation terminated.
Makefile:579: recipe for target '.build_release/src/caffe/util/upgrade_proto.o'                                                                              failed
make: *** [.build_release/src/caffe/util/upgrade_proto.o] Error 1
In file included from src/caffe/util/insert_splits.cpp:7:0:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file                                                                              or directory
compilation terminated.
Makefile:579: recipe for target '.build_release/src/caffe/util/insert_splits.o'                                                                              failed
make: *** [.build_release/src/caffe/util/insert_splits.o] Error 1
In file included from src/caffe/util/io.cpp:18:0:
./include/caffe/common.hpp:4:32: fatal error: boost/shared_ptr.hpp: No such file                                                                              or directory
compilation terminated.
Makefile:579: recipe for target '.build_release/src/caffe/util/io.o' failed
make: *** [.build_release/src/caffe/util/io.o] Error 1

my python version is 3.6,and the protobuf is 3.2