This is an unofficial PyTorch re-implementation of paper "Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network" published in ICCV 2019.
License: MIT License
I have trained the model on my own dataset which is the same as IC15, How can i evaluate the model or i want to check the output of the model on other set of images and desplay them ?
@liuch37
i am trying to train using the following example.zip, but i get error, eventhough the labels are 14 points.
(mmocr) home@home-lnx:~/programs/pan-pytorch$ python train.py --batch 1 --epoch 100 --dataset_type ctw --gpu True
Namespace(batch=1, dataset_type='ctw', epoch=100, gpu=True, model='', output='outputs', worker=4)
Random Seed: 408
GPU being used!
Create dataset......
Length of train dataset is: 1
Create model......
Training starts......
Epoch: [1 | 100]
Traceback (most recent call last):
File "train.py", line 183, in <module>
for iter, data in enumerate(train_dataloader):
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in __next__
data = self._next_data()
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data
return self._process_data(data)
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data
data.reraise()
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/_utils.py", line 395, in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/home/programs/pan-pytorch/dataset/ctw1500.py", line 358, in __getitem__
return self.prepare_train_data(index)
File "/home/home/programs/pan-pytorch/dataset/ctw1500.py", line 258, in prepare_train_data
bboxes, words = get_ann(img, gt_path)
File "/home/home/programs/pan-pytorch/dataset/ctw1500.py", line 49, in get_ann
bbox = [np.int(gt[i]) for i in range(4, 32)]
File "/home/home/programs/pan-pytorch/dataset/ctw1500.py", line 49, in <listcomp>
bbox = [np.int(gt[i]) for i in range(4, 32)]
IndexError: list index out of range
Hello, thank you very much for your sharing. Can you share the download link of the data,which is so helpful for me!Thanks again!
Hi there,
i am trying to train the SROIE dataset, so i used the ctw1500 config in training, basically i replaced the ctw1500 images and annotations with the SROIE images and annotations, but i am getting an error when trying to train:
(mmocr) home@home-lnx:~/programs/pan-pytorch$ python train.py --batch 4 --epoch 100 --dataset_type ctw --gpu True
Namespace(batch=4, dataset_type='ctw', epoch=100, gpu=True, model='', output='outputs', worker=4)
Random Seed: 5153
GPU being used!
Create dataset......
Length of train dataset is: 525
Create model......
Training starts......
Epoch: [1 | 100]
Traceback (most recent call last):
File "train.py", line 183, in <module>
for iter, data in enumerate(train_dataloader):
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in __next__
data = self._next_data()
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data
return self._process_data(data)
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data
data.reraise()
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/_utils.py", line 395, in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/home/programs/pan-pytorch/dataset/ctw1500.py", line 358, in __getitem__
return self.prepare_train_data(index)
File "/home/home/programs/pan-pytorch/dataset/ctw1500.py", line 258, in prepare_train_data
bboxes, words = get_ann(img, gt_path)
File "/home/home/programs/pan-pytorch/dataset/ctw1500.py", line 49, in get_ann
bbox = [np.int(gt[i]) for i in range(4, 32)]
File "/home/home/programs/pan-pytorch/dataset/ctw1500.py", line 49, in <listcomp>
bbox = [np.int(gt[i]) for i in range(4, 32)]
IndexError: list index out of range
I have used backbone ResNet 152 for training with neck_channel = (256, 512, 1024, 2048) with single GPU Tesla T4. I have used pretrained model taken from here"https://download.pytorch.org/models/resnet152-394f9c45.pth"
Loss is not reducing as epoch increases. Could you tell me if anything i missed?
training started with this command "python train.py --batch 6 --epoch 10000 --dataset_type tt --gpu True"
here below i am giving last 3 epochs logs
Epoch: [313 | 10000]
(1/209) LR: 0.000972 | Loss: 0.110 | Loss (text/kernel/emb): 0.063/0.037/0.010 | IoU (text/kernel): 0.905/0.802
(21/209) LR: 0.000972 | Loss: 0.127 | Loss (text/kernel/emb): 0.071/0.042/0.015 | IoU (text/kernel): 0.906/0.780
(41/209) LR: 0.000972 | Loss: 0.141 | Loss (text/kernel/emb): 0.081/0.046/0.014 | IoU (text/kernel): 0.901/0.772
(61/209) LR: 0.000972 | Loss: 0.137 | Loss (text/kernel/emb): 0.078/0.045/0.014 | IoU (text/kernel): 0.903/0.772
(81/209) LR: 0.000972 | Loss: 0.136 | Loss (text/kernel/emb): 0.078/0.045/0.013 | IoU (text/kernel): 0.903/0.773
(101/209) LR: 0.000972 | Loss: 0.133 | Loss (text/kernel/emb): 0.076/0.045/0.013 | IoU (text/kernel): 0.904/0.775
(121/209) LR: 0.000972 | Loss: 0.133 | Loss (text/kernel/emb): 0.076/0.045/0.013 | IoU (text/kernel): 0.904/0.774
(141/209) LR: 0.000972 | Loss: 0.134 | Loss (text/kernel/emb): 0.077/0.045/0.013 | IoU (text/kernel): 0.903/0.774
(161/209) LR: 0.000972 | Loss: 0.133 | Loss (text/kernel/emb): 0.076/0.045/0.012 | IoU (text/kernel): 0.904/0.774
(181/209) LR: 0.000972 | Loss: 0.131 | Loss (text/kernel/emb): 0.075/0.045/0.012 | IoU (text/kernel): 0.905/0.775
(201/209) LR: 0.000972 | Loss: 0.130 | Loss (text/kernel/emb): 0.074/0.044/0.012 | IoU (text/kernel): 0.905/0.776
Epoch: [314 | 10000]
(1/209) LR: 0.000972 | Loss: 0.104 | Loss (text/kernel/emb): 0.059/0.040/0.005 | IoU (text/kernel): 0.916/0.781
(21/209) LR: 0.000972 | Loss: 0.131 | Loss (text/kernel/emb): 0.077/0.044/0.009 | IoU (text/kernel): 0.903/0.769
(41/209) LR: 0.000972 | Loss: 0.129 | Loss (text/kernel/emb): 0.075/0.045/0.009 | IoU (text/kernel): 0.906/0.774
(61/209) LR: 0.000972 | Loss: 0.129 | Loss (text/kernel/emb): 0.074/0.045/0.010 | IoU (text/kernel): 0.907/0.775
(81/209) LR: 0.000972 | Loss: 0.126 | Loss (text/kernel/emb): 0.071/0.044/0.010 | IoU (text/kernel): 0.908/0.777
(101/209) LR: 0.000972 | Loss: 0.128 | Loss (text/kernel/emb): 0.073/0.045/0.010 | IoU (text/kernel): 0.908/0.775
(121/209) LR: 0.000972 | Loss: 0.126 | Loss (text/kernel/emb): 0.071/0.044/0.010 | IoU (text/kernel): 0.908/0.777
(141/209) LR: 0.000972 | Loss: 0.127 | Loss (text/kernel/emb): 0.072/0.045/0.010 | IoU (text/kernel): 0.908/0.776
(161/209) LR: 0.000972 | Loss: 0.127 | Loss (text/kernel/emb): 0.072/0.044/0.010 | IoU (text/kernel): 0.907/0.776
(181/209) LR: 0.000972 | Loss: 0.129 | Loss (text/kernel/emb): 0.074/0.045/0.010 | IoU (text/kernel): 0.907/0.775
(201/209) LR: 0.000972 | Loss: 0.129 | Loss (text/kernel/emb): 0.074/0.045/0.010 | IoU (text/kernel): 0.906/0.775
Epoch: [315 | 10000]
(1/209) LR: 0.000972 | Loss: 0.102 | Loss (text/kernel/emb): 0.053/0.039/0.011 | IoU (text/kernel): 0.920/0.808
(21/209) LR: 0.000972 | Loss: 0.157 | Loss (text/kernel/emb): 0.092/0.052/0.013 | IoU (text/kernel): 0.900/0.771
(41/209) LR: 0.000972 | Loss: 0.147 | Loss (text/kernel/emb): 0.084/0.049/0.014 | IoU (text/kernel): 0.901/0.773
(61/209) LR: 0.000972 | Loss: 0.137 | Loss (text/kernel/emb): 0.078/0.046/0.013 | IoU (text/kernel): 0.905/0.779
(81/209) LR: 0.000972 | Loss: 0.136 | Loss (text/kernel/emb): 0.077/0.046/0.013 | IoU (text/kernel): 0.905/0.776
(101/209) LR: 0.000972 | Loss: 0.134 | Loss (text/kernel/emb): 0.076/0.045/0.013 | IoU (text/kernel): 0.906/0.776
(121/209) LR: 0.000972 | Loss: 0.134 | Loss (text/kernel/emb): 0.076/0.046/0.012 | IoU (text/kernel): 0.905/0.775
(141/209) LR: 0.000972 | Loss: 0.134 | Loss (text/kernel/emb): 0.076/0.046/0.012 | IoU (text/kernel): 0.905/0.775
(161/209) LR: 0.000972 | Loss: 0.133 | Loss (text/kernel/emb): 0.076/0.045/0.012 | IoU (text/kernel): 0.905/0.775
(181/209) LR: 0.000972 | Loss: 0.134 | Loss (text/kernel/emb): 0.076/0.046/0.012 | IoU (text/kernel): 0.905/0.775
(201/209) LR: 0.000972 | Loss: 0.133 | Loss (text/kernel/emb): 0.076/0.045/0.012 | IoU (text/kernel): 0.905/0.775
.
it's a good code ,but i wonder if it can show the text recognition in result?
hello, does it have a pre-trained model or always starts from scratch ?
How can we see the testing results and visualization results?
Hello,
When running inference.py i get error:
(mmocr) home@home-lnx:~/programs/pan-pytorch$ python inference.py --input ./mytest/ --output ./results --model ./outputs/model_epoch_20.pth --bbox_type poly
Namespace(bbox_type='poly', gpu=False, input='./mytest/', model='./outputs/model_epoch_20.pth', output='./results', worker=4)
CPU being used!
Create dataset......
Length of test dataset is: 1
Create model......
Prediction on testset......
Testing 0/1
/home/home/anaconda3/envs/mmocr/lib/python3.7/site-packages/torch/nn/functional.py:2973: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
Traceback (most recent call last):
File "inference.py", line 114, in <module>
det_res = get_results(det_out, data['img_metas'], min_area, min_score, bbox_type)
File "/home/home/programs/pan-pytorch/utils/helper.py", line 55, in get_results
label = cv2.resize(label, (img_size[1], img_size[0]), interpolation=cv2.INTER_NEAREST)
cv2.error: OpenCV(4.5.2) :-1: error: (-5:Bad argument) in function 'resize'
> Overload resolution failed:
> - Can't parse 'dsize'. Sequence item with index 0 has a wrong type
> - Can't parse 'dsize'. Sequence item with index 0 has a wrong type
(lib) swap@pop-os:~/Downloads/pan-pytorch-master$ python inference.py --input ./data/MSRA-TD500/test --model ./outputs/model_epoch_0.pth --bbox_type poly
Namespace(bbox_type='poly', gpu=False, input='./data/MSRA-TD500/test', model='./outputs/model_epoch_0.pth', output='results', worker=4)
CPU being used!
Create dataset......
Length of test dataset is: 400
Create model......
Prediction on testset......
./data/MSRA-TD500/test/IMG_0507.gt
./data/MSRA-TD500/test/IMG_1732.gt
./data/MSRA-TD500/test/IMG_2106.gt
./data/MSRA-TD500/test/IMG_1722.gt
Traceback (most recent call last):
File "inference.py", line 104, in
for idx, data in enumerate(test_dataloader):
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in next
data = self._next_data()
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data
return self._process_data(data)
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data
data.reraise()
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/_utils.py", line 394, in reraise
raise self.exc_type(msg)
cv2.error: Caught error in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/swap/Downloads/pan-pytorch-master/dataset/testdataset.py", line 79, in getitem
return self.prepare_test_data(index)
File "/home/swap/Downloads/pan-pytorch-master/dataset/testdataset.py", line 58, in prepare_test_data
img = get_img(img_path)
File "/home/swap/Downloads/pan-pytorch-master/dataset/testdataset.py", line 15, in get_img
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
cv2.error: OpenCV(4.5.3) /tmp/pip-req-build-l1r0y34w/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
I have changed the backbone to Resnet101 and started the training,
got the following error
"RuntimeError: Given groups=1, weight of size 128 64 1 1, expected input[8, 256, 160, 160] to have 64 channels, but got 256 channels instead"
Could you tell me the reason for this error?
i try to load the tensorboard to see the graph of training performance (loss)
I am getting message saying
Probable causes:
You haven’t written any data to your event files.
TensorBoard can’t find your event files.
how could i load loss graph and other graphs
runfile('/home/swap/Downloads/pan-pytorch-master/inference.py', args=['--input', './data/MSRA-TD500/test', '--model', './outputs/model_epoch_0.pth', '--bbox_type', 'poly'], wdir='/home/swap/Downloads/pan-pytorch-master')
Namespace(bbox_type='poly', gpu=False, input='./data/MSRA-TD500/test', model='./outputs/model_epoch_0.pth', output='results', worker=4)
CPU being used!
Create dataset......
Length of test dataset is: 400
Create model......
Prediction on testset......
./data/MSRA-TD500/test/IMG_0507.gt
./data/MSRA-TD500/test/IMG_2106.gt
./data/MSRA-TD500/test/IMG_1732.gt
./data/MSRA-TD500/test/IMG_1800.gt
./data/MSRA-TD500/test/IMG_1722.gt
Traceback (most recent call last):
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3331, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 1, in
runfile('/home/swap/Downloads/pan-pytorch-master/inference.py', args=['--input', './data/MSRA-TD500/test', '--model', './outputs/model_epoch_0.pth', '--bbox_type', 'poly'], wdir='/home/swap/Downloads/pan-pytorch-master')
File "/home/swap/.local/share/JetBrains/Toolbox/apps/PyCharm-C/ch-0/202.7660.27/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "/home/swap/.local/share/JetBrains/Toolbox/apps/PyCharm-C/ch-0/202.7660.27/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/swap/Downloads/pan-pytorch-master/inference.py", line 104, in
for idx, data in enumerate(test_dataloader):
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in next
data = self._next_data()
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data
return self._process_data(data)
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data
data.reraise()
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/_utils.py", line 394, in reraise
raise self.exc_type(msg)
cv2.error: Caught error in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/swap/miniconda3/envs/lib/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/swap/Downloads/pan-pytorch-master/dataset/testdataset.py", line 79, in getitem
return self.prepare_test_data(index)
File "/home/swap/Downloads/pan-pytorch-master/dataset/testdataset.py", line 58, in prepare_test_data
img = get_img(img_path)
File "/home/swap/Downloads/pan-pytorch-master/dataset/testdataset.py", line 15, in get_img
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
cv2.error: OpenCV(4.5.3) /tmp/pip-req-build-l1r0y34w/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
./data/MSRA-TD500/test/IMG_0666.gt
./data/MSRA-TD500/test/IMG_2177.gt
hi there
did you upload the pretrained model for ctw1500
Resnet 101 training is working now, but i am getting following error, but still training is happening
Below is the error trace. Could you tell me what is the issue ?
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=383 error=11 : invalid argument
/root/.virtualenvs/panplus_env/lib/python3.6/site-packages/torch/nn/functional.py:2539: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
(1/209) LR: 0.001000 | Loss: 1.758 | Loss (text/kernel/emb): 0.540/0.262/0.955 | IoU (text/kernel): 0.195/0.283
If i stop the training, how can i restart the training?
I have completed the training using the totaltext. I want to evaluate the model to find the Prediction, Recall, F Score.
I have noticed the file "Deteval.py". But I don't know how to use it and what are all the argument to be passed.
could you suggest me?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.