swhl / rapidvideocr Goto Github PK

使用项目中的demo.py
但是打印：| WARNING | rapid_videocr.main:110 - Extracting frames from tests/test_files/tiny/2.mp4 is 0, skip
换了其他视频也是 0，skip

Traceback (most recent call last):
File "D:\Programs\RapidVideOCR\rapid_ocr\text_detect.py", line 23, in
from .utils_det import (DBPostProcess, check_and_read_gif,
File "D:\Programs\RapidVideOCR\rapid_ocr\utils_det.py", line 27, in
from shapely.geometry import Polygon
File "D:\Programs\anaconda3\envs\videocr\lib\site-packages\shapely\geometry_init_.py", line 4, in
from .base import CAP_STYLE, JOIN_STYLE
File "D:\Programs\anaconda3\envs\videocr\lib\site-packages\shapely\geometry\base.py", line 19, in
from shapely.coords import CoordinateSequence
File "D:\Programs\anaconda3\envs\videocr\lib\site-packages\shapely\coords.py", line 8, in
from shapely.geos import lgeos
File "D:\Programs\anaconda3\envs\videocr\lib\site-packages\shapely\geos.py", line 154, in
lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll'))
File "D:\Programs\anaconda3\envs\videocr\lib\ctypes_init.py", line 364, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] 找不到指定的模块。

When I run the command

Looking in indexes: https://pypi.douban.com/simple/ Collecting rapidocr==1.0.0 Using cached https://github.com/RapidAI/RapidOCR/raw/main/release/python_sdk/sdk_rapidocr_v1.0.0/rapidocr-1.0.0-py3-none-any.whl (18 kB) ERROR: Could not find a version that satisfies the requirement onnxruntime>=1.7.0 (from rapidocr) (from versions: none) ERROR: No matching distribution found for onnxruntime>=1.7.0

However I have onnxruntime located here

/opt/homebrew/Cellar/onnxruntime/1.9.1: 77 files, 11.9MB

Hi there,
your software is very interesting, thanks for your voluntary work on it.

We wanna suggest you to build a Google-Colab notebook version in order to let users to try it online.

Check out (and, whly not, ask for help/collaboration) @oliverfei's videocr-PaddleOCR which forked @apm1467's videocr too.

Hope that inspires !

RapidOCR安装方式貌似有变化，必须要clone整个repo进行安装

• 当多次选择图像目录时，需要重新选择目录，不太方便，改为上次选择的默认值

填入两个路径，点击ok闪退了，我的图像路径里面是不是只能有图片，不能有其他的文件存在？而且我没看到有那个视频的解析呢

Running with concat recognition.
OCR: 34%|████████████████████████████████████████████████████████████████████▌ | 10/29 [00:12<00:22, 1.20s/it]
Traceback (most recent call last):
File "d:\RapidVideOCR-2.1.1\RapidVideOCR-2.1.1\demo.py", line 11, in
extractor(rgb_dir, save_dir)
File "d:\RapidVideOCR-2.1.1\RapidVideOCR-2.1.1\rapid_videocr\rapid_videocr.py", line 64, in call
srt_result, txt_result = self.concat_rec(img_list, is_txt_dir)
File "d:\RapidVideOCR-2.1.1\RapidVideOCR-2.1.1\rapid_videocr\rapid_videocr.py", line 107, in concat_rec
srt_part, txt_part = self.get_match_results(start_i,
File "d:\RapidVideOCR-2.1.1\RapidVideOCR-2.1.1\rapid_videocr\rapid_videocr.py", line 153, in get_match_results
matched_path = img_paths[matched_index]
TypeError: list indices must be integers or slices, not list

直接用的VSF切的RGB图片，单图识别正常

problem incomprehensible characters in srt file
https://pastebin.com/uXvd48pT

感谢作者！
其他情况下都很不错 但是字幕只有单字的情况下大概率识别不出来？
VSF 是有识别到并且输出了RGB图片的

考虑：校对的话可能找不到哪里漏轴

我测试了一下直接使用图片路径传递图片给RapidOcr如果图片两端空白太多，会识别成空。
如果用padding_img 处理一下图片用cv2.copyMakeBorder给图片添加相框，确实能识别出来但是有时候会把一行分成多段，没有直接传图片路径准确。

请提供下述完整信息以便快速定位问题
(Please provide the following information to quickly locate the problem)

• 系统环境/System Environment: windowns
• 使用的是哪门语言的程序/Which programing language: python 3.10
• 所使用语言相关版本信息/Version: latest
• OnnxRuntime版本/OnnxRuntime Version: latest
• 使用当前库的版本/Use version: latest
• 可复现问题的demo和文件/Demo of reproducible problems:
• 完整报错/Complete Error Message:
  CR: 0%| | 0/303 [00:02<?, ?it/s]
  Traceback (most recent call last):
  File "C:\Users\X\Desktop\rapidOCR\RapidVideOCR\onlyOCR.py", line 10, in
  extractor(rgb_dir, save_dir, save_name=save_name)
  File "C:\Users\X\Desktop\rapidOCR\venv\lib\site-packages\rapid_videocr\rapid_videocr.py", line 85, in call
  srt_result, txt_result = self.single_rec(img_list)
  File "C:\Users\X\Desktop\rapidOCR\venv\lib\site-packages\rapid_videocr\rapid_videocr.py", line 103, in single_rec
  dt_boxes, rec_res = self.run_ocr(img)
  File "C:\Users\X\Desktop\rapidOCR\venv\lib\site-packages\rapid_videocr\rapid_videocr.py", line 224, in run_ocr
  ocr_result, _ = self.rapid_ocr(img)
  File "C:\Users\X\Desktop\rapidOCR\venv\lib\site-packages\rapidocr_onnxruntime\rapid_ocr_api.py", line 91, in call
  rec_res, rec_elapse = self.text_recognizer(img_crop_list)
  File "C:\Users\X\Desktop\rapidOCR\venv\lib\site-packages\rapidocr_onnxruntime\ch_ppocr_v3_rec\text_recognize.py", line 71, in call
  rec_result = self.postprocess_op(preds)
  File "C:\Users\X\Desktop\rapidOCR\venv\lib\site-packages\rapidocr_onnxruntime\ch_ppocr_v3_rec\utils.py", line 36, in call
  text = self.decode(preds_idx, preds_prob, is_remove_duplicate=True)
  File "C:\Users\X\Desktop\rapidOCR\venv\lib\site-packages\rapidocr_onnxruntime\ch_ppocr_v3_rec\utils.py", line 68, in decode
  char_list.append(self.character[int(text_index[batch_idx][idx])])
  IndexError: list index out of range
• 可能的解决方案/Possible solutions:
  

用VideoSubFinder生成的图片超过一小时的部分命名格式如图

识图处理后生成的字幕时间轴变成了十小时

程序在打包前可运行，但打包后报一个和rapid_videoc相关的错误如下：
File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
File "rapid_videocr_init_.py", line 4, in
File "", line 1176, in _find_and_load
File "", line 1147, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
File "rapid_videocr\logger.py", line 34, in
File "rapid_videocr\logger.py", line 20, in get_logger
File "loguru_logger.py", line 865, in add
TypeError: Cannot log to objects of type 'NoneType'

RGBImages.zip

Example result:

0:02:36,720 --> 0:02:37,879
请各单元组

0:02:37,880 --> 0:02:39,599
认真填写工作日志

1. Needs indexes
2. Hours require leading zeroes

Should be:
1
00:02:36,720 --> 00:02:37,879
请各单元组

2
00:02:37,880 --> 00:02:39,599
认真填写工作日志

这是个非常好用的OCR字幕工具，但README.md里优先传入TXTImages进行识别的建议似乎不准确
经过测试TXTImages相比RGBImages的识别方式可以很好地保留字幕中的空格，但它漏轴的地方非常多
相比而言RGBImages识别方式基本不会漏轴

测试对比文本：
RGBImages.txt
TestImages.txt

工作进度：

• 已经在Windows上通过命令行调用成功，并且找到官方CLI参数，详情可参见VideoSubFinder CLI参数解析

“

PS F:\RapidVideOCR-main> python main.py
Loading assets/test_video/2.mp4
Auto set the subtitle height....
Traceback (most recent call last):
File "main.py", line 45, in
is_dilate=is_dilate)
File "F:\RapidVideOCR-main\rapid_videocr\api.py", line 20, in get_subtitles
error_num, output_format, text_det, is_dilate)
File "F:\RapidVideOCR-main\rapid_videocr\video.py", line 48, in init
self.get_subtitle_height()
File "F:\RapidVideOCR-main\rapid_videocr\video.py", line 91, in get_subtitle_height
print(f'The subtitle value: {self.subtitle_height}')
AttributeError: 'Video' object has no attribute 'subtitle_height'

”

樓主您好
我裝完環境執行
python main.py的時候出現了以下錯誤

Traceback (most recent call last):
  File "C:\Users\hongyun\Desktop\ntucool\RapidVideOCR\rapidocr\text_detect.py", line 24, in <module>
    from .utils_det import (DBPostProcess, check_and_read_gif,
  File "C:\Users\hongyun\Desktop\ntucool\RapidVideOCR\rapidocr\utils_det.py", line 27, in <module>  
    from shapely.geometry import Polygon
  File "C:\ProgramData\Miniconda3\envs\ocr\lib\site-packages\shapely\geometry\__init__.py", line 4, in <module>
    from .base import CAP_STYLE, JOIN_STYLE
  File "C:\ProgramData\Miniconda3\envs\ocr\lib\site-packages\shapely\geometry\base.py", line 19, in <module>
    from shapely.coords import CoordinateSequence
  File "C:\ProgramData\Miniconda3\envs\ocr\lib\site-packages\shapely\coords.py", line 8, in <module>
    from shapely.geos import lgeos
  File "C:\ProgramData\Miniconda3\envs\ocr\lib\site-packages\shapely\geos.py", line 154, in <module>
    _lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll'))
  File "C:\ProgramData\Miniconda3\envs\ocr\lib\ctypes\__init__.py", line 364, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] 找不到指定的模組。

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\hongyun\Desktop\ntucool\RapidVideOCR\rapidocr\rapidocr.py", line 16, in <module>
    from .text_detect import TextDetector
  File "C:\Users\hongyun\Desktop\ntucool\RapidVideOCR\rapidocr\text_detect.py", line 28, in <module>
    from utils_det import (DBPostProcess, check_and_read_gif, create_operators,
ModuleNotFoundError: No module named 'utils_det'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".\main.py", line 8, in <module>
    from rapidocr import TextDetector, TextSystem
  File "C:\Users\hongyun\Desktop\ntucool\RapidVideOCR\rapidocr\__init__.py", line 5, in <module>
    from .rapidocr import TextSystem, TextDetector
  File "C:\Users\hongyun\Desktop\ntucool\RapidVideOCR\rapidocr\rapidocr.py", line 19, in <module>
    from text_cls import TextDirectionClassifier
ModuleNotFoundError: No module named 'text_cls'

是哪邊出問題了呢!?

在这里的日文模型：[https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.6/doc/doc_ch/models_list.md。准确率没有那么高。还有其他地方有训练好的准确率更高的模型吗？

This is the error I get in the Terminal.app when I run the code pip install /Users/*******/Downloads/rapidocr-1.0.0-py3-none-any.whl

Processing ./Downloads/rapidocr-1.0.0-py3-none-any.whl
Collecting Shapely==1.7.1
  Using cached Shapely-1.7.1.tar.gz (383 kB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/lf/05rwnwxs5n5f8pj438g9ts780000gn/T/pip-install-a03r8vsg/shapely_6a0c11476e6a42bca8be3537bede5cb2/setup.py'"'"'; __file__='"'"'/private/var/folders/lf/05rwnwxs5n5f8pj438g9ts780000gn/T/pip-install-a03r8vsg/shapely_6a0c11476e6a42bca8be3537bede5cb2/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/lf/05rwnwxs5n5f8pj438g9ts780000gn/T/pip-pip-egg-info-inozfapk
       cwd: /private/var/folders/lf/05rwnwxs5n5f8pj438g9ts780000gn/T/pip-install-a03r8vsg/shapely_6a0c11476e6a42bca8be3537bede5cb2/
  Complete output (12 lines):
  Failed `CDLL(/Library/Frameworks/GEOS.framework/Versions/Current/GEOS)`
  Failed `CDLL(/opt/local/lib/libgeos_c.dylib)`
  Failed `CDLL(/usr/local/lib/libgeos_c.dylib)`
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/private/var/folders/lf/05rwnwxs5n5f8pj438g9ts780000gn/T/pip-install-a03r8vsg/shapely_6a0c11476e6a42bca8be3537bede5cb2/setup.py", line 85, in <module>
      from shapely._buildcfg import geos_version_string, geos_version, \
    File "/private/var/folders/lf/05rwnwxs5n5f8pj438g9ts780000gn/T/pip-install-a03r8vsg/shapely_6a0c11476e6a42bca8be3537bede5cb2/shapely/_buildcfg.py", line 190, in <module>
      lgeos = load_dll('geos_c', fallbacks=alt_paths)
    File "/private/var/folders/lf/05rwnwxs5n5f8pj438g9ts780000gn/T/pip-install-a03r8vsg/shapely_6a0c11476e6a42bca8be3537bede5cb2/shapely/_buildcfg.py", line 162, in load_dll
      raise OSError(
  OSError: Could not find library geos_c or load any of its variants ['/Library/Frameworks/GEOS.framework/Versions/Current/GEOS', '/opt/local/lib/libgeos_c.dylib', '/usr/local/lib/libgeos_c.dylib']
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/42/f3/0e1bc2c4f15e05e30c6b99322b9ddaa2babb3f43bc7df2698efdc1553439/Shapely-1.7.1.tar.gz#sha256=1641724c1055459a7e2b8bbe47ba25bdc89554582e62aec23cb3f3ca25f9b129 (from https://pypi.org/simple/shapely/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement Shapely==1.7.1 (from rapidocr) (from versions: 1.0a7, 1.0b1, 1.0b2, 1.0b3, 1.0b4, 1.0rc1, 1.0rc2, 1.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.0.11, 1.0.12, 1.0.13, 1.0.14, 1.0.15, 1.2b1, 1.2b2, 1.2b3, 1.2b4, 1.2b5, 1.2b6, 1.2b7, 1.2rc1, 1.2rc2, 1.2, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.2.8, 1.2.9, 1.2.10, 1.2.12, 1.2.13, 1.2.14, 1.2.15, 1.2.16, 1.2.17, 1.2.18, 1.2.19, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.5.5, 1.5.6, 1.5.7, 1.5.8, 1.5.9, 1.5.10, 1.5.11, 1.5.12, 1.5.13, 1.5.14, 1.5.15, 1.5.16, 1.5.17, 1.6a1, 1.6a2, 1.6b1, 1.6b2, 1.6b3, 1.6b4, 1.6b5, 1.6.0, 1.6.1, 1.6.2, 1.6.2.post1, 1.6.3, 1.6.4, 1.6.4.post1, 1.6.4.post2, 1.7a1, 1.7a2, 1.7a3, 1.7b1, 1.7.0, 1.7.1, 1.8a1, 1.8a2, 1.8a3, 1.8rc1, 1.8rc2, 1.8.0)
ERROR: No matching distribution found for Shapely==1.7.1

(videocr) D:\Programs\RapidVideOCR>python main.py
D:\Programs\anaconda3\envs\videocr\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:56: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'
"Available providers: '{}'".format(name, ", ".join(available_provider_names)))
Loading C:\Users\YC-\Downloads\从这次疫情看**AI产业的落地问题_哔哩哔哩_bilibili 2022-05-23 20_04.mp4
[15:10:35] D:\a\decord\decord\src\video\video_reader.cc:312: Failed to seek file to position: 0
[15:10:35] D:\a\decord\decord\src\video\video_reader.cc:312: Failed to seek file to position: 0
Traceback (most recent call last):
File "main.py", line 43, in
ocr_result = extractor(mp4_path, time_start, time_end, batch_size)
File "D:\Programs\RapidVideOCR\rapid_videocr\rapid_videocr.py", line 41, in call
self.height = int(self.vr[0].shape[0])
File "D:\Programs\anaconda3\envs\videocr\lib\site-packages\decord\video_reader.py", line 104, in getitem
self.seek_accurate(idx)
File "D:\Programs\anaconda3\envs\videocr\lib\site-packages\decord\video_reader.py", line 234, in seek_accurate
raise RuntimeError("Failed to seek_accurate to frame {}".format(pos))
RuntimeError: Failed to seek_accurate to frame 0

I like the the idea of detecting the subtitle size automatically but this is not working well at the moment. It is problematic to detect the right text if there are other texts like broadcasting logos or show titles and credits visible.
In the following case only the "Kiku" text was recognized as one long subtitle:

This is most likely because it was displayed first and never changed.

It would be nice if you make it possible to choose a region of interest. Every text that is not inside this region would be discarded or you cut out the ROI. I did something similar in the past:

def getROI(videopath):
    # init RegionOfInterest
    ROI = (0, 0, 0, 0)
    # create VideoCapture object
    cap = cv2.VideoCapture(videopath)
    # get the number of frames from the video for the endpoint of the trackbar
    numberofframes = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))

    # since cv2.createTrackbar requires a callbackfunction we use
    # a function that does nothing :)
    def onTrackbarChange(current_frame):
        pass

    # create a window called video and a trackbar called position
    cv2.namedWindow('video')
    cv2.createTrackbar( 'position', 'video', 0, numberofframes, onTrackbarChange)
    # displayOverlay is not working with the pip install of opencv
    # cv2.displayOverlay('video', "Press 'R-Button' to select region of interest", 5000)

    while True:
        # get videoposition from position in video
        videoposition = cv2.getTrackbarPos('position','video')
        #print("Videopos: " + str(videoposition))
        # goto videoposition and read frame
        cap.set(cv2.CAP_PROP_POS_FRAMES,videoposition)
        ret, frame = cap.read()
        # show frame in video if reading was successful
        if ret == True:
            cv2.imshow("video", frame)
        # keep the video open till a key gets pressed
        key = cv2.waitKey(1)

        # break when the s key gets pressed or get ROI when r gets pressed
        if key == ord("s"):
            break
        elif key == ord("r"):
            ROI = cv2.selectROI('video', frame, fromCenter=False, showCrosshair=False)
            # only end the loop if a ROI was choosen
            if ROI != (0, 0, 0, 0):
                break
    # release the video capture object and close all windows
    cap.release()
    cv2.destroyAllWindows()
    return ROI

ROI = getROI(args.video)
ret, frame = cap.read()
frame = frame[int(ROI[1]):int(ROI[1]+ROI[3]), int(ROI[0]):int(ROI[0]+ROI[2])]

Can you please implement a preview window where one can choose a frame of the video and pick a region of interest?
Maybe a "bad word" list, where you enter the broadcasting logo text, would also do.
If the results of the detection and recognition are depending on the size of the image it would also be nice to upscale frames that have a very low resolution.

I uploaded a sample video in case you need one:
https://drive.google.com/file/d/1rptaZGN0yzQ5ZEOuSB-HqGB-CpMqOFSY/view?usp=sharing

Thanks for your work!

2024-08-14 13:03:07,232 - INFO: Extracting subtitle images with VideoSubFinder (takes quite a long time) ...
2024-08-14 13:03:07,232 - INFO: [1/1] Starting to extract C:/Users/win11/Desktop/test.mp4 key frame
2024-08-14 13:03:08,475 - INFO: [1/1] Starting to run C:/Users/win11/Desktop/test.mp4 ocr
2024-08-14 13:03:08,476 - WARNING: Extracting frames from C:/Users/win11/Desktop/test.mp4 is 0, skip

然后就报找不到test.srt字幕，用的VSF，字幕样式如下：

有遇到过类似字幕无法处理的吗？
试过几十个视频了，就这个字幕处理报错哦。。

使用版本

v1.2.8

复现代码

rapid_videocr -i /content/RGBImages -s outputs

存在问题

在终端模式下使用时，不想使用vsf，只传入img_dir时，会报错误：

Traceback (most recent call last):
  File "/usr/local/bin/rapid_videocr", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/dist-packages/rapid_videocr/main.py", line 299, in main
    raise ValueError(
ValueError: --vsf_exe_path or --video_dir must not be None at the same time.

测试样本RGBImages.zip

Not working in google colab, please update it