Comments (8)
@mmokrejs sorry for the delay, thank you for the samples I did receive them. I'll leave this open for now and re-word it as an investigation into improved logging on frame corruption. I haven't had any time to experiment with the videos you sent yet, but will let you know if I can come up with any improvements. In the meantime happy to accept any PRs if folks have similar fixes.
Many thanks for the test material.
from dvr-scan.
If you are able, can you test running the video with another related project as suggested in the linked issue? It entails running PySceneDetect on the failing video with the following commands:
scenedetect -i video.mp4 --verbosity debug --backend opencv detect-content
scenedetect -i video.mp4 --verbosity debug --backend pyav detect-content
I would like to know if using a different library to handle video IO would resolve the situation, or at least be able to return an error that can be acted on. Will also try this once I have a sample, sent you an email just now. I might be missing some debug logging from PySceneDetect so will look into making sure that gets logged in DVR-Scan.
With OpenCV we try to read at least 5 more frames after a failure to decode (see https://github.com/Breakthrough/PySceneDetect/blob/main/scenedetect/backends/opencv.py#L282). Unfortunately there is no error that gets returned from OpenCV in the case of a decode failure.
Would be nice if the List of motion events: table contained also the frame starts and ends, so that one could see if the input was processed up to the very last frame or not.
As long as you are not seeking in the input, you should be able to tell how many frames were processed from the log output:
[DVR-Scan] Processed 387014 frames read in 4459.8 secs (avg 86.8 FPS).
Thank you for the detailed report!
from dvr-scan.
Here I ran the command on just the problematic 51MB file (unlike merged HEVC file for whole day):
$ scenedetect -i input.3gp --verbosity debug --backend opencv detect-content
INFO: context.handle_options(): PySceneDetect 0.6.3
DEBUG: context.handle_options(): User config file not found.
DEBUG: context.handle_options(): Parsing program options.
DEBUG: __init__.open_video(): Opening video with opencv...
DEBUG: context._open_video_stream(): Video opened using backend VideoStreamCv2
DEBUG: context.handle_options(): Initializing SceneManager.
DEBUG: __init__.detect_content_command(): Adding detector: ContentDetector({'weights': Components(delta_hue=1.0, delta_sat=1.0, delta_lum=1.0, delta_edges=0.0), 'kernel_size': None, 'luma_only': False, 'min_scene_len': 10, 'threshold': 27.0})
DEBUG: controller.run_scenedetect(): Running controller.
INFO: scene_manager.detect_scenes(): Downscale factor set to 10, effective resolution: 256 x 192
INFO: scene_manager.detect_scenes(): Detecting scenes...
Detected: 0 | Progress: 37%|█████████████████████████████████████████████████████▍ | 295/801 [00:02<00:04, 111.83frames/s][hevc @ 0x5641b3d58d80] Duplicate POC in a sequence: 13.
DEBUG: opencv.read(): Frame failed to decode.
Detected: 0 | Progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 800/801 [00:07<00:00, 103.77frames/s]
INFO: controller._detect(): Processed 800 frames in 7.7 seconds (average 103.76 FPS).
INFO: controller.run_scenedetect(): Detected 1 scenes, average shot length 46.9 seconds.
$ scenedetect -i input.3gp --verbosity debug --backend pyav detect-content
INFO: context.handle_options(): PySceneDetect 0.6.3
DEBUG: context.handle_options(): User config file not found.
DEBUG: context.handle_options(): Parsing program options.
Usage: scenedetect [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...
Try 'scenedetect -h' for help.
Error: Invalid value for -b/--backend: Specified backend pyav is not available on this system!
$
Hmm, I have dvr-scan
and scenedetect from pip
. I have opencv on Gentoo system-wide.
OK, I installed https://pypi.org/project/av/ into same environemnt like dvr-scan and scenedetect.
$ pip install av
Collecting av
Downloading av-12.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.6 kB)
Downloading av-12.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 34.7/34.7 MB 5.9 MB/s eta 0:00:00
Installing collected packages: av
Successfully installed av-12.0.0
$
Now I have:
$ scenedetect -i input.3gp --verbosity debug --backend pyav detect-content
INFO: context.handle_options(): PySceneDetect 0.6.3
DEBUG: context.handle_options(): User config file not found.
DEBUG: context.handle_options(): Parsing program options.
DEBUG: __init__.open_video(): Opening video with pyav...
DEBUG: pyav.__init__(): Restoring default ffmpeg log callbacks.
DEBUG: pyav.__init__(): Threading mode set: AUTO
DEBUG: context._open_video_stream(): Video opened using backend VideoStreamAv
DEBUG: context.handle_options(): Initializing SceneManager.
DEBUG: __init__.detect_content_command(): Adding detector: ContentDetector({'weights': Components(delta_hue=1.0, delta_sat=1.0, delta_lum=1.0, delta_edges=0.0), 'kernel_size': None, 'luma_only': False, 'min_scene_len': 15, 'threshold': 27.0})
DEBUG: controller.run_scenedetect(): Running controller.
INFO: scene_manager.detect_scenes(): Downscale factor set to 10, effective resolution: 256 x 192
INFO: scene_manager.detect_scenes(): Detecting scenes...
Detected: 0 | Progress: 0%| | 0/801 [00:00<?, ?frames/s][swscaler @ 0x7f631cd76f80] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdae880] deprecated pixel format used, make sure you did set range correctly
Detected: 0 | Progress: 0%|▏ | 1/801 [00:00<01:45, 7.56frames/s][swscaler @ 0x7f631cd87800] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cd947c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdd2840] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cd87800] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cd947c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdae880] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cd87800] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cd947c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdae880] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdbb840] deprecated pixel format used, make sure you did set range correctly
Detected: 0 | Progress: 1%|██▌ | 12/801 [00:00<00:13, 59.30frames/s][swscaler @ 0x7f631cdae880] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdbb840] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdae880] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdbb840] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdae880] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdbb840] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdae880] deprecated pixel format used, make sure you did set range correctly
Detected: 0 | Progress: 2%|███▉ | 19/801 [00:00<00:13, 59.98frames/s][swscaler @ 0x7f631cdbb840] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdae880] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdbb840] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdae880] deprecated pixel format used, make sure you did set range correctly
...
[swscaler @ 0x7f631ce4b380] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce93940] deprecated pixel format used, make sure you did set range correctly
Detected: 0 | Progress: 37%|████████████████████████████████████████████████████████████▋ | 293/801 [00:02<00:04, 116.16frames/s][swscaler @ 0x7f631cdbd080] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cef3040] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce76ac0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cea1980] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce84b40] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ced4600] deprecated pixel format used, make sure you did set range correctly
[hevc @ 0x7f636ced2480] Duplicate POC in a sequence: 13.
[hevc @ 0x7f636ced2480] Error parsing NAL unit #0.
[swscaler @ 0x7f631ce84b40] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cee3b40] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce0c700] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce4df80] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce68a80] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce4df80] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce1a1c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce4df80] deprecated pixel format used, make sure you did set range correctly
Detected: 0 | Progress: 38%|███████████████████████████████████████████████████████████████▍ | 306/801 [00:02<00:04, 113.30frames/s][swscaler @ 0x7f631cebff00] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce4df80] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce3f0c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce935c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce3f0c0] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631cdede40] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x7f631ce3f0c0] deprecated pixel format used, make sure you did set range correctly
CRITICAL: scene_manager._decode_thread(): Fatal error: Exception raised in decode thread.
Detected: 0 | Progress: 39%|█████████████████████████████████████████████████████████████████ | 314/801 [00:02<00:04, 105.20frames/s]
Traceback (most recent call last):
File "/home/mmokrejs/virtualenv/dvr-scan/bin/scenedetect", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/mmokrejs/virtualenv/dvr-scan/lib/python3.11/site-packages/scenedetect/__main__.py", line 47, in main
run_scenedetect(cli_ctx)
File "/home/mmokrejs/virtualenv/dvr-scan/lib/python3.11/site-packages/scenedetect/_cli/controller.py", line 58, in run_scenedetect
scene_list, cut_list = _detect(context)
^^^^^^^^^^^^^^^^
File "/home/mmokrejs/virtualenv/dvr-scan/lib/python3.11/site-packages/scenedetect/_cli/controller.py", line 101, in _detect
num_frames = context.scene_manager.detect_scenes(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mmokrejs/virtualenv/dvr-scan/lib/python3.11/site-packages/scenedetect/scene_manager.py", line 895, in detect_scenes
raise self._exception_info[1].with_traceback(self._exception_info[2])
File "/home/mmokrejs/virtualenv/dvr-scan/lib/python3.11/site-packages/scenedetect/scene_manager.py", line 918, in _decode_thread
frame_im = video.read()
^^^^^^^^^^^^
File "/home/mmokrejs/virtualenv/dvr-scan/lib/python3.11/site-packages/scenedetect/backends/pyav.py", line 279, in read
self._frame = next(self._container.decode(video=0))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "av/container/input.pyx", line 208, in decode
File "av/packet.pyx", line 80, in av.packet.Packet.decode
File "av/video/stream.pyx", line 35, in av.video.stream.VideoStream.decode
File "av/video/stream.pyx", line 44, in av.video.stream.VideoStream.decode
File "av/codec/context.pyx", line 507, in av.codec.context.CodecContext.decode
File "av/codec/context.pyx", line 404, in av.codec.context.CodecContext._send_packet_and_recv
File "av/error.pyx", line 328, in av.error.err_check
av.error.InvalidDataError: [Errno 1094995529] Invalid data found when processing input
$
from dvr-scan.
Would be nice if the List of motion events: table contained also the frame starts and ends, so that one could see if the input was processed up to the very last frame or not.
As long as you are not seeking in the input, you should be able to tell how many frames were processed from the log output:
[DVR-Scan] Processed 387014 frames read in 4459.8 secs (avg 86.8 FPS).
So how many total frames were in the input file, I mean how many ffmpeg would find otherwise regardless of eventual failure)?
Or show column with percentages of the total input?
from dvr-scan.
So how many total frames were in the input file, I mean how many ffmpeg would find otherwise regardless of eventual failure)?
Or show column with percentages of the total input?
There's no easy way to show the expected total, some containers don't report it accurately and it is often incorrect. Using other libraries like PyAV allows ensuring that you have exhausted the full stream, but using OpenCV like this library currently does, it's not possible to do things like that. I think PyAV allows capturing those warnings, although I recall running into some deadlocks when doing that... I was recently made aware of some progress on the issue at PyAV-Org/PyAV#751 though so it probably is worth looking into to see if it's possible.
In the meantime this is similar enough to #118 and has been a huge help in the debugging process. I'll mark this as a duplicate of that, but feel free to continue the discussion there.
from dvr-scan.
Thank you for your efforts so far. I tend to believe the [ffmpeg/video] hevc: Error parsing NAL unit #0
is not always criticial and maybe the testcase cannt be processed due to the Error while decoding frame (hardware decoding)!
instead. I see multiple [ffmpeg/video] hevc: Error parsing NAL unit #0
messages in a row while playng the movies.
I have several more testcases of mpv
unplayable HEVC files (and causing dvr-scan
to skip the remainder), but with different errors:
attempt 1
[ffmpeg] Failed to reallocate parser buffer to -2147482773
[ffmpeg] Failed to reallocate parser buffer to -2147483584
[ffmpeg] Failed to reallocate parser buffer to -2147483584
[ffmpeg] NULL: missing picture in access unit with size 579234816
(Buffering) V: 00:00:38 / unknown (71%) Dropped: 262
attempt 2
[ffmpeg] Failed to reallocate parser buffer to -2147483292
[ffmpeg] Failed to reallocate parser buffer to -2147483584
[ffmpeg] Failed to reallocate parser buffer to -2147483584
[ffmpeg] Failed to reallocate parser buffer to -2147483584
[ffmpeg] Failed to reallocate parser buffer to -2147483584
[ffmpeg] Failed to reallocate parser buffer to -2147483584
[ffmpeg] Failed to reallocate parser buffer to -2147483584
[ffmpeg] NULL: missing picture in access unit with size 1418189824
(Buffering) V: 00:00:24 / unknown (71%) Dropped: 129
https://trac.ffmpeg.org/ticket/8732
mpv-player/mpv#5893
mpv-player/mpv#5755
My CPU is Intel i-13900K
chip so I use Intel QSV for hardware speedup (i915 kernel module).
My assumption is that all of this happens due to flakey ONVIF cameras and a switch dropping packets. Because of that, sometimes h264
video playing slows down or even stops, possibly while scanning for next full frame. Probably some combination of long series of events triggers a bug somewhere. It seems I will really need to re-encode the original 3gp
files into different stream (h265
) to ensure the incomplete frames are discarded. This also means the FPS (originally 15 or 25 fp) will appear variable frame rate (VFR
) instead of fixed because many of the frames were randomly lost along the way to NVR. But then I will run into a different issue with dvr-scan as it does not support variable frame rate.
I think the design could be completely changed. A recording should be analyzed, regions of repeated movement (trees blowing in the wind) detected as glossy regions and their median color (a car may show up in front of them later so do not ask this region completely). In another sweep dvr-scan should look for motions in other places, even in a single frame. Further, dvr-scan detects some kernel size during it run. Provided I merge 3gp
per 24-hrs into a single HEVC
container (no re-encoding so far) all the videos start in night, in b/w color. I do not know what this kernel size is but certainly dvr-scan should split the video into ranges. This would be the very first pass of reading the file. Notably, the recordings at about the switch of IR cur filter are more grainy with distorted color and should be treated as a separate. Comparing individual/adjacent frames with each other does not make sense. The scene is largely static, sources of errors (small windblow movements can be learned) and then the real scene changes be detected more easily, even if recorded in a single frame.
Manual does not describe expectations so this is how I evolved my scripts to handle thousands of files per day and what I think is meaningful. Merging all videos per day into a single file is asking for a trouble but technically is the only way for me not to get lost in the zillions of files. It already sucks enough that ffmpeg -i list-of-files.txt ... -o merged.hevc
dies due to the moov atom not found
issue which means I have to edit the list-of-files.txt
file and drop from it the problematic file and re-run ffmpeg
.
from dvr-scan.
It seems I will really need to re-encode the original 3gp files into different stream (h265) to ensure the incomplete frames are discarded. This also means the FPS (originally 15 or 25 fp) will appear variable frame rate (VFR) instead of fixed because many of the frames were randomly lost along the way to NVR. But then I will run into a different issue with dvr-scan as it does not support variable frame rate.
The video output should be correct, as long as you are using the default OpenCV output mode. VFR video is only an issue for the accuracy of timestamps, which is why the ffmpeg ouptut mode won't work with it currently. That being said, there should be a filter for ffmpeg that will allow you to convert the VFR video into a fixed framerate one - you may want to consider that as a workaround.
A recording should be analyzed, regions of repeated movement (trees blowing in the wind) detected as glossy regions and their median color (a car may show up in front of them later so do not ask this region completely). In another sweep dvr-scan should look for motions in other places, even in a single frame
This sounds possible and like a good iea, but am lacking a lot of test videos which makes it difficult to develop these kinds of features. That being said, you may have some luck experimenting with the kernel size setting, which should help reduce small movements due to trees for example, and using the region editor to mask out problematic areas that aren't of interest.
from dvr-scan.
@Breakthrough Did you get my emails with links to the testcases causing dvr-scan
to stop processing input files?
from dvr-scan.
Related Issues (20)
- Request: Support for OpenCL HOT 2
- Docker Documentation run command
- When using with wildcard, dvr-scan detects motion across two videos HOT 1
- ROI / region of interest not working HOT 1
- Request: Process video stream from stdin HOT 2
- wildcards produce different detection than file by file run HOT 4
- how do you install this program? HOT 6
- cv2 package got renamed to opencv-python and is now at 4.8.0.76 HOT 2
- Timecode format of hh:mm:ss not recognized HOT 3
- Process ends early and displays error: Element exceeds containing master element HOT 1
- Traceback TypeError for any dvr-scan command HOT 1
- cnt mode stuck HOT 4
- Invalid duration specification for ss: 00:04:60.000 when using -m copy or -m ffmpeg HOT 6
- Require version >= 0.6.2 of scenedetect
- Exception in encode thread when extracting motion events on v1.6 HOT 7
- Unsupported config option: region-file HOT 3
- region-editor config option gets ignored HOT 3
- Request: produce image file (thumb) of the frame with the highest amount of motion per output video HOT 3
- Closing the ROI editor should warn if you have unsaved changes HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dvr-scan.