• Camera Util
  • Overview
  • Requirements
  • Installation
    • To install OpenCV on Raspi:
  • How to create camera config files
  • Run Example
    • Example Programs
  • Capture modules
    • blackflyCapture
    • nanoCapture
    • cv2Capture
    • rtspCapture
    • piCapture
  • Changes
  • References
  • Camera Settings
    • Sony IMX287 FLIR Blackfly S BFS-U3-04S2M
    • OV5647 OmniVision RasPi
    • IMX219 Sony RasPi
    • ELP USB Camera
    • Dell Internal USB

Table of contents generated with markdown-toc

A collection of python threaded camera support routines for

• USB and laptop internal webcams
• RTSP streams
• MIPI CSI cameras (Raspberry Pi, Jetson Nano)
• FLIR blackfly (USB)

Also support to save as

• HD5
• tiff
• avi, mkv

Supported OS

• Windows
• MacOS
• Unix

The routines primarily use OpenCV or PySpin to interface with the camera. The image acquisition runs in a background thread to achieve maximal frame rate and minimal latency.

This work is based on efforts from Mark Omo and Craig Post.

• PySpin for FLIR cameras
• opencv for USB, CSI cameras, RTSP network streams
  • Windows uses cv2.CAP_MSMF
  • Darwin uses cv2.CAP_AVFOUNDATION
  • Linux uses cv2.CAP_V4L2
  • Jetson Nano uses cv2.CAP_GSTREAMER
  • RTSP uses cv2.CAP_GSTREAMER

On windows GSTREAMER is not enabled by default. If you want RTSP functionaly you need to custom built opencv. See my windows installation instructions on Github.

camera

1. cd "folder where you have this Readme.md file"
2. pip install . or python setup.py bdist_wheel and pip3 install .\dist\*.whl

opencv
3. pip3 install opencv-contrib-python

tiff and hd5

• https://www.lfd.uci.edu/~gohlke/pythonlibs/#imagecodecs
• https://www.lfd.uci.edu/~gohlke/pythonlibs/#tifffile
• https://www.lfd.uci.edu/~gohlke/pythonlibs/#h5py

Make sure the version matches your python installation (e.g. 3.8) and CPU architecture (e.g. 64).

blackfly
Spinnaker provides SDK and python bindings. The versions of those two programs need to match.

• Spinnaker SDK is at https://flir.app.boxcn.net/v/SpinnakerSDK, install development options
• download spinnaker_python from same location

To install the downloaded wheels, in CMD window:

4. cd Downloads

5. pip3 install imagecodecs....

6. pip3 install tifffile....

7. pip3 install h5py....

8. pip3 install spinnaker_python...

9. Make sure you have C:\temp directory if you use the example storage programs.

10. To get better tiff performance, installing libtiff is advised: https://github.com/uutzinger/Windows_Install_Scripts/blob/master/Buildinglibtiff.md

3. cd ~
4. sudo pip3 install opencv-contrib-python==4.5.3.56 as time progresses the version numnber might need to be increased.
5. sudo pip3 install tifffile h5py platform imagecodecs

A. Specifications of your camera

On Windows, the Camera utility will give you resolution options and frames per second. To investigate other options you can use OBS studio (or any other capture program), establish camera capture device and inspect video options. py -3 list_cv2CameraProperties.py will show all camera options the video system offers. When an option states -1 it likely is not available for that camera.

B. Configuration file

Use one of the existing camera configutrations in examples/configs or create your own. As first step set appropriate resolution and frames per second. As second step figure out the exposure and autoexposure settings.

Run capture_display.py from .\examples You need to set the proper config file in the program. You should not need to edit python files in capture or streamer folder.

Display:
In general display should occur in main program. OpenCV requires waitkey to be executed in order to update the display and limits update rate to about 50-90 fps.

Queue: Data transfer between the main program and capture and storage threads.

Examples:

• capture_display.py tests camera capture for all capture platforms except blackfly.

• blackfly_display.py tests the blackfly capture module, displays images and reports framerate.

• capture_saveavi_display.py display and save to avi files

• capture_savemkv_display.py display and save to mkv files

• test_display.py testing of opencv display framerate, no camera, just refresh rate.

• test_savehd5.py testing of the disk throughput with hdf5, no camera

• test_savetiff.py testing of the disk throughput with tiff, no camera

• test_saveavi.py testing of the disk throughput with avi, no camera, only 3 color planes per image possible

• test_savemkv.py testing of the disk throughput with mkv/mp4v, no camera, only 3 color planes per image possible

• test_blackfly.py tests the blackfly capture module and reports framerate, no display

• blackfly_savehdf5.py no display but incoporates saving to disk

• blackfly_savetiff.py no display but incoporates saving to disk

• blackfly_savehdf5_display.py display and incoporates saving to disk

• blackfly_savetiff_display.py display andincoporates saving to disk

py -3 setup.py check
py -3 setup.py sdist
py -3 setup.py bdist_wheel
pip3 install dist/thenewpackage.whl
twine upload dist/*

Simplifies the settings needed for the Blackfly camera.
Supports trigger out during frame exposure and trigger in for frame start.
Optimized settings to achieve full frame rate with S BFS-U3-04S2M.

Uses gstreamer pipeline for Jetson Nano.
Pipline for nvidia conversion and nvarguscamera capture.
Settings optimized for Sony IMX219 Raspi v2 Module.

Uses the cv2 capture architecture.
The video subsystem is choosen based on the operating system.

gstreamer based rtsp network stream capture for all platforms.
gstreamer is called through OpenCV.
By default OpenCV supports ffmpeg and not gstreamer. Jetson Nano does not support ffmpeg but opencv is prebuilt with gstreamer for that platform.

Interface for picamera module. Depricated since cv2Capture is more efficient for the Raspberry Pi.

2022 - February added libcamera capture for Raspian Bullseye
2022 - January added queue as intialization option, updated cv2Capture
2021 - November moved queue into class
2021 - November added rtp server and client
2021 - November added mkvServer, wheel installation, cleanup
2021 - October added aviServer and multicamera example, PySpin trigger fix
2021 - September updated PySpin trigger out polarity setting  
2020 - Release  
Urs Utzinger

realpython:
https://realpython.com/python-concurrency/
https://realpython.com/python-sleep/ https://realpython.com/async-io-python/

python for the lab:
https://www.pythonforthelab.com/blog/handling-and-sharing-data-between-threads/

• 720x540 524fps
• auto_exposure off

• auto_exposure 0: auto, 1:manual
• exposure in microseconds
• Max Resolution 2592x1944
• YU12, (YUYV, RGB3, JPEG, H264, YVYU, VYUY, UYVY, NV12, BGR3, YV12, NV21, BGR4)
• 320x240 90fps
• 640x480 90fps
• 1280x720 60fps
• 1920x1080 6.4fps
• 2592x1944 6.4fps

• auto_exposure 0: auto, 1:manual
• exposure in microseconds
• Max Resolution 3280x2464
• YU12, (YUYV, RGB3, JPEG, H264, YVYU, VYUY, UYVY, NV12, BGR4)
• 320x240 90fps
• 640x480 90fps
• 1280x720 60fps
• 1920x1080 4.4fps
• 3280x2464 2.8fps

• MJPG
• 320x240 and 640/480, 120fps
• auto_exposure, can not figure out out in MJPG mode
• auto_exposure = 0 -> static exposure
• exposure is about (exposure value / 10) in ms
• WB_TEMP 6500

• 320x240, 30fps
• YUY2
• autoexposure ? 0.25, 0.74 -1. 0
• WB_TEMP 4600
• 1280x720, 30fps
• 620x480, 30fps
• 960x540, 30fps