xfOpenCV Optical Flow implemented on Zedboard with built aarch32 OpenCV libraries

This simple project extends xfOpenCV Optical Flow in different ways for various scenarios on Zedboard.

Xilinx reVision provides many supports for high-end FPGA SoCs like ZCU102, while Zedboard is not supported. Compared to those high-end products, Zedboard is equiped with 32-bit Arm Core and less resource on FPGA. Besides, in the examples provided in xfOpenCV, no video application is presented.

Therefore, we compile the OpenCV 2.4.9 and other required libraries, e.g. ffmpeg, with aarch32 compiler, to support wider usage of Zedboard. Furthermore, we optimize the hardware design of dense optical flow providing in xfOpenCV and the software process to reduce the cost of resource and improve performance. Moreover, we adjust MakeFile for Zedboard.

This project is implemented based on SDSoC 2019.1 and Zedboard. For these open source projects, we provide the version with clear and simple code running at 0.5-5FPS. A close source version is also redesigned, properly scheduled and implemented for parallelisms, which can run at 23-24 FPS with streaming output via TCP and will be open to community later.

If you have any problem, I am happy to help ^_^.

INPUT: OUTPUT:

Usage:

Please note that four versions of optical flow implementation are provided:

1. for the processing of two images
2. for video file
3. for webcam streaming input
4. for UDP Ethernet output with webcam streaming input

You can make them work through the following steps:

1. enter the corresponding directory:

   cd Zedboard-xfOpenCV-Optical-Flow/xfopencv-master/examples/lkdensepyrof_videoinput

   cd Zedboard-xfOpenCV-Optical-Flow/xfopencv-master/examples/lkdensepyrof_photoinput

   cd Zedboard-xfOpenCV-Optical-Flow/xfopencv-master/examples/lkdensepyrof_camerainput

   cd Zedboard-xfOpenCV-Optical-Flow/xfopencv-master/examples/lkdensepyrof_UDPoutput_camerainput

2. source /tools/Xilinx/SDx/2019.1/settings64.sh

3. make all -j4

4. If you want to try the examples with video/webcam, you HAVE TO build the petalinux for video/webcam libraries and device drivers according THE README.MD in the directory "petalinux" and the note below. Then you can use the built SDSoC design for your purpose. For example, copy all the SD card files, the libraries and data (video/photos) into SD card. And boot the board.

5. the system can boot via a command "boot" in serial terminal (braudrate=115200). To login, the system account is "root" and the password is "root".

Note

(1) Please remember to copy the libraries including opencv and other libs and libsds_lib.so (this file is in /Xilinx/SDx/2019.1/target/aarch32-linux/lib) to /usr/lib/ or /lib in you SD card for dynamic linking.

(2) If you want to set a WebCam for Zedboard for this application, you can follow the instruction: Interfacing a USB WebCam and Enable USB Tethering on ZYNQ-7000 AP SoC Running Linux. Important: The instruction missed some important parts:

(2a) when you do "petalinux-config -c kernel", find the driver for your WebCam in "Device Drivers-->Multimedia support[y]-->Media USB Adapters[y]-->YOUR WEBCAM[y] (e.g. for our system, SONIX JPEG USB Camera Driver)"

(2b) when you do "petalinux-config -c kernel", set up xlnk for your FPGA accelerator:

i   ) Device Drivers -> Staging drivers (ON) 
ii  ) Device Drivers -> Staging drivers -> Xilinx APF Accelerator driver (ON)
iii ) Device Drivers -> Staging drivers -> Xilinx APF Accelerator driver -> Xilinx APF DMA engines support (ON)

(2c) when you do "petalinux-config -c rootfs", add glibc and libstdc++ in "File Packages System / misc"

(2d) a modified ZED BSP and hdf file are provided (here), for which you just need to set the webcam driver according to (2a). With this BSP, you don't need to follow the instruction mentioned in (2). A more detailed guide is provided.