the setting bitrate should be a step of 25000

Codec Controls

             video_bitrate_mode 0x009909ce (menu)   : min=0 max=1 default=0 value=0 flags=update
                                0: Variable Bitrate
                                1: Constant Bitrate
                  video_bitrate 0x009909cf (int)    : min=25000 max=25000000 step=25000 default=10000000 value=10000000
           sequence_header_mode 0x009909d8 (menu)   : min=0 max=1 default=1 value=1
                                0: Separate Buffer
                                1: Joined With 1st Frame
         repeat_sequence_header 0x009909e2 (bool)   : default=0 value=0
                force_key_frame 0x009909e5 (button) : flags=write-only, execute-on-write
          h264_minimum_qp_value 0x00990a61 (int)    : min=0 max=51 step=1 default=20 value=20
          h264_maximum_qp_value 0x00990a62 (int)    : min=0 max=51 step=1 default=51 value=51
            h264_i_frame_period 0x00990a66 (int)    : min=0 max=2147483647 step=1 default=60 value=60
                     h264_level 0x00990a67 (menu)   : min=0 max=15 default=11 value=11
                                0: 1
                                1: 1b
                                2: 1.1
                                3: 1.2
                                4: 1.3
                                5: 2
                                6: 2.1
                                7: 2.2
                                8: 3
                                9: 3.1
                                10: 3.2
                                11: 4
                                12: 4.1
                                13: 4.2
                                14: 5
                                15: 5.1
                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=4 value=4
                                0: Baseline
                                1: Constrained Baseline
                                2: Main
                                4: High

There are no changes to the code. Maybe tidy up the headers.

simply trigger via data channel first

你好, 我在该项目中没有发现raspi视频流的接收方式,
请问我应该用什么方式去接收视频流呢?

Capture in another thread to avoid HW being suddenly blocked turning out the latency rise sharply.
If sending the raw frame to the HW output queue fails, drop it and return WEBRTC_VIDEO_CODEC_ERROR.

No matter if webrtc is connecting or not.

1. from ioctl frame buffer data
2. save segment video files

by the way, is it possible to encode the raw frame as h264 into the queue and then share encoded frames to webrtc/ffmpeg?

Write the video file in another thread, not call async every time.
If webrtc dropped frames will come across a memory leak, maybe creating a new thread and using an queue buffer is a better way.

Decouple the signalr and let the signaling mechanism be replaceable.

Hello,

Great project! I would like to use this library for a specific use case, but I am not sure if it would be suitable so I'll ask here.

The use case involves using two Raspberry PI Zero 2 W devices to stream video feeds in parallel to a Linux terminal. The critical requirement is to align each video frame from one stream with the corresponding frame from the other stream, based on their timestamps.

To achieve this, the following functionalities are necessary:

Each video frame must be accompanied by a unique timestamp when sent from the Raspberry PIs.
The receiving terminal should be capable of processing the streams by fetching frames individually, retrieving their timestamps, and synchronizing them with the frames from the alternate stream.
Could you please advise whether your library supports these features, or if there is a possibility to implement such a synchronization mechanism using your library?

Thanks in advance!

It makes the framerate more stable and lowers latency.

prepare to receive the commands for recording video

1. use <boost/program_options.hpp> to read arguments, including fps/width/height/...
2. default values are set in arg.h

Symmetric NAT will change ports

sudo apt-get install libunwind-dev zip unzip tar
git clone https://github.com/google/boringssl.git
cd boringssl
mkdir build
cd build
cmake -GNinja .. -DBUILD_SHARED_LIBS=1
ninja
sudo ninja install
sudo cp -r ../install/include/* /usr/local/include/
sudo cp ../install/lib/*.a /usr/local/lib/
sudo cp ../install/lib/*.so /usr/local/lib/

cmake -GNinja .. -DBUILD_SHARED_LIBS=1

should be instead

cmake -GNinja .. 

otherwise the .a files dont get generated

Capturing the h264 packets directly from the camera module, written into mp4 can reduce by 80% space compared to mjpeg.

Use webrtc::VideoEncoderFactory to produce some custom "encoder"
The encoder subclass passes encoded frames (e.g. H264) to the webrtc::EncodedImage::OnEncodedImage callback.
The encoder doesn't really have to encode anything, it just delivers pre-encoded frames.

keep a relatively high bitrate when using h264 v4l2m2m
stay within the range that YouTube recommends

Sometimes, the video usually got blocked in the first 5 seconds but the audio did not.
I tried to increase the h264 I-frame interval, it seems a workaround solution.

Client-side can't reach the target fps.
Set the codec parameter from CBR to VBR will make it smooth!

Since the compilation of all libraries is quite heavy, let's try to find the right QEMU setting to compile them all on an x86_64 machine. These are my cross compilation instructions that so far managed to compile libwebrtc.a. However, I encountered issues when trying to compile microsoft-signalr.so at the casablanca step (see also here).

Let me first give the specs of the system I used

# Operating System and Version:
lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.5 LTS
Release:	20.04
Codename:	focal

# kernel
uname -r
5.15.0-88-generic

# architecture
uname -m
x86_64

1. ARM64 Root Filesystem Setup:

# Install the ARM64 cross-compilation toolchain and other necessary packages
sudo apt-get update
sudo apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu binutils-aarch64-linux-gnu qemu-user-static debootstrap
sudo debootstrap --arch arm64 --foreign bullseye rootfs
sudo cp /usr/bin/qemu-aarch64-static rootfs/usr/bin/

2. Chroot Environment Setup:

sudo mount -o bind /dev rootfs/dev
sudo mount -t devpts -o gid=5,mode=620 devpts rootfs/dev/pts
sudo chroot rootfs /bin/bash
mount -t proc /proc /proc

3. Debootstrap Initialization:

/debootstrap/debootstrap --second-stage

4. Basic Package Installation:

apt-get update
apt-get install -y git lsb-release sudo curl

5. User Environment Setup:

useradd -m pi
echo "pi ALL=(ALL) NOPASSWD:ALL" | tee -a /etc/sudoers
su - pi
bash

6. Additional Dependencies Installation:

sudo apt-get update
sudo apt-get install -y g++ libasound2-dev libpulse-dev libudev-dev libexpat1-dev libnss3-dev libgtk-3-dev python pulseaudio libpulse-dev build-essential libncurses5 libx11-dev
sudo apt remove -y libssl-dev
pulseaudio --start

7. Depot Tools Setup:

cd /home/pi
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
echo "export PATH=$(pwd)/depot_tools:\$PATH" >> ~/.bashrc
source ~/.bashrc

8. Sysroot Dependencies for ARM64:

sudo apt-get install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu binutils-aarch64-linux-gnu

9. WebRTC Source Fetching:

mkdir -p ~/webrtc && cd ~/webrtc
fetch --nohooks webrtc
src/build/linux/sysroot_scripts/install-sysroot.py --arch=arm64
gclient sync -D
cd src
git checkout -b local-5790 branch-heads/5790
git gc --aggressive
gclient sync -D --force --reset --with_branch_heads --no-history

10. Clang Setup:

cd /home/pi
curl -OL https://github.com/llvm/llvm-project/releases/download/llvmorg-16.0.6/clang+llvm-16.0.6-aarch64-linux-gnu.tar.xz
tar Jxvf clang+llvm-16.0.6-aarch64-linux-gnu.tar.xz
echo 'export PATH=/home/pi/clang+llvm-16.0.6-aarch64-linux-gnu/bin:$PATH' >> ~/.bashrc
source ~/.bashrc

11. Ninja Building and Installation:

git clone https://github.com/martine/ninja.git
cd ninja
./configure.py --bootstrap
mv /home/pi/depot_tools/ninja /home/pi/depot_tools/ninja_org
cp /home/pi/ninja/ninja /home/pi/depot_tools/ninja

12. GN Building and Installation:

cd /home/pi
git clone https://gn.googlesource.com/gn
cd gn
sed -i -e "s/-Wl,--icf=all//" build/gen.py
python build/gen.py
ninja -C out
sudo mv /home/pi/webrtc/src/buildtools/linux64/gn /home/pi/webrtc/src/buildtools/linux64/gn_org
cp /home/pi/gn/out/gn /home/pi/webrtc/src/buildtools/linux64/gn
sudo mv /home/pi/depot_tools/gn /home/pi/depot_tools/gn_org
cp /home/pi/gn/out/gn /home/pi/depot_tools/gn

13. Extra Dependencies for GN:

sudo apt-get update
sudo apt-get install python3-pip -y
pip3 install setuptools

14. WebRTC Build Configuration:

gn gen out/Release64 --args='target_os="linux" target_cpu="arm64" rtc_include_tests=false rtc_use_h264=false use_rtti=true is_component_build=false is_debug=false rtc_build_examples=false use_custom_libcxx=false rtc_build_tools=false rtc_use_pipewire=false clang_base_path="/home/pi/clang+llvm-16.0.6-aarch64-linux-gnu" clang_use_chrome_plugins=false'
ninja -C out/Release64

15. Header Files Synchronization:

sudo apt-get install rsync
rsync -amv --include=*/ --include=*.h --include=*.hpp --exclude=* ./ ./include

16. Compiled libwebrtc.a Location:

/home/pi/webrtc/src/out/Release64/obj/libwebrtc.a

ffmpeg -i ./20230617_155615.mp4 -s 480x270 -c:v h264_v4l2m2m test.mp4
An input 1280x720 sample video is resized to 480x270 by h264_v4l2m2m fine.
Is the problem between the libyuv transform mjepg to yuv420?

Pipeline data via DMA can decrease by 5~10% CPU usage on rpi 3b.

Receiving commands from the client via the data channel can be used sooner.