Comments (6)
This is an area we've quite wanted to do work on, but haven't got round to it.
It should be possible to synchronise a pair of cameras with a software control loop. The things you'd have to do would be:
-
Look at the
"SensorTimestamp"
value in the metadata with each frame. By comparing these you can obviously tell how unsynchronised the cameras are. -
Adjust the
"FrameDurationLimits"
control value to bring the frames together.
from picamera2.
Thank you, that was the starter information I needed. I will check it out!
from picamera2.
``So, this is cool, seems to work.
Finally I wanted to save stills so I started directly using the create_still_configuration
which implicitely uses just one buffer. When the control loop is close to zero delta it could lead to a jump in the SensorTimestamp
by 1/FPS seconds due to a dropped frame. I fixed by using a buffer, here buffer_count=3
.
This is my very basic script, that just syncronizes, but nothing more yet:
#!/usr/bin/python
from picamera2 import Picamera2
def P_controller(Kp: float = 0.05, setpoint: float = 0, measurement: float = 0, output_limits=(-10000, 10000)):
e = setpoint - measurement
P = Kp * e
output_value = P
# output and limit if output_limits set
lower, upper = output_limits
if (upper is not None) and (output_value > upper):
return upper
elif (lower is not None) and (output_value < lower):
return lower
return output_value
if len(Picamera2.global_camera_info()) <= 1:
print("SKIPPED (one camera)")
quit()
# Primary (leads)
picam2a = Picamera2(0)
# need buffer_count > 1 because if a frame is skipped, there will be a jump in SensorTimestamp due to dropped frame which messes with the control
config2a = picam2a.create_still_configuration(controls={"FrameRate": 10}, buffer_count=3)
picam2a.configure(config2a)
# Secondary (follows)
picam2b = Picamera2(1)
# need buffer_count > 1 because if a frame is skipped, there will be a jump in SensorTimestamp due to dropped frame which messes with the control
config2b = picam2b.create_still_configuration(controls={"FrameRate": 10}, buffer_count=3)
picam2b.configure(config2b)
picam2a.start()
picam2b.start()
print("Press Ctrl+C to exit")
try:
while True:
metadata_picam2a = picam2a.capture_metadata()
metadata_picam2b = picam2b.capture_metadata()
timestamp_picam2a = metadata_picam2a["SensorTimestamp"] / 1000 # convert ns to µs because all other values are in µs
timestamp_picam2b = metadata_picam2b["SensorTimestamp"] / 1000 # convert ns to µs because all other values are in µs
timestamp_delta = timestamp_picam2b - timestamp_picam2a
controller_output_frameduration_delta = int(P_controller(0.05, 0, timestamp_delta, (-10000, 10000)))
control_out_frameduration = int(metadata_picam2a["FrameDuration"] + controller_output_frameduration_delta) # sync to a, so use that for ref
print("Cam A: SensorTimestamp: ", timestamp_picam2a, " FrameDuration: ", metadata_picam2a["FrameDuration"])
print("Cam B: SensorTimestamp: ", timestamp_picam2b, " FrameDuration: ", metadata_picam2b["FrameDuration"])
print("SensorTimestampDelta: ", round(timestamp_delta / 1000, 1), "ms")
print("FrameDurationDelta: ", controller_output_frameduration_delta, "new FrameDurationLimit: ", control_out_frameduration)
with picam2b.controls as ctrl:
# set new FrameDurationLimits based on P_controller output.
ctrl.FrameDurationLimits = (control_out_frameduration, control_out_frameduration)
except KeyboardInterrupt:
print("got Ctrl+C, exiting")
picam2a.stop()
picam2b.stop()
This is the output. The delta remains very stable close to zero after first start:
Cam A: SensorTimestamp: 10227724610.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10227752441.0 FrameDuration: 97300 SensorTimestampDelta: 27.8 ms
FrameDurationDelta: -1391 new FrameDurationLimit: 98591
Cam A: SensorTimestamp: 10227824592.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10227849742.0 FrameDuration: 97510 SensorTimestampDelta: 25.1 ms
FrameDurationDelta: -1257 new FrameDurationLimit: 98725
Cam A: SensorTimestamp: 10227924577.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10227947255.0 FrameDuration: 97721 SensorTimestampDelta: 22.7 ms
FrameDurationDelta: -1133 new FrameDurationLimit: 98849
Cam A: SensorTimestamp: 10228024561.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10228044979.0 FrameDuration: 97931 SensorTimestampDelta: 20.4 ms
FrameDurationDelta: -1020 new FrameDurationLimit: 98962
Cam A: SensorTimestamp: 10228124544.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10228142911.0 FrameDuration: 98115 SensorTimestampDelta: 18.4 ms
FrameDurationDelta: -918 new FrameDurationLimit: 99064
Cam A: SensorTimestamp: 10228224528.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10228241027.0 FrameDuration: 98273 SensorTimestampDelta: 16.5 ms
FrameDurationDelta: -824 new FrameDurationLimit: 99158
Cam A: SensorTimestamp: 10228324509.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10228339302.0 FrameDuration: 98430 SensorTimestampDelta: 14.8 ms
FrameDurationDelta: -739 new FrameDurationLimit: 99243
Cam A: SensorTimestamp: 10228424491.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10228437736.0 FrameDuration: 98588 SensorTimestampDelta: 13.2 ms
FrameDurationDelta: -662 new FrameDurationLimit: 99320
Cam A: SensorTimestamp: 10228524475.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10228536326.0 FrameDuration: 98720 SensorTimestampDelta: 11.9 ms
FrameDurationDelta: -592 new FrameDurationLimit: 99390
Cam A: SensorTimestamp: 10228624458.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10228635047.0 FrameDuration: 98825 SensorTimestampDelta: 10.6 ms
FrameDurationDelta: -529 new FrameDurationLimit: 99453
Cam A: SensorTimestamp: 10228724440.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10228733874.0 FrameDuration: 98956 SensorTimestampDelta: 9.4 ms
FrameDurationDelta: -471 new FrameDurationLimit: 99511
Cam A: SensorTimestamp: 10228824423.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10228832833.0 FrameDuration: 99061 SensorTimestampDelta: 8.4 ms
FrameDurationDelta: -420 new FrameDurationLimit: 99562
Cam A: SensorTimestamp: 10228924407.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10228931899.0 FrameDuration: 99140 SensorTimestampDelta: 7.5 ms
FrameDurationDelta: -374 new FrameDurationLimit: 99608
Cam A: SensorTimestamp: 10229024390.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10229031037.0 FrameDuration: 99219 SensorTimestampDelta: 6.6 ms
FrameDurationDelta: -332 new FrameDurationLimit: 99650
Cam A: SensorTimestamp: 10229124372.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10229130260.0 FrameDuration: 99298 SensorTimestampDelta: 5.9 ms
FrameDurationDelta: -294 new FrameDurationLimit: 99688
Cam A: SensorTimestamp: 10229224355.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10229229557.0 FrameDuration: 99377 SensorTimestampDelta: 5.2 ms
FrameDurationDelta: -260 new FrameDurationLimit: 99722
Cam A: SensorTimestamp: 10229324338.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10229328939.0 FrameDuration: 99430 SensorTimestampDelta: 4.6 ms
FrameDurationDelta: -230 new FrameDurationLimit: 99752
Cam A: SensorTimestamp: 10229424322.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10229428371.0 FrameDuration: 99508 SensorTimestampDelta: 4.0 ms
FrameDurationDelta: -202 new FrameDurationLimit: 99780
Cam A: SensorTimestamp: 10229524308.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10229527881.0 FrameDuration: 99561 SensorTimestampDelta: 3.6 ms
FrameDurationDelta: -178 new FrameDurationLimit: 99804
Cam A: SensorTimestamp: 10229624287.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10229627440.0 FrameDuration: 99587 SensorTimestampDelta: 3.2 ms
FrameDurationDelta: -157 new FrameDurationLimit: 99825
Cam A: SensorTimestamp: 10229724271.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10229727031.0 FrameDuration: 99640 SensorTimestampDelta: 2.8 ms
FrameDurationDelta: -138 new FrameDurationLimit: 99844
Cam A: SensorTimestamp: 10229824253.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10229826672.0 FrameDuration: 99666 SensorTimestampDelta: 2.4 ms
FrameDurationDelta: -120 new FrameDurationLimit: 99862
Cam A: SensorTimestamp: 10229924236.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10229926340.0 FrameDuration: 99719 SensorTimestampDelta: 2.1 ms
FrameDurationDelta: -105 new FrameDurationLimit: 99877
Cam A: SensorTimestamp: 10230024220.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10230026062.0 FrameDuration: 99745 SensorTimestampDelta: 1.8 ms
FrameDurationDelta: -92 new FrameDurationLimit: 99890
Cam A: SensorTimestamp: 10230124202.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10230125808.0 FrameDuration: 99771 SensorTimestampDelta: 1.6 ms
FrameDurationDelta: -80 new FrameDurationLimit: 99902
Cam A: SensorTimestamp: 10230224185.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10230225581.0 FrameDuration: 99798 SensorTimestampDelta: 1.4 ms
FrameDurationDelta: -69 new FrameDurationLimit: 99913
Cam A: SensorTimestamp: 10230324169.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10230325381.0 FrameDuration: 99824 SensorTimestampDelta: 1.2 ms
FrameDurationDelta: -60 new FrameDurationLimit: 99922
Cam A: SensorTimestamp: 10230424151.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10230425207.0 FrameDuration: 99824 SensorTimestampDelta: 1.1 ms
FrameDurationDelta: -52 new FrameDurationLimit: 99930
Cam A: SensorTimestamp: 10230524135.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10230525032.0 FrameDuration: 99850 SensorTimestampDelta: 0.9 ms
FrameDurationDelta: -44 new FrameDurationLimit: 99938
Cam A: SensorTimestamp: 10230624117.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10230624885.0 FrameDuration: 99876 SensorTimestampDelta: 0.8 ms
FrameDurationDelta: -38 new FrameDurationLimit: 99944
Cam A: SensorTimestamp: 10230724099.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10230724763.0 FrameDuration: 99876 SensorTimestampDelta: 0.7 ms
FrameDurationDelta: -33 new FrameDurationLimit: 99949
Cam A: SensorTimestamp: 10230824083.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10230824641.0 FrameDuration: 99876 SensorTimestampDelta: 0.6 ms
FrameDurationDelta: -27 new FrameDurationLimit: 99955
Cam A: SensorTimestamp: 10230924066.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10230924525.0 FrameDuration: 99903 SensorTimestampDelta: 0.5 ms
FrameDurationDelta: -22 new FrameDurationLimit: 99960
Cam A: SensorTimestamp: 10231024049.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10231024430.0 FrameDuration: 99903 SensorTimestampDelta: 0.4 ms
FrameDurationDelta: -19 new FrameDurationLimit: 99963
Cam A: SensorTimestamp: 10231124032.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10231124335.0 FrameDuration: 99929 SensorTimestampDelta: 0.3 ms
FrameDurationDelta: -15 new FrameDurationLimit: 99967
Cam A: SensorTimestamp: 10231224016.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10231224265.0 FrameDuration: 99929 SensorTimestampDelta: 0.2 ms
FrameDurationDelta: -12 new FrameDurationLimit: 99970
Cam A: SensorTimestamp: 10231323998.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10231324197.0 FrameDuration: 99929 SensorTimestampDelta: 0.2 ms
FrameDurationDelta: -9 new FrameDurationLimit: 99973
Cam A: SensorTimestamp: 10231423981.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10231424128.0 FrameDuration: 99929 SensorTimestampDelta: 0.1 ms
FrameDurationDelta: -7 new FrameDurationLimit: 99975
Cam A: SensorTimestamp: 10231523964.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10231524058.0 FrameDuration: 99929 SensorTimestampDelta: 0.1 ms
FrameDurationDelta: -4 new FrameDurationLimit: 99978
Cam A: SensorTimestamp: 10231623948.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10231623985.0 FrameDuration: 99955 SensorTimestampDelta: 0.0 ms
FrameDurationDelta: -1 new FrameDurationLimit: 99981
Cam A: SensorTimestamp: 10231723930.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10231723949.0 FrameDuration: 99955 SensorTimestampDelta: 0.0 ms
FrameDurationDelta: 0 new FrameDurationLimit: 99982
Cam A: SensorTimestamp: 10231823927.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10231823907.0 FrameDuration: 99955 SensorTimestampDelta: -0.0 ms
FrameDurationDelta: 1 new FrameDurationLimit: 99983
Cam A: SensorTimestamp: 10231923889.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10231923865.0 FrameDuration: 99955 SensorTimestampDelta: -0.0 ms
FrameDurationDelta: 1 new FrameDurationLimit: 99983
Cam A: SensorTimestamp: 10232023872.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10232023823.0 FrameDuration: 99955 SensorTimestampDelta: -0.0 ms
FrameDurationDelta: 2 new FrameDurationLimit: 99984
Cam A: SensorTimestamp: 10232123855.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10232123780.0 FrameDuration: 99955 SensorTimestampDelta: -0.1 ms
FrameDurationDelta: 3 new FrameDurationLimit: 99985
Cam A: SensorTimestamp: 10232223843.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10232223737.0 FrameDuration: 99955 SensorTimestampDelta: -0.1 ms
FrameDurationDelta: 5 new FrameDurationLimit: 99987
Cam A: SensorTimestamp: 10232323825.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10232323693.0 FrameDuration: 99955 SensorTimestampDelta: -0.1 ms
FrameDurationDelta: 6 new FrameDurationLimit: 99988
Cam A: SensorTimestamp: 10232423809.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10232423652.0 FrameDuration: 99955 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 7 new FrameDurationLimit: 99989
Cam A: SensorTimestamp: 10232523791.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10232523607.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 9 new FrameDurationLimit: 99991
Cam A: SensorTimestamp: 10232623774.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10232623592.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 9 new FrameDurationLimit: 99991
Cam A: SensorTimestamp: 10232723757.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10232723574.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 9 new FrameDurationLimit: 99991
Cam A: SensorTimestamp: 10232823740.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10232823558.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 9 new FrameDurationLimit: 99991
Cam A: SensorTimestamp: 10232923723.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10232923543.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 9 new FrameDurationLimit: 99991
Cam A: SensorTimestamp: 10233023706.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10233023525.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 9 new FrameDurationLimit: 99991
Cam A: SensorTimestamp: 10233123689.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10233123509.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 9 new FrameDurationLimit: 99991
Cam A: SensorTimestamp: 10233223674.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10233223493.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 9 new FrameDurationLimit: 99991
Cam A: SensorTimestamp: 10233323655.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10233323476.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 8 new FrameDurationLimit: 99990
Cam A: SensorTimestamp: 10233423638.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10233423461.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 8 new FrameDurationLimit: 99990
Cam A: SensorTimestamp: 10233523621.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10233523444.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 8 new FrameDurationLimit: 99990
Cam A: SensorTimestamp: 10233623604.0 FrameDuration: 99982
Cam B: SensorTimestamp: 10233623428.0 FrameDuration: 99982 SensorTimestampDelta: -0.2 ms
FrameDurationDelta: 8 new FrameDurationLimit: 99990
^Cgot Ctrl+C, exiting
from picamera2.
Very cool, thanks for the update!
from picamera2.
I added also sync capture triggered by gpio on pi5. Pictures came out synced perfectly and got no hiccups after photos were taken in the synchronization.
You mind if I send a minimal example via pullrequest?
from picamera2.
Yes, please do send new examples. If you can check the contribution guidelines that would be helpful - I think it boils down mostly to running flake8, signing the commit, and submitting it to the "next" branch.
from picamera2.
Related Issues (20)
- [HOW-TO] change the exposition mode and constraint? HOT 4
- [HOW-TO] Reduce delay after switching sensor modes. HOT 4
- [SUGGESTION] HOT 1
- [OTHER] Please improve/update Documentation HOT 3
- [BUG]Program blocking due to camera disconnection HOT 1
- Unexpected warning from v4l2 HOT 1
- strange video size increase with waiting time between video captures
- [SUGGESTION] Mirror DRM/KMS camera preview, across multiple monitors HOT 6
- disable logging when taking picture HOT 5
- [HOW-TO] Improve streaming quality of Raspberry Pi Camera v3 during drone flight HOT 4
- [HOW-TO] Improve video quality of Raspberry Pi Camera v3 for detecting aruco markers in very high or low light conditions HOT 1
- [BUG] capture.array not working with imx296 and rpi-update > 6.6.31 on compute module 4 Bullseye 64bit HOT 1
- [BUG] libav_h264 encoder uses pyav function wrong HOT 6
- [OTHER] IndexError HOT 2
- [BUG] Continuous AutoFocus fails to converge HOT 11
- [HOW-TO] Camera Flash HOT 3
- [HOW-TO] Continuously stream lores, Occasionally record main HOT 2
- [BUG] No exit from function capture_request() HOT 8
- [BUG] Numpy 2.x not supported HOT 1
- [OTHER] Blocks of code overflowing in PDF documentation HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from picamera2.