dalecurtis / video-animation-frame Goto Github PK
View Code? Open in Web Editor NEW[Deprecated] Explainer for video.requestAnimationFrame().
[Deprecated] Explainer for video.requestAnimationFrame().
One still missing field in the VideoFrameMetadata dictionary is remote capture time (e.g. "int64 remoteCaptureTime".
The rationale is as follows: in WebRTC scenario the video originates at a separate machine, which has a separate clock. There might be clock offset or even drift.
WebRTC estimates capture time in both local and remote time-bases. It's logical, that capture time in local clock is already intended to be in captureTime field. This can be used to measure E2E delay.
But the same capture time in remote clock is very useful for synchronization purposes. Suppose the application wants to e.g. render emojis on top of the video or show chat messages, synchronized with the source video. The send side can easily obtain current local clock and attach that as a label to the chat message. Then receiving side could use remoteCaptureTime to decide if this is the frame suitable for showing the message, because they would be in the same timebase.
As this remoteCaptureTime is not in local clock it doesn't even have to be a timstamp type, just some monotonically increasing number, presumably in milliseconds at the send machine should work.
This field will duplicate rtpTimestamp a little, but it's quite different: it's not present for all the frames, unlike rtpTimestamp; It doesn't wrap and can be simply compared with the labels associated with a extra synchonizable content.
If the function passed to video.requestAnimationFrame
doesn't complete before expectedPresentationTime
, will the frame still be presented, or would it be delayed until the function completes?
The WebGL use cases, and interaction with the WEBGL_video_texture extension, should be filled out more in this explainer.
Currently the documentation for presentationTimestamp
says:
The presentation timestamp in seconds of the frame presented. May not be known to the compositor or exist in all cases.
For a use case I have in mind, the presentationTimestamp
is the field I care about the most (in fact, it's the only field I care about). I would like it to specified under which circumstances this field would not exist as it directly impacts the usefulness of this method to me.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.