dalecurtis / mediarecorder-muxer Goto Github PK

Instead of plugging the muxer API as an extension to MediaRecorder API, I think one option is to have a new standard API for W3C Muxers. I take as inspiration the WebM-Muxer API, as described here:

https://github.com/Vanilagy/webm-muxer

I think it would be a cleaner solution, which would support all the features for an audio / video / subtitles muxer, which is what we want when designing an audio or video editor, for example. I suggest starting with just webm as mandatory muxer, and than adding more as stack-holders demand it.

I was curious if this could be achieved by using a MediaStreamTrackGenerator. I build a little demo for it.

https://stackblitz.com/edit/js-1jfqfv?file=index.js,index.html

It uses a canvas to render red and green rectangles which get then converted into VideoFrames and fed into the writabe of a MediaStreamTrackGenerator. It almost works.

The problem is that the MediaStreamTrackGenerator doesn't seem to respect the duration and the timestamp of a VideoFrame. I'm not sure if this is a bug or if that's intentional. If it would respect that one could generate and "record" a MediaStream using the MediaRecorder at least in real time.

I know that it's already possible to use captureStream() on a canvas to achieve a similar thing. The problem with that solution however is that the timing is unreliable. Being able to craft VideoFrames with a precise duration and timestamp would solve this problem.

Maybe this idea could even be expanded to support faster than realtime rendering.