Add stdin stream support. about ffmpeg.wasm HOT 17 OPEN

@davedoesdev managed to asyncify ffmpeg.js's emscripten_read_async to get streaming outputs Kagami/ffmpeg.js#166 so we could potentially manage to asyncify emscripten's stdin/stdout's to finally get streaming support?

from ffmpeg.wasm.

Is there an ETA on that major refactoring ?

after the major refectoring

If the refactor is not gonna happen(since i have not seen any commits on this repo & your fork of FFMPEG for months), I'm really interested in the stdin stream support and would like to work on it if possible, but i have very small knowledge of any native/wasm tools that this project uses, so if you could tell me where to start/what to look for i'd really appreciate it.

from ffmpeg.wasm.

@Banou26 Thanks for the explanation, it helps a lot. Right now looks like FS is the bottleneck for the whole thing to happen, I will think about it and see if there is a way to keep most of the features while not to use FS.

from ffmpeg.wasm.

@creativefctr sadly, the answer is no and I would say it is hard to achieve considering the limitation we have right now.

from ffmpeg.wasm.

This is an interested one, you can check the latest version (^0.8.1) to see if we can do it after the major refectoring.
I will take a look when available, thanks.

from ffmpeg.wasm.

Hey, sorry, sadly i cannot share my actual code as it's very project-specific, If/When i have more time i potentially wanna revisit it to make it an actual library to share with everyone.

Though what i can do now is give you a rough explanation on how it works.

You can basically retake this example https://github.com/leandromoreira/ffmpeg-libav-tutorial/blob/master/2_remuxing.c

And modify it in a way that it doesn't use the filesystem but in memory buffers pushed from javascript.

But most importantly, you need to replace the main loop, while(true) by something non blocking, so for example, you could make it process 5GOPs per javascript calls.
This is to leave the thread it's currently running on, time for javascript to interact with it, e.g push more buffers, get back the previously processed buffers, make more process calls, ect...

So in the end the API you end up with is something like this in javascript

const remuxer = (libavInstance ?? new libavInstance = await (await import('../dist/libav.js'))()).Remuxer(size)
remuxer.init(BUFFER_SIZE)

const process = () => {
  remuxer.clearInput()
  remuxer.push(buffer)
  remuxer.process()
  transmuxedBuffer.push(new Uint8Array(remuxer.getInt8Array()))
  remuxer.clearOutput()
}

process()

// waits for more data to download

process()

// ...ect

remuxer.close()

Which you can then conveniently turn into a Stream based API for ease of use.

Only problem with your project i can see, is that the entire codebase of FFMPEG is based around the fact that it expects to be run on a FS with the full video at its disposition, you could probably hack it into a way that it uses the remote fetching mechanism because this code probably has some IO waiting mechanism, but it still would require probably a LOT of hacking to make it work because of the nature of this project being based on the actual FFMPEG CLI codebase.

Hope this gave you ideas/a start on how you could potentially implement it for this project.
If you have any more questions i'd be happy to try to answer them as much as possible.

from ffmpeg.wasm.

If anyone ever wants a simple streaming remuxing lib for the web, i've just opened https://github.com/Banou26/libav-wasm (for now it simply remux mkv->mp4 with seeking and streaming support)
I'll try to clean it up a little bit, publish it on npm & add features if anyone actually wants it.

from ffmpeg.wasm.

Yep. i can certainly have a look.

from ffmpeg.wasm.

Just following up about this important feature. Is the last git can let us experiment it?

from ffmpeg.wasm.

This feature would be very helpful.

from ffmpeg.wasm.

@jeromewu any updates on this?

from ffmpeg.wasm.

Hello @jeromewu. The limitation you mentioned... is that lack of SharedArrayBuffer support in browsers? It seems that SharedArrayBuffer is available again. Is it now possible to add STDIN support? Or, is there some other limitation?

from ffmpeg.wasm.

Hmm, it is not SharedArrayBuffer that introduces this limitation. It is more like the current state of WebAssembly cannot handle stdin, so I cannot find a way to support stdin at the moment.

from ffmpeg.wasm.

Not sure if most people wanna get into this but you can pretty easily make a small LibAV C++ API compiled to WASM that deals with only computing when necessary to stream data in and out of WASM-land, this is obviously not as simple as this library's plug and play API but if you really want it you can manage to do smth with a little bit of work, I have almost no prior experience with C++ but still managed to make a transmuxer work with streaming in WASM.

from ffmpeg.wasm.

Hi @Banou26, that sounds cool! Would you mind sharing a working sample code that handles streaming? Thanks.

from ffmpeg.wasm.

I am planning to work on a live stream studio web app that allow our streamer to customise their stream with on the web app before they stream to Facebook. MVP require a few key features:

• Stream from Webcam
• Work in both web and mobile
• Add on screen banner that showcase product information
• Able to stream to Facebook

I am looking for ways to actually do it and is currently experimenting ffmpeg.wasm which bump into the problem of reading constant stream from webcam to ffmpeg before modifying it. Most of the solution online is recording a video and post process it but there weren't any reliable source for live streaming editing.

If anyone could shine me some light on how to achieve this would be greatly appreciated.

from ffmpeg.wasm.

I have a similar use case as @HamptonMakes in #141 , but to convert webm to mp4 container and subsequently transport it over WebSocket.

It seems like MEMFS could be used to simulate stdin/stdout, as shown in https://github.com/Kagami/ffmpeg.js#user-content-files .

@HamptonMakes @vKongv have you found a solution?

from ffmpeg.wasm.