Comments (6)
Hi Brad,
At the moment we are considering a fairly big rewrite of the transcoder, and this is one of the things that has crossed our minds. What's holding me back at the moment is the many different object storages we would have to support, each with it's one backing library and API plus the added complexity of having to deal with network transfers.
As an alternative at this moment, you might want to consider mounting your object storage directly on your system. Many providers have some form of FUSE support or similar. This removes the need for the transcoder to concern itself with these transfers and the file system driver can focus on transfers and storage.
from codem-transcode.
@tieleman Thanks for the super-quick reply! And thanks for the work on codem, it's great. I was actually going to go the FUSE route and I have a proof-of-concept working, however I'm looking at deploying on a cloud provider that doesn't provide sufficient privilege control over the container to allow mounting the device.
I actually do have access to such an environment but given that transcoding is so elastic in its resource needs, I'm looking to deploy in an environment that charges by actual usage, rather than having to spin up nodes that mostly sit idle between jobs. (I'm looking at IBM Bluemix, specifically.)
Regarding the various types of file systems to support, that pkgcloud library looks rather promising as it has support for many of the common providers, which is why it caught my eye. I'll follow-up here if I end up going that route. Regardless, support for either object stores natively or perhaps just a pre/post job hook API would be awesome and get closer to a true microservices type approach.
from codem-transcode.
I'll be sure to look at pkgcloud, thanks for the pointer. There is a callback url though that you can specify to the transcoder job that allows you to receive a notification when a job finishes, maybe that's of use to you. You'd need a small HTTP service that accepts those callbacks. You could use that for a post hook.
from codem-transcode.
FYI @tieleman I'm working on this here - not sure if you'd be looking for a PR since you're considering a major refactor but thought I'd mention it here in case anyone else was interested.
from codem-transcode.
@bradjones1: Thanks for the PR and the heads up. At the moment I'm going to leave it dangling, but I'll keep your code in mind when we initiate the rewrite. Will your fork remain public? One of the things that still worries me is the fact that FFmpeg can have so many different input/output configurations ("plain" files, segmented files, multiple output formats, etc.) that it will be hard to support such a thing completely.
from codem-transcode.
@tieleman Yes I'll keep it public and thanks for the response. I attacked the input/output question by tar/gzipping output directories on segment/thumbnail jobs, but you're right, it's a complicated question. Regardless, I imagine others might be interested which is why I put it on here. Cheers.
from codem-transcode.
Related Issues (20)
- Is it possible to generate multiple bit-rate as well as corresponding master m3u8 in a single job? HOT 8
- Idea: Incorporate provision for AWS Lambda HOT 2
- node-sqlite3 is throwing error, does it affect Transcodem? HOT 1
- Database configuration using port isn't working HOT 2
- /jobs/:id is MUCH slower than /jobs HOT 5
- Transcoder sometimes doesn't clear jobs from memory correctly
- Transcoding file streams HOT 1
- External mysql2 connection HOT 4
- Init script with codem-transcode and CentOS 7 not working HOT 7
- How to add file input name in the email report
- changing sqlite3 to mysql HOT 1
- Install depends on old packages and non-existing too HOT 3
- PID file
- -ss and or -t should be before -i HOT 2
- ffmpeg stderr buffer size HOT 1
- Generate SPRITE image for thumbnail HOT 7
- v.1.0-alpha: job failed processing with bunyan usage error HOT 4
- cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc
- hướng dẫn tạo video lên top google
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from codem-transcode.