serverlesspub / ffmpeg-aws-lambda-layer Goto Github PK

FFmpeg/FFprobe AWS Lambda layer

License: Other

Makefile 31.99% JavaScript 68.01%

ffmpeg-aws-lambda-layer's Introduction

FFmpeg/FFprobe for AWS Lambda

A Lambda layer containing a static version of FFmpeg/FFprobe utilities from the FFmpeg Linux package, compatible with Amazon Linux 2.x and Amazon Linux 1.x instances (including the nodejs10.x runtime, and the updated 2018.03 Amazon Linux 1 runtimes).

Usage

Absolutely the easiest way of using this is to pull it directly from the AWS Serverless Application repository into a CloudFormation/SAM application, or deploy directly from the Serverless Application Repository into your account, and then link as a layer.

The ffmpeg and ffprobe binaries will be in /opt/bin/ after linking the layer to a Lambda function.

For more information, check out the ffmpeg-lambda-layer application in the Serverless App Repository.

For manual deployments and custom builds, read below...

Prerequisites

Unix Make environment
AWS command line utilities (just for deployment)

Deploying to AWS as a layer

This package includes FFmpeg 4.1.3, packaged by John Van Sickle. Please consider supporting him for maintaining statically built FFmpeg packages. For more information, check out https://johnvansickle.com/ffmpeg/

The output will be in the result dir.

Run the following command to deploy the compiled result as a layer in your AWS account.

make deploy DEPLOYMENT_BUCKET=<YOUR BUCKET NAME>

configuring the deployment

By default, this uses ffmpeg-lambda-layer as the stack name. Provide a STACK_NAME variable when calling make deploy to use an alternative name.

example usage

An example project is in the example directory. It sets up two buckets, and listens to file uploads on the first bucket to convert and generate thumbnails from uploaded video files. You can deploy it from the root Makefile using:

make deploy-example DEPLOYMENT_BUCKET=<YOUR BUCKET NAME>

For more information on using FFmpeg and FFprobe, check out https://ffmpeg.org/documentation.html

Author

Gojko Adzic https://gojko.net

License

These scripts: MIT
FFmpeg: GPLv2.1 http://ffmpeg.org/legal.html, John Van Sickle's static build GPL v3 https://johnvansickle.com/ffmpeg/

LGPL version

Giuseppe Battista created a build that contains only LGPL components, for organisations that are concerned about GPL licensing. See it at https://github.com/giusedroid/ffmpeg-aws-lambda-layer/tree/license/lgpl

ffmpeg-aws-lambda-layer's People

Contributors

Stargazers

Watchers

Forkers

turneremeato suavepirate rahulsoibam fivetalent danylohetmantsev hemant-brb sromic atroche sachinahj ronyitc hadryan seafoodrice stvngrsh rasathus dlozinski flavioalves ajitsinghit2 code-is-art adic-idea samvelchan liuyezaici jirodrig-tec zeuslabs perugan andriygd tvkpz emilpicsio sidd47 krzemienski armonge tomeduarte teebu boxabirds alirezaasadi jgonext digitalvideosherpa khuongdv93 patrick91 eyesnareinc angelarw abest0 iqnupe mulxcode pudepang raedatoui capsulelp talc23 aditya-nx ch donpark adi87 hoai alycda giusedroid yongfook ronalcelaya franco1215 admariner cinnamonlabs biteable himanshi1483 withai-innovation motivq bbc kirschd jonemo tima-gt wan2land olxbr littlewat ai-communis jaredswarren zarr12steven mdarby comberry-development mohan252 lummm hopperhq lgndart dukeofbay alaincavel verisoul akrizs wooodhead teambeacon stanyq4 q4mobile jinzaizhichi ziczhu kevinwgutierrez makcanca enparadigm weilshen openreel zut2020 srgssr gildardoperez josh-1851 popeyesailormike thorndyke

ffmpeg-aws-lambda-layer's Issues

Support for current versions of Python

python 3.6 is end of life. To continue supporting python, support for more recent versions should be added.

Output video is significantly sped up

For the project I'm working on, I'm mainly just taking mp4s recorded by users in a web browser and trying to transcode them to be manageable sizes as well as playable in all device/browser combos. I was using AWS MediaConvert for this but later found out that mobile Safari records video in a "fragmented" mp4 which is not supported by MediaConvert so I'm turning to replacing MC with an ffmpeg layer.

I've deployed the ffmpeg lambda layer from the SAR and copied most of the code from the example but for some reason all the output videos are significantly sped up. When I run the same commands with my local ffmpeg, everything works fine. Any help is appreciated!

export const onVideoUpload: Handler = Sentry.AWSLambda.wrapHandler(async (event: S3Event, context: Context) => {
  const EXTENSION = '.mp4',
	  MIME_TYPE =  'video/mp4';

  const eventRecord = event.Records && event.Records[0],
		inputBucket = eventRecord.s3.bucket.name,
		key = eventRecord.s3.object.key,
		id = context.awsRequestId,
		resultKey = key.replace(/\.[^.]+$/, EXTENSION),
		workdir = os.tmpdir(),
		inputFile = path.join(workdir,  id + path.extname(key)),
		outputFile = path.join(workdir, 'out_' + id + EXTENSION);

	console.log('converting', inputBucket, key, 'using', inputFile);
  // ffmpeg -i 2254.webm -c copy output.webm
  // ffmpeg -i input.mp4 -b:v 1M -b:a 192k output.avi
	return downloadFileFromS3(inputBucket, key, inputFile)
		.then(() => childProcess(
			'/opt/bin/ffmpeg',
			['-i', inputFile, '-b:v', '1M', '-b:a', '192k', '-nostdin', outputFile],
			{env: process.env, cwd: workdir}
		))
		.then(() => uploadFileToS3(process.env['DestinationBucket']!, resultKey, outputFile, MIME_TYPE));
});

Error: You are not authorized to perform: lambda:GetLayerVersion

I created a lambda function (Node.js 10.x) and in the AWS web console added the following layer:
arn:aws:lambda:us-east-1:145266761615:layer:ffmpeg:4

When I try to save, I get the error :
"You are not authorized to perform: lambda:GetLayerVersion."

Is this layer made to be used from any AWS account?
What do I need to do to allow this function to use the layer?

Silently failing with fluent-ffmpeg

I am using this layer on one lambda function absolutely fine. But no matter what I try if I setup another lambda function with the exact same layers attached (my node_modules and this layer) it fails silently on any call. It just completely ignores the fluent-ffmpeg functions as if they are not there. I am completely baffled. I can see ffmpeg and ffprobe are there using fs readDirSync. I have tried deploying it as a second application so it gets a new arn and it still fails with 0 errors. Can anyone help me?

FFMPEG is truncating the audio files i'm trying to convert.

I'm trying to convert a .webm file to .mp3 running the following command within my lambda function.

os.system('/opt/bin/ffmpeg -acodec libopus -i /tmp/audioFile.{} -y -map 0 /tmp/output.mp3'.format(event['audioFormat']))

As long as the original audio file is bellow more or less 5 seconds, it gets transcoded correctly. However, if the file's longer, for some reason ffmpeg truncates it, and only convert the first seconds of the file. I've noticed that when truncating, the new file is always either 38.9, 39.4 or 39.8 KB in size.

Any Idea what might be going on???

Here are the logs:

ffmpeg version 4.1.3-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzvbi --enable-libzimg
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
Input #0, matroska,webm, from '/tmp/audioFile.webm':
Metadata:
encoder : Chrome
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0(eng): Audio: opus, 48000 Hz, mono, s16 (default)
Stream mapping:
Stream #0:0 -> #0:0 (opus (libopus) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
Output #0, mp3, to '/tmp/output.mp3':
Metadata:
TSSE : Lavf58.20.100
Stream #0:0(eng): Audio: mp3 (libmp3lame), 48000 Hz, mono, s16p (default)
Metadata:
encoder : Lavc58.35.100 libmp3lame
Truncating packet of size 669363911 to 31787
size= 40kB time=00:00:05.05 bitrate= 64.5kbits/s speed=31.9x
video:0kB audio:40kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.585012%

ffmpeg was killed with signal SIGSEGV at ChildProcess

I am getting this error on a particular ffmpeg command I am calling in lambda. I have various other commands that are working fine. I have copied the exact command I use from ECS where it works fine as well. In this particular function I am using fluent-ffmpeg to concat a series of clips from a txt file. Do you have any idea why I would get this error?

Subtitles doesn't work

I'm getting the below error

Fontconfig error: Cannot load default config file [Parsed_subtitles_0 @ 0x5c39300] No usable fontconfig configuration file found, using fallback. Fontconfig error: Cannot load default config file

I'm guessing libass is not enabled by default, any idea on how to resolve this?

Can't process .webm files

Currently I'm trying to convert a .webm file using this layer on my Lambda function, but it fails when it tries to fetch the probe of my input file.

Here's the error output:

2020-07-06 16: 27: 51.363 (-03: 00)	287bc4c2-75be-42e0-a3d5-39343ccb9793	ERROR	Unhandled Promise Rejection	{
    "errorType": "Runtime.UnhandledPromiseRejection",
    "errorMessage": "Error: ffprobe exited with code 1\nffprobe version 4.2.2-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2007-2019 the FFmpeg developers\n  built with gcc 8 (Debian 8.3.0-6)\n  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg\n  libavutil      56. 31.100 / 56. 31.100\n  libavcodec     58. 54.100 / 58. 54.100\n  libavformat    58. 29.100 / 58. 29.100\n  libavdevice    58.  8.100 / 58.  8.100\n  libavfilter     7. 57.100 /  7. 57.100\n  libswscale      5.  5.100 /  5.  5.100\n  libswresample   3.  5.100 /  3.  5.100\n  libpostproc    55.  5.100 / 55.  5.100\n[mpegts @ 0x7274c40] Format mpegts detected only with low score of 2, misdetection possible!\n[mpegts @ 0x7274c40] Could not detect TS packet size, defaulting to non-FEC/DVHS\n/tmp/c69847bd-9c0e-4f13-84ac-1a447939907e.webm: End of file\n",
    "reason": {
        "errorType": "Error",
        "errorMessage": "ffprobe exited with code 1\nffprobe version 4.2.2-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2007-2019 the FFmpeg developers\n  built with gcc 8 (Debian 8.3.0-6)\n  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg\n  libavutil      56. 31.100 / 56. 31.100\n  libavcodec     58. 54.100 / 58. 54.100\n  libavformat    58. 29.100 / 58. 29.100\n  libavdevice    58.  8.100 / 58.  8.100\n  libavfilter     7. 57.100 /  7. 57.100\n  libswscale      5.  5.100 /  5.  5.100\n  libswresample   3.  5.100 /  3.  5.100\n  libpostproc    55.  5.100 / 55.  5.100\n[mpegts @ 0x7274c40] Format mpegts detected only with low score of 2, misdetection possible!\n[mpegts @ 0x7274c40] Could not detect TS packet size, defaulting to non-FEC/DVHS\n/tmp/c69847bd-9c0e-4f13-84ac-1a447939907e.webm: End of file\n",
        "stack": [
            "Error: ffprobe exited with code 1",
            "ffprobe version 4.2.2-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2007-2019 the FFmpeg developers",
            "  built with gcc 8 (Debian 8.3.0-6)",
            "  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg",
            "  libavutil      56. 31.100 / 56. 31.100",
            "  libavcodec     58. 54.100 / 58. 54.100",
            "  libavformat    58. 29.100 / 58. 29.100",
            "  libavdevice    58.  8.100 / 58.  8.100",
            "  libavfilter     7. 57.100 /  7. 57.100",
            "  libswscale      5.  5.100 /  5.  5.100",
            "  libswresample   3.  5.100 /  3.  5.100",
            "  libpostproc    55.  5.100 / 55.  5.100",
            "[mpegts @ 0x7274c40] Format mpegts detected only with low score of 2, misdetection possible!",
            "[mpegts @ 0x7274c40] Could not detect TS packet size, defaulting to non-FEC/DVHS",
            "/tmp/c69847bd-9c0e-4f13-84ac-1a447939907e.webm: End of file",
            "",
            "    at ChildProcess.<anonymous> (/var/task/node_modules/fluent-ffmpeg/lib/ffprobe.js:233:22)",
            "    at ChildProcess.emit (events.js:198:13)",
            "    at ChildProcess.EventEmitter.emit (domain.js:448:20)",
            "    at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)"
        ]
    },
    "promise": {},
    "stack": [
        "Runtime.UnhandledPromiseRejection: Error: ffprobe exited with code 1",
        "ffprobe version 4.2.2-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2007-2019 the FFmpeg developers",
        "  built with gcc 8 (Debian 8.3.0-6)",
        "  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg",
        "  libavutil      56. 31.100 / 56. 31.100",
        "  libavcodec     58. 54.100 / 58. 54.100",
        "  libavformat    58. 29.100 / 58. 29.100",
        "  libavdevice    58.  8.100 / 58.  8.100",
        "  libavfilter     7. 57.100 /  7. 57.100",
        "  libswscale      5.  5.100 /  5.  5.100",
        "  libswresample   3.  5.100 /  3.  5.100",
        "  libpostproc    55.  5.100 / 55.  5.100",
        "[mpegts @ 0x7274c40] Format mpegts detected only with low score of 2, misdetection possible!",
        "[mpegts @ 0x7274c40] Could not detect TS packet size, defaulting to non-FEC/DVHS",
        "/tmp/c69847bd-9c0e-4f13-84ac-1a447939907e.webm: End of file",
        "",
        "    at process.on (/var/runtime/index.js:37:15)",
        "    at process.emit (events.js:198:13)",
        "    at process.EventEmitter.emit (domain.js:448:20)",
        "    at emitPromiseRejectionWarnings (internal/process/promises.js:140:18)",
        "    at process._tickCallback (internal/process/next_tick.js:69:34)"
    ]
}

When I run ffprobe in my machine, it works:

ffprobe "Best Camera for IGTV (9x16 vertical video examples).webm" 
ffprobe version 4.2.2-1ubuntu1~16.04.york0 Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.12) 20160609
  configuration: --prefix=/usr --extra-version='1ubuntu1~16.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, matroska,webm, from 'Best Camera for IGTV (9x16 vertical video examples).webm':
  Metadata:
    encoder         : google/video-file
  Duration: 00:01:45.44, start: 0.000000, bitrate: 3458 kb/s
    Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv, bt709), 1080x1920, SAR 1:1 DAR 9:16, 23.98 fps, 23.98 tbr, 1k tbn, 1k tbc (default)

Should I make any modifications in order for this to work? Installing additional plugins maybe?

Incorrect License

Hi Gojko et al,

I think you have the license wrong for this project. You've listed GPLv2, however you say you build on Johan Van Sickle's work and he states that he builds his ffmpeg binaries as GPLv3 packages.

I think your LICENSE file should note this.

Function code combined with layers exceeds the maximum allowed size.

Hello,
I just tried to run your cli with make deploy DEPLOYMENT_BUCKET={bucket_name}
When I try to associate the layer to a fresh new Lambda, I got the following message :
Function code combined with layers exceeds the maximum allowed size of 262144000 bytes. The actual size is 349762174 bytes.

Do you have any idea of how I could handle that ?

Is this can used for Node.js 12.x?

I tried to use this layer and I added it to my function, but when I check if there exists about file /opt/ffmpeg/ffmpeg, I got not exists. may I have to do anything else except to add layer to my lambda function?

Can you create layer with the newest Ffmpeg?

Your layer is used v4.1.3 that is fewer old, can you create a new layer with the newest ffmpeg static version?

Implementation Question

I am trying to just do a hello world test with this layer. I deployed to my account successfully and am attempting to use the layer in a python3.6 runtime.
I know this might not be appropriate for this forum but could you point me in the right direction for how to call the binary?

Unable to deploy with AWS-CLI

I've tried to deploy layer from my local machine.But while deploying I'm facing some issue.

Is nodejs.12 supported?

AWS is deprecating the nodejs.10 runtime.
Will this lambda layer keep working for lambda functions with nodejs.12 runtimes?

Difference between using ffmpeg binary file and this layer

If I just need to use ffmpeg, is there any difference between deploying lambda code with ffmpeg binary and using an ffmpeg layer?

ffmpeg burn chinese, japanese, korean and thai subtitle(srt) with error

When I burn the subtitles to video for chinese, japanese, korean and thai languages, it is not working properly. Boxes are coming instead of characters. Could you please look into it and try to resolve this issue.

Question: How would I export an MP3?

I'm aware this doesn't have the MP3 module, but how would I be able to implement that with yours? Or would I need to start from scratch?

Support more runtimes

The ffmpeg layer is currently limited to the following runtimes:

nodejs10.x
python3.6
ruby2.5
java8
go1.x

It is therefore not possible to use it with other runtimes like dotnetcore* or python3.8. Would it be possible to include more runtimes or remove the runtimes limitation? Since the binaries included in the layer are self contained, I guess that it could run in any runtime.