https://aws.amazon.com/blogs/compute/java-17-runtime-now-available-on-aws-lambda/

AWS Labs has a new "Low Latency Runtime" in beta with some pretty impressive cold start times: https://github.com/awslabs/llrt

AWS just released this: https://aws.amazon.com/fr/blogs/compute/introducing-the-amazon-linux-2023-runtime-for-aws-lambda/
Update is comiiiiing! 🔥

We're recommending people deploy .NET 8 Native AOT projects to the .NET 8 managed runtime (instead of Al2 or AL2023). Since it will have the needed libraries available (like libicu). It also removes the need for the assembly name requirement of bootstrap and brings billing inline with managed runtimes.

PR #935 added AL2 and AL2023.

Can we replace provided.al2 with dotnet8?

This looks like a really interesting project, but when I cloned it, it wasn’t really clear what I needed to do to build and deploy it so that I could try to add a new target.

It would be really handy to have a “Getting Started” guide for developers that describes the tools needed and the steps to build and deploy.

Thanks!

First off, this is a useful tool. Thanks for setting it up.

Mean is a useful metric. But to get a feel of how much the cold starts can vary from the mean, it would be nice to have p95 and standard deviation (assuming normal distribution here) of the measurements.

This would be awesome if we could compare the cold start times of container vs zip

Given all the recent hype around the launch of bun v1, it would be awesome if the official bun Lambda runtime could be added to the benchmark suite:

• https://github.com/oven-sh/bun/tree/main/packages/bun-lambda
• https://medium.com/@mitchellkossoris/serverless-bun-vs-node-benchmarking-on-aws-lambda-ecd4fe7c2fc2

This is particularly interesting as Bun is supposed to index highly in start-up speed, but from the blog post above, it appears that the lambda cold start time is currently slower than Node.

There’s a comment in llrt’s repo, where llrt compile is mentioned to convert JavaScript to QuickJS bytecode prior to deployment.

This should decrease the cold start a bit.

Please add support for Dart. This post on the AWS Open Source blog seems to cover how.

It would make more sense to compile the Java workload with GraalVM native image, cold start with JVM make no sense for this kind of use case.

I guess that's similar to #2 raised for .Net.

Please include benchmark where you don't re-deploy the lambdas.

The reason is I observe (using Log insights) sometime Lambda that has been idle for long time (more than 30 minutes) have higher cold starts than a freshly deployed one.

Great suggestion from @lmammino
(ref: https://twitter.com/_maxday/status/1756363285140795869)

Hi!

Thanks a lot for the up-to-date information regarding lambda cold start time.

Did you think about adding more cloud providers like Azure, GCP, etc? This information could be useful for the users as well.

Thanks in advance!

I've seen good improvements for Lambda functions written in Java using tiered compilation, as described in this article:

https://aws.amazon.com/blogs/compute/increasing-performance-of-java-aws-lambda-functions-using-tiered-compilation/

Even just using TieredStopAtLevel=1 my team has seen the cold start times cut in half for many of our Lambda functions. I do think this is something that has more of an affect when you have more initialization code though, so it might not provide as much benefit in this package, since the code in the lambda is minimal. It might be worth a try, and would be a good data point.

I have not measured the results for the combination yet, but Tiered Compilation can also be combined with SnapStart.

I understand rust and go enjoying fast cold starts like:

But what could be the reason behind C++ being this slow on the same runtime?

Does it have to do with Rust and Go binaries being built with the standard tools? cargo and go?
While the C++ uses an external build tool like aws-lambda-cpp?

Or is Lambda just less optimized for C++ given it's not used as much?

For now cpp builds are cloning https://github.com/awslabs/aws-lambda-cpp
It could be great to auto-dectect releases from https://github.com/awslabs/aws-lambda-cpp/releases so we could rebuild as soon as a new release is cut.
For now dependabot does not support that (see: dependabot/dependabot-core#2027)

So maybe one solution would be to create a simple bot posting an issue for every new release of awslabs/aws-lambda-cpp so we could rebuild as soon as possible

cc @marcomagdy

NodeJS 20 has been released! https://aws.amazon.com/blogs/compute/node-js-20-x-runtime-now-available-in-aws-lambda/

Happy to make this contribution.

Would it be possible to add Haskell?

I assume the setup would be similar to the one of Rust on provided.al2. I've used https://theam.github.io/aws-lambda-haskell-runtime/ in the past and maybe could provide some help to integrate a Haskell setup.

Quarkus native build lambda sometimes fails with the following error message

It seems that we need to add quarkus.native.additional-build-args=-march=compatibility to application.properties according to https://github.com/quarkusio/quarkus/wiki/Migration-Guide-3.2#native-compilation---work-around-missing-cpu-features

By CSS only, according by media query prefers-colos-scheme.

I think it would be just the coolest thing if someone would contribute a function running Java 11 Corretto with and without SnapStart.

Love this project!! 😍

For now it seems that the layer arm64 is not working, let's add this back when the layer will be fixed

Mojo promises Python syntax with the speed of C.
Does that sound too good to be true? Yes, but let's see how well it does.

Maybe thats a non issue, but I am not sure about the goal of the comparsion between different java runtimes.

Following questions popped up.

1. Runtimes prefixed with java avoid using com.amazonaws:aws-lambda-java-core (which is listed as a required lib in Building Lambda functions with Java). In real world scenarios you most likely use this library because of that it would e a bit slower. Is this the desired solution for a comparison? Related PR: #410
2. The runtime graalvm java17 (prov.al2), which is currently the fastest java runtime in the comparsion, has a similar optimization by using an alternative java lambda runtime formkiq/lambda-runtime-graalvm instead of the aws java-runtime-interface-client. I think this should reflect in the name of the runtime, something like graalvm java17 (prov.al2, formkiq runtime).
3. quarkus (prov.al2) name should include the java version (17 in this case) e.g. quarkus java17 (prov.al2)
4. quarkus (prov.al2) should use the same optimizations (or not) like the other java runtimes (mentioned in 1).

I can split this to different issues, if you think its worth fixing them

Is your feature request related to a problem? Please describe.

Not a problem, but while going through the source code, I noticed that DocumentClient is used from AWS SDK for JavaScript (v2), while all other clients are from v3.

• DocumentClient

  lambda-perf/report-log-processor/app.js

  Line 2 in 8e8853a

  const DocumentClient = require("aws-sdk/clients/dynamodb").DocumentClient;

• DynamoDBClient

  lambda-perf/result-builder/app.js

  Line 2 in 8e8853a

  const { DynamoDBClient, ScanCommand } = require("@aws-sdk/client-dynamodb");

Describe the solution you'd like

Use aws-sdk-js-codemod to use DocumentClient from v3, and remove dependency on v2

Describe alternatives you've considered

N/A

The version is currently hardcoded, we need a way to get the latest one automatically.
In addition, there is a specific binary for 'image' deployment, we need two different artifacts deployed on s3 for llrt, one for zip, one for image

In theory al2023 should be a better and faster base image now than al2. Would love to see the native builds replaced or added complimentary to the existing al2 builds.

Saw https://mikhail.io/serverless/coldstarts/aws/ and thought it'd be neat to have an up-to-date reference for how quickly docker-based images spin up (and if they're still relatively slow)

Bun released a custom runtime AWS Lambda layer. Here is the actual PR.

It would be awesome to see how Bun custom runtime compares to Rust 😅

I don't think the current updating function configuration trick makes it so your tests are actually using snapstart based on the numbers. It possibly takes some time before AWS has made the snapstart feature enabled on newly updated functions

Could the cold start invoke frequency be updated to at least every 30 min for cold starts?
Looks like the code is getting 10 cold start samples daily for each language/platform but the numbers I have seen are much longer if hit frequency is at least 30 min apart for each of the invocations.

I have impressive cold start time with the latest JDK 21 release. Unforutnately building the native image with docker gives worse performance. I built directly in t4g large instance. Since the build and runtime environment are same I believe it is optimized for the processor. I want to add my lambda here if possible. Please let me know.

t4g large : 9.34ms
Docker    : 234ms

I believe Lambda now supports Python 3.10

https://aws.amazon.com/about-aws/whats-new/2023/04/aws-lambda-python-3-10/

Thanks for the awesome work!

If you can provide a Bar chart for an easy comparison view for each metric, that would be great in easily comparing and finding the best platform / architecture performance.

Basically i want to see the average cold start for all platforms and both architectures all in 1 chart against each other for easy visual comparison.

Amazing work developing and releasing this benchmark for all Lambda consumers like me. I think the frontend of the app could really benefit from some quick "sorting" capabilities.

E.g. sort by "cold start time" or "memory".

I know the title is "cold start" but it would be great to have the same for warm start, so that we can see when functions are warmed, if the ranking is the same. Especially Java which should be much better then.

for now the runtimes array is at 3 different places, let's refactor it at one place :)

Would be interesting to see results with .NET 7 Lambda functions with Native AOT compilation (also with provided.al2 runtime)
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/build-dotnet7.html

to demystify the believe in x10 performance boost - https://twitter.com/maloo321/status/1619788488316170240

LLRT (Low Latency Runtime): Amazon's New JS Runtime — Amazon has open sourced a runtime entirely focused on fast startups – something that’s useful for ephemeral use cases like serverless (cough AWS Lambda). It uses Fabrice Bellard’s QuickJS as the underlying engine, so has almost full ES2023 spec support out of the box.