Git Product home page Git Product logo

rust.aws-cdk-lambda's Introduction

Amazon Lambda Rust Library


rust.aws-cdk-lambda: Stable npm

This is an unofficial CDK library based on the Amazon Lambda Node.js and aws-lambda-rust Libraries.

It's intended for use with the new AWS CDK v2.


This library provides a construct for a Rust Lambda function.

It uses cargo-lambda under the hood, and follows best practices as outlined in the official AWS documentation.

Rust Function

The RustFunction construct creates a Lambda function with automatic bundling and compilation of Rust code.

Getting Started

  1. Install this npm package, and zig:

    $ npm i rust.aws-cdk-lambda --save-optional
  2. Install the aarch64-unknown-linux-gnu toolchain with Rustup by running:

    $ rustup target add aarch64-unknown-linux-gnu
  3. Use cargo to install cargo-lambda:

    $ cargo install cargo-lambda

Examples

You can find sample CDK apps built using Typescript or Node.js in the cdk-examples/ folder of the GitHub project repo.

Usage

First, import the construct:

import { RustFunction } from 'rust.aws-cdk-lambda';

Now define a RustFunction:

new RustFunction(this, 'my-handler', {});

By default, the construct will use directory where cdk was invoked as directory where Cargo files are located.

If no bin or package argument is passed in, it will default to the package name as defined in the main Cargo.toml.

That is, the above usage should work for a project structure that looks like this:

.
├── Cargo.toml
└── src
    └── main.rs

Alternatively, directory and bin can be specified:

new RustFunction(this, 'MyLambdaFunction', {
    directory: '/path/to/directory/with/Cargo.toml',
    // Optional
    bin: 'my_lambda',
});

All other properties of lambda.Function are supported, see also the AWS Lambda construct library.

How It Works

When bundling the code, the RustFunction runs the following steps in order:

  • First it runs cargo lambda, and passes in the --release and --target flags, so it compiles for a Lambda environment - which defaults to the aarch64-unknown-linux-gnu target, as mentioned above. Note that cargo lambda does initially confirm that the Rust code can compile.

  • The directory path to the executable is then passed in to lambda.Code.fromAsset, which creates a zip file from the release binary asset.

Use cross for Deployment

If you instead prefer to use Docker and cross for deployment, as outlined in the official AWS docs, you can install and use the latest v0.x release instead:

$ npm i [email protected] --save-optional

Multiple Rust Lambdas

Assuming you have a CDK project with more than one Rust lambda, there are a couple approaches - as outlined below - that you can use to deploy with cdk.

Multiple Binaries

Suppose your project layout looks like this:

.
├── Cargo.toml
└── src
    └── bin
        ├── lambda1.rs
        └── lambda2.rs

Here's one way to deploy that via cdk:

new RustFunction(this, 'my-function-1', {
    bin: 'lambda1',
});

new RustFunction(this, 'my-function-2', {
    bin: 'lambda2',
});

You can find a more complete project structure in the rust-bins/ CDK sample project.

Multiple Packages

Suppose you use Workspaces in your Cargo project instead.

The full contents of the main Cargo.toml would need to be updated to look like this:

[workspace]
members = [
    "lambda1",
    "lambda2"
]

And your new project layout would now look similar to this:

.
├── Cargo.lock
├── Cargo.toml
├── lambda1
│   ├── Cargo.toml
│   └── src
│       ├── main.rs
│       └── utils.rs
└── lambda2
    ├── Cargo.toml
    └── src
        ├── main.rs
        └── utils.rs

Where the utils.rs files are optional, but the point is that they can be imported by the lambda handler code in main.rs if desired.

Now you will only need to update your CDK code to pass package instead, for each workspace member:

new RustFunction(this, 'MyFirstRustFunction', {
    package: 'lambda1',
});

new RustFunction(this, 'MySecondRustFunction', {
    package: 'lambda2',
});

You can find a more complete project structure in the rust-workspaces/ CDK sample project.

Conditional Compilation

A common use case is building Rust code with enabled features, and compile- time environment variables that can be used with the env! macro.

For example, we might want to run different logic in our code for development and production environments, or call a different API endpoint depending on which environment we are deploying code to.

For a sample CDK app that demonstrate such usage, check out the rust-bins/ example.

Enabling Features

You can achieve conditional compilation by introducing features which can later be enabled in Rust code.

In the Cargo.toml, create a new features section:

[features]
my-feature = [] # feature has no explicit dependencies

In your code, add the line #[cfg(feature = "my-feature")] before a function declaration, or before a statement to execute.

In your CDK code in the lib/ folder, add the following line:

// Enable features at compile or build time.
Settings.FEATURES = ['my-feature'];

Build Environment Variables

You can also introduce environment variables which are resolved at build or compile time. These values can be used in code via the env! macro in Rust.

In your code, add a call to the env!() macro:

// Retrieve an environment variable set at build (compile) time.
const BUILD_VALUE: &str = env!("MY_BUILD_VAR");

In your CDK code in the lib/ folder, add the following line:

// Enable environment variables at compile or build time.
Settings.BUILD_ENVIRONMENT = {
    MY_BUILD_VAR: 'Hello World! Testing 123.',
};

Rust Function Properties

Below lists some commonly used properties you can pass in to the RustFunction construct.

Name Description
target Build target to cross-compile to. Defaults to the target for the arm64 architecture, aarch64-unknown-linux-gnu.
directory Entry point where the project's main Cargo.toml is located. By default, the construct will use directory where cdk was invoked as the directory where Cargo files are located.
buildDir Default Build directory, which defaults to a .build folder under the project's root directory.
bin Executable name to pass to --bin
package Workspace package name to pass to --package
setupLogging Determines whether we want to set up library logging - i.e. set the RUST_LOG environment variable - for the lambda function.

The format defaults to warn,module_name=debug, which means that the default log level is warn, and the executable or library's log level is debug.
features A list of features to activate when compiling Rust code. These must also be added to the Cargo.toml file.
buildEnvironment Key-value pairs that are passed in at compile time, i.e. to cargo build or cargo lambda. This differs from environment, which determines the environment variables which are set on the AWS Lambda function itself.
extraBuildArgs Additional arguments that are passed in at build time to cargo lambda. For example, [--all-features].

Settings

Settings can be imported as follows:

import { Settings } from 'rust.aws-cdk-lambda';

Below are some useful global defaults which can be set for all Rust Lambda Functions in a CDK app.

Name Description
BUILD_INDIVIDUALLY Whether to build each executable individually, either via --bin or --package.
DEFAULT_LOG_LEVEL Log Level for non-module libraries. Note that this value is only used when RustFunctionProps.setupLogging is enabled. Defaults to warn.
MODULE_LOG_LEVEL Log Level for a module (i.e. the executable). Note that this value is only used when RustFunctionProps.setupLogging is enabled. Defaults to debug.
WORKSPACE_DIR Sets the root workspace directory. By default, the workspace directory is assumed to be the directory where cdk was invoked.

This directory should contain at the minimum a Cargo.toml file which defines the workspace members.
FEATURES A list of features to activate when compiling Rust code. These must also be added to the Cargo.toml file.
BUILD_ENVIRONMENT Key-value pairs that are passed in at compile time, i.e. to cargo build or cargo lambda. This differs from environment, which determines the environment variables which are set on the AWS Lambda function itself.
EXTRA_BUILD_ARGS Additional arguments that are passed in at build time to both cargo lambda. For example, [--all-features].
SKIP_BUILD Whether to skip building Rust Lambdas -- e.g. skip the compile step with cargo lambda.

Alternatively, this can be set in the build context, when invoking cdk -- for example:
$ cdk synth -c build=[T | true | 1 | Y | yes | ok | on]
Any values other than the listed evaluate to false. So, this would skip the build step on synth:
$ cdk synth -c build=0

rust.aws-cdk-lambda's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

rust.aws-cdk-lambda's Issues

Use cargo-lambda for compilation

Hi 👋

This is a really cool project, thanks for putting it together!

I wanted to let you know about this other project I started a few weeks ago and we recommend for people using the Rust runtime for Lambda:

https://github.com/calavera/cargo-lambda

It abstracts away cargo-zigbuild, and also installs Zig for people automatically. It would allow you to reduce the code necessary to compile and locate functions significantly. Check it out and let me what you think about it. I can also help you integrate it if you're interested.

openssl when cross compiling

It seems that some crates rely on native openssl. Building on my M1 mac I receive an error that the openssl cannot be found. I assume it is because I am building for Linux. Did you encounter this problem and how did you manage to solve it?

Unhelpful error thrown when cargo is missing

TypeError: Cannot read properties of null (reading 'toString')

This rather cryptic error is thrown from:

console.error(cargo.stderr.toString().trim());

... when trying to call cargo.stderr.toString(). The reason is that cargo.stderr may be null if a cargo executable was not found by the call to spawnSync('cargo' /* ... other args... */).

Troubleshooting:

If, when you run which cargo in your usual shell, you get cargo not found or similar, then you need to make sure the cargo executable is both installed and on your shell's path.

Recommendations:

I think ideally this library should have some way of double-checking whether cargo is indeed on-path (I'm not sure what the usual approach for this is), but this kind of environment health-check may arguably be out-of-scope.

At the very least, it would save some onboarding friction if this library could, in the event of cargo.stderr being null, suggest that perhaps cargo is either not installed or not on path.

Hardcode or specify a glibc version for compilation

This issue derives from an understandable concern in this Reddit thread. Essentially it can be summed up as follows:

[...] I'm just worried that this is just working so far by chance, because if the aarch64-unknown-linux-gnu default glibc version ever increases to a version higher than the AL2 glibc then everything will stop working.

So basically, in our case it would be a good idea to specify the glibc version - when running zigbuild at least, I'm not sure if cross supports it.

It might be worth it to hardcode glibc=2.26, as I believe that's what the docs on amazon linux 2 seem to indicate is the version of glibc that the AL2 runtime uses.

Resolution

As suggested in the same Reddit thread, we easily specify glibc version when running zigbuild - for example, to compile for glibc 2.26 (which is what AL2 uses) with the aarch64-unknown-linux-gnu target:

cargo zigbuild --target aarch64-unknown-linux-gnu.2.26

Don't build on synth

Hey, it looks like RustFunction is trying to execute cargo lambda upon synth, which in my opinion shouldn't happen. I have multiple stacks and don't want to compile the rust function every time I deploy one of the stacks (most don't include rust functions)

Local dependencies

I'm using local dependencies in my Lambda cargo.toml.
Like this:

my_tools = { path = "../../shared/rust/my_tools/" }

And it seams like rust.aws-cdk-lambda does not likes it. cdk synth fails with an error

Error: Cannot find asset at /Users/....

Meanwhile cargo lambda build --output-format zip works well.

Would appreciate any workaround.

cdk destroy shouldn't rebuild the lambda?

Since the whole stack is about to be deleted, there's no need to rebuild it, right?

% cdk destroy
🍺  Building Rust code with `cargo lambda`. This may take a few minutes...
🎯  Cross-compile `s3-rust-noodles-bam`: 119.11s
🎯  Cross-compile `apigw`: 0.00s
Are you sure you want to delete: s3-rust-noodles-bam (y/n)?

buildEnvironment does not pass correct params to build process

Using the latest build 1.2.1, it seems that specifying buildEnvironment as a param to multiple RustFunction instances causes only the first passed env vars for the first hit RustFunction instance to be passed for all following creations of RustFunction.

Minimum example:

const foo_lambda = new RustFunction(this, 'foo_lambda', {
	directory: path.resolve(path.join(__dirname, '../lambda')),
	package: 'foo',
	buildEnvironment: {
		FOO_ENV: 'foo',
	},
})

const bar_lambda = new RustFunction(this, 'bar_lambda', {
	directory: path.resolve(path.join(__dirname, '../lambda')),
	package: 'bar',
	buildEnvironment: { 
		BAR_ENV: 'bar' 
	},
})

With the rust code doing e.g.:

const FOO_ENV: &str = env!("FOO_ENV");
const BAR_ENV: &str = env!("BAR_ENV");

I'm getting the error when compiling the second rust functionenvironment variable "BAR_ENV" not defined, however - the FOO_ENV is available for the second function. If reversing the order of the two RustFunctions, now BAR_ENV is available, and compilation will fail with FOO_ENV not defined.

This issue can be worked-around by building each package individually by doing:

import { Settings } from 'rust.aws-cdk-lambda'
Settings.BUILD_INDIVIDUALLY = true

Add an option to compile code in a Docker container

The goal here is quite simple, but ideally we want to provide a way for users to compile rust code in a containerized manner, perhaps leveraging Docker and an Amazon Linux build image. If the host machine doesn't have rustup or cargo-zigbuild installed, we should by default use Docker to build and compile rust code.

The end goal would probably be to find a suitable Docker container, so if the container for example has zigbuild and zig already installed, that would be ideal; though of course if the architecture in the Docker container matches the target, then there's a chance we might not need zigbuild in such a case.

We could also take inspiration from other similar NPM libraries such as aws-lambda-go, which does also support building code in a Docker environment.

Docker Image

For the actual Docker image to use, I'm still open to suggestions as mentioned above, so ideally an image that comes with tools like cargo-zigbuild and cross already installed. I think an excellent starting point would be to look into the image that softprops/lambda-rust provides:

https://github.com/softprops/lambda-rust

How can I test locally using `sam local start-lambda` with this setup?

I've created a repo using the setup in cdk-examples/rust-standalone.

I want to do local testing (run this lambda as a server on localhost and make HTTP GET requests to it), so I've tried running this:

sam local start-lambda --port 3000 --template ./cdk.out/RustStandaloneStack.template.json

The server successfully starts up, but I can't figure out what sort of request to send it. It just responds with a 404 no matter what path or headers I specify to it with my GET requests.

By comparison, one-off invocations work just fine:

# ./tests/sample.json is an event matching the same format of the one seen in this issue comment:
# https://github.com/awslabs/aws-lambda-rust-runtime/issues/365#issue-1054795934
$ sam local invoke MyRustLambda --event ./tests/sample.json --template ./cdk.out/RustStandaloneStack.template.json

Requesting using the exact same path and headers specified in my ./tests/sample.json file, I am met with a 404 response.

Usage with the SAM CLI may be out of scope, but I was hoping you might happen to know!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.