consider replacing rm -rf with rimraf for crossplatform compatibility

> rm -rf dist && pnpm package-cjs && pnpm package-esm && pnpm package-types && pnpm package-types-aliases

 ELIFECYCLE  Command failed with exit code 1.
'rm' is not recognized as an internal or external command,
       operable program or batch file.

When we generate a project with Swarmion, it needs to have a CD

As shown in: https://dev.to/oieduardorabelo/top-level-await-in-aws-lamba-with-typescript-1bf0?utm_source=newsletter&utm_medium=email&utm_content=offbynone&utm_campaign=Off-by-none%3A%20Issue%20%23198

We can easily have top-level await support with the following esbuild config:

format: "esm"
outputFileExtension: ".mjs"
platform: "node"
target: "esnext"
banner:
  js: import { createRequire } from 'module';const require = createRequire(import.meta.url);

Adding it as a default will definitely be an improvement.

Caveat 😖

It might not work with aws-sdk v2:
We get the following error:

Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'aws-sdk' imported from

The AWS SDK for JavaScript v2 is imported with the name aws-sdk (non-scoped). This is available in AWS Lambda, but only with require.
aws/aws-sdk-js-v3#3230 (comment)

Indeed, if there are import statement in the source code using aws-sdk, serverless-esbuild will let them pass as we excluded aws-sdk. Thus there are lines import ... from 'aws-sdk' in the lamda code, which will not work as stated in the quoted comment.

Possible solution 💡

Use https://github.com/floydspace/serverless-esbuild#esbuild-plugins to rewrite import ... from 'aws-sdk' statements to const ... = require('aws-sdk').

When launching yarn deploy at the root of the package, the build target isn't launched in the user-docs/documentation package.

We need to fix this.

Hello everyone,

On our project, we have some mandatory configuration requirements applied to any serverless service we generate.
In our precise case, we need to have some serverless plugins, as well as some custom keys.

The problem is we need manual intervention for this on any service we generate, leaving room for manual error.

The ideal way to go about it would be to extend what is done with sharedProviderConfig and sharedParam to allow us to have default values in any key of the serverless configuration object, existing and generated in the future.

To avoid the serverless object to become an unpacking hell, we could use something like a lodash merge before exporting the configuration.

import { sharedConfiguration } from '@myProject/serverless-configuration` // ⬅ sharedProviderConfig, sharedParams are all here now

const serverlessConfiguration: AWS {
    ...
};

const exportedConfiguration = lodash.merge(serverlessConfiguration, sharedConfiguration)

module.exports = exportedConfiguration

An alternative would be to allow the user to override the template, but any change would have to be propagated to all existing services, which could be very cumbersome on an older project. It also goes away from the highly opinionated stance of swarmion so far.

Some links are broken in the documentation

This seems to happen only in prod, for links within the same folder

Le linter est cassé sur main pour le package serverless-helpers:

Hi all,
On the subject of serverless contracts, I've struggled with a cryptic issue 🤷.

Problem: getFetchRequest in @swarmion/serverless-contracts sets POST requests' Content-Type header as text/plain. When using getLambdaHandler to write a lambda handler, middy middleware enforces and validates application/json Content-Type header.
On the frontend, when I make a POST request to my lambda with the above setup, it errors out. The resulting error does not give information over the problem.
Reference: https://www.swarmion.dev/docs/how-to-guides/use-serverless-contracts/api-gateway

I've struggled to understand clearly the difference between getLambdaHandler and getHandler. I've come to understand getHandler implements middleware of its own and accepts text/plain as a content-type header.

Quick Solution:

• When using getFetchRequest to be able to type-safely call a lambda that uses serverless contracts, I've come to understand one MUST use getHandler helper from @swarmion/serverless-helpers.

Long-term solution:

• Recommend using getHandler in 99% of cases.
• Add application/json Content-Type header in getFetchRequest from @swarmion/serverless-contracts.

Thanks for your time:)!

PS: Using serverless-contracts feels really good. Type-safety FTW! 🚀 🫡

In order to provide a useful tool to developers, we need to clearly explain our architectural choices and guide users.

I have drafted a first structure, this is open for suggestions!

Tutorials

• Generate your first Swarmion app
• Get started on AWS
• Add your first Serverless service
• Add your first shared Typescript library
• Add your first frontend
• Create your first contract between the frontend and the backend

How-to guides

• Add authentication to your Serverless contracts
• Managing package upgrades with Swarmion
• Generating libraries with Swarmion
• Generating Serverless services with Swarmion
• Migrating from a monolith to Swarmion
• Share variables between your services
• Configuring your CI/CD
  • Github actions
  • CircleCI
  • GitlabCI

Explanation

• Why use a microservices approach?
• Why use a monorepo?
• Why use yarn3 and workspaces?
• Why share code between services?
• Why build Typescript packages?
• Serverless bundling → from TS to JS (why esbuild)
• Why use Nx as a monorepo management tool?
• Understanding configuration composition
  • Typescript
  • Eslint
  • Jest
  • linstaged
• Understanding the Swarmion VSCode configuration
• Why check for circular references?
• Why use a precommit hook?
• Understanding Serverless contracts
  • Types of contracts
  • Concepts
  • Why are they using functionnal programming?
• Miscellaneous tools
  • Why Fresko?
  • Why synckpack?
  • Why Commitlint?

References

• @swarmion/serverless-contracts
• @swarmion/nx-plugin
• @swarmion/serverless-plugin
• @swarmion/nx-plugin
• Code structure
  • Monorepo structure
  • Root config files
  • Serverless service structure
  • Package structure
  • Frontend shared package structure
  • Frontend structure

The @swarmion/serverless-contracts package currently exports two functions to build the lambda handler of an ApiGatewayContract:

• getLambdaHandler which only adds the type inference but is bound to be used with middlewares as it does not handle either parsing not validation of the input
• getHttpLambdaHandler introduced in #185 which handles parsing and validation

The second function is far better as it can be used on its own (without any middleware). However getLambdaHandler is a more generic name and in the future we can use it for other types of contracts (using overloading). This is why the naming of getHttpLambdaHandler seems a bit too specific.

I order to fix this, either:

• rename getLambdaHandler to getLambdaHandlerType and getHttpLambdaHandler to getLambdaHandler
• keep getLambdaHandler and rename getHttpLambdaHandler to getHandler

The second options seems the best to me, as it is more concise, and it matches the getTrigger function. WDYT?

⚠️ This will introduce a breaking change, so we should do it before releasing V1.0.0

When generating a new Swarmion App, the generated root package.json does not contain a lint-fix script. Therefore the husky hook will fail when editing root configuration files.

• Add a lint-fix target in examples/swarmion-starter/package.json similar to what exists in Swarmion root package.json

Since https://github.com/swarmion/swarmion/releases/tag/v0.8.2, the service generator of @swarmion/nx-plugin is broken. It generates a package with a dependency to a no longer existing local serverless-helpers package.

yarn generate-service myService 

>  NX  Generating @swarmion/nx-plugin:service

CREATE backend/my-service/.dependency-cruiser.js
CREATE backend/my-service/.eslintrc.js
CREATE backend/my-service/.lintstagedrc.js
CREATE backend/my-service/functions/health/config.ts
CREATE backend/my-service/functions/health/handler.mock.json
CREATE backend/my-service/functions/health/handler.test.ts
CREATE backend/my-service/functions/health/handler.ts
CREATE backend/my-service/functions/index.ts
CREATE backend/my-service/jest.config.ts
CREATE backend/my-service/serverless.test.ts
CREATE backend/my-service/serverless.ts
CREATE backend/my-service/package.json
CREATE backend/my-service/tsconfig.json
CREATE backend/my-service/project.json
UPDATE workspace.json
UPDATE my-app.code-workspace
➤ YN0027: @my-app/serverless-helpers@unknown can't be resolved to a satisfying range
➤ YN0035: The remote server failed to provide the requested resource
➤ YN0035:   Response Code: 404 (Not Found)
➤ YN0035:   Request Method: GET
➤ YN0035:   Request URL: https://registry.yarnpkg.com/@my-app%2fserverless-helpers

➤ Errors happened when preparing the environment required to run this command.
Command failed: yarn workspace       @my-app/backend-my-service       add --cached @my-app/serverless-configuration @my-app/serverless-helpers

To fix it, we must change the dependency to the now remote one: @swarmion/serverless-helpers

As of lately, we have started getting warnings on our Github Actions Workflows

#287 already tackled some of these warnings. However, some still remain:

• aws-actions/configure-aws-credentials is still using Node12, see aws-actions/configure-aws-credentials#489
  
• pnpm/action-setup is still using Node12, should be fixed by pnpm/action-setup#56 and therefore by #288
  
• nwtgck/actions-netlify is still using Node12, see nwtgck/actions-netlify#837
  
• Deprecation of the set-output command
  
• Deprecation of the save-cache command, maybe due to pnpm/action-setup#57
  

Hello,

Here is a follow-up issue to our slack conversation so we can keep a track of things.

• Jest 28 integration has proven to be a bit painful as shown here: #198 and #199
• However, the serverless-cdk-plugin package aims to use the exports feature of package.json to have multiple entry poins (necessary for serverless plugin particularities) -> #207 for more details
• Support for exports in jest has only been added in the v28
  • jestjs/jest#9771
  • https://jestjs.io/docs/upgrading-to-jest28#packagejson-exports

Just like lodash, exports from aws-sdk must not be imported from its root, but from a specific path, such asaws-sdk/clients/.....

This is because aws-sdk is included as cjs in the lambda runtime but cannot be tree-shaked. This is a big performance issue.

Steps to reproduce:

• Start from a fresh install of Swarmion
• Run yarn generate-service my-service
• Then yarn deploy

The deployment of the generated package fails with error:

       Error:
       Serverless plugin "serverless-iam-roles-per-function" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file. Run "serverless plugin install -n serverless-iam-roles-per-function" to install it.

We need to add it in the service package.json during the generation

yarn watch on a specific package fails in both swarmion/swarmion and the published example

It seems the issue comes from the packaging of types

I think node-fetch is more standard as the fetch already available in browsers.

import fetch from "node-fetch" // <= Or nothing in browser

const response = await fetch(
  baseUrl,
  myApiGatewayContract.getFetchArgs(event)
)

// alternatively
const response = await myApiGatewayContract.fetch(
  baseUrl,
  event
)

Hello swarmion team,

When undefined queryStringParameters are given to getFetchRequest, they are not ignored.

For example:

getFetchRequest(getScenariosForListContract, authFetch, {
          queryStringParameters:  { pageToken: undefined },
        })

results in the following call:
/scenarios?pageToken=undefined

Wouldn't it be better to ignore undefined values and call the following route:
/scenarios

Thanks for the great work!

Hi,

I tried to generate a new library with the nx-plugin but got the following error.

I get the same error when generating a new service.
Any ideas on how to solve that?

Thanks!

For the moment I have to do:

export const ApiGatewayDomainNameIdExport: CloudFormationOutput = {
  Condition: logicalId({ NotInDev }),
  ...ApiGatewayDomainNameIdContract.exportValue({
    description: "Common Api Gateway Domain Name Id",
    value: ref({ ApiGatewayDomainName }),
  }),
};

I recently migrated from @swarmion/serverless-contract v0.5.6 to v0.7.1 to use getLambdaHandler function to instantiate lambda handlers.

However, handler() now expects an argument {requestContext}

In a unit test context, I becomes painful to call handler()
As a temporary fix, I added a ts-ignore but it would be great if swarmion could provide a documentation on the best practices to:

• mock the {requestContext} argument
• Unit test a handler

Currently, create-swarmion-app fetches the examples folder on its latest version (main). However, this is risky since changes can be made on main that have not yet been validated.

In order to fix this issue, we should get the current version in create-swarmion-app and use it to build the remote url in create-packages/swarmion-app/src/create-app.ts

It is not possible to directly setup cognito as an authorizer when using HTTP APIs as stated in the docs.

This issue is to restrict authorizerType to 'jwt' | 'lambda' when using integrationType: 'httpApi' in an API Gateway Contract.

Hey again,

Context

I have a lambda function directly integrated to an ApiGateway, meaning I can curl this gateway and it should execute the lambda and return its return value to me.

Issue

According to the Lambda Proxy Integration doc, it is possible to simply return an object like

{
  statusCode: 401,
  headers: {
    "x-custom-header" : "my custom header value"
  },
  body: JSON.stringify(responseBody)
};

in the lambda function and have a curl call to the ApiGateway return that same error code (401 in this example).

However, my lambda also uses swarmion's ApiGatewayContract with getHttpLambdaHandler. As can be seen in this function's code, if the handler does not crash, the getHttpLambdaHandler calls handlerResponseToLambdaResult which automatically returns a 200.

I did not know that at first and it can lead to some funny curl output like these

(notice the HTTP/2 200 at the start and the response body at the end)

Fixing

Don't know exactly how this could be handled, as it could have been a conscious choice to make it like that. I think depending on the decision I'd be able to fix this issue, but I'll first let some cofounders discuss the problem here :)

In order to make our tests faster, we should migrate from jest to vitest

As stated in https://www.serverless.com/framework/docs/providers/aws/guide/deploying#deployment-method:

Since Serverless Framework v3, deployments are done using CloudFormation change sets. It is possible to use CloudFormation direct deployments instead.
Direct deployments are faster and have no downsides (unless you specifically use the generated change sets). They will become the default in Serverless Framework 4.
You are encouraged to enable direct deployments via the deploymentMethod option:

provider:
 name: aws
 deploymentMethod: direct

We should add it to the default provider options

Similar to the awesome job by @StanHannebelle and @charlesgery in swarmion/template#120, we could implement the same check in the internal packages of Swarmion.

Use https://github.com/theodo/clinter to update the eslint config both in Swarmion and the starter

Rationale for migration from yarn to pnpm

• This is the package manager that is used by all the major frameworks: next, vue, vite, etc...
• It uses hardlinks for node_modules by default, so you won't end up with 100GB of node_modules across the projects on your machine.
• The way they handle workspaces (monorepo) and node_modules in each package is much better than yarn: you have a node_modules folder in all your packages, rather than everything in the root. The impact is that if you have a dependency that does something crazy (like serverless that dynamically requires modules), it will work.
• The performance is much better for installation, update, etc...

It would be good to implement some of the logic for this authorizer type, especially the whole signin process.

After generating an app with yarn create swarmion-app, it seems that commands are hitting the nx cache even if there is a change in the code.

A service using an axios version different of the one used by swarmion/serverless-contracts, it can result in a ts error.

Exemple:
swarmion/serverless-contracts: "axios": "0.27.2"
service using swarmion/serverless-contracts : "axios": "0.26.1"

Possible solutions:

• add axios to peer dependencies of swarmion/serverless-contracts
• use fetch instead of axios
• remove getAxiosRequest

Hi, at Kooalys, we encountered an issue that might also bother you.

We created a new lib in our Library folder.
In dev, no problem all tests are green, but as soon as we pushed it and ran tests in CI, we've got some :
Error: xxx/Xxxx/xxx.ts (1,30): error TS2307: Cannot find module '@kooalys/xxxx-xxx' or its corresponding type declarations.
on all of our tests.

It took a bit of digging, but it seems that the issue originated from the yarn modules cache that was still hit in CI, even after the new library addition, because the cache key is a yarn.lock hash and this file has not been modified.

We currently have to manally upgrade our packages. This is inefficient and time-consuming.

In order to do this, I have considered two solutions:

• dependabot: however, it currently does not support yarn3. See dependabot/dependabot-core#1297
• renovate which seems to support yarn3

For the moment I would rather select Renovate for this reason, do you have other solutions?

Hello @fargito @Sc0ra,

Great work with this PR, it was really missing !

However, it seems that you only handle a minority of the possible cases.
Indeed I use an HTTP API (api gateway v2) and the type you used for the event (APIGatewayProxyWithCognitoAuthorizerEvent) is not correct for JWT Authorizer in API Gateway V2 (and certainly other cases of API Gateway V2).

@fargito do you have an idea of what API we could use to handle all the cases ?

In the mean time, it would be great to add in the doc that the only supported case is API Gateway V1 with CognitoAuthorizer.

Since the awesome #112, we have an examples folder, but it is not checked by the current CI.

It is paramount that we implement a CI for this example, otherwise we would risk to ship a broken app for the create-swarmion-app

getTrigger doesn't get authorizer typing from ApiGatewayContract :

• When providing an authorizerType in ApiGatewayContract, no authorizer parameter is required in getTrigger additionalConfig parameter in lambda configuration.
• When setting authorizerType to undefined in ApiGatewayContract, no type error is raised if the getTrigger function is provided with an authorizer parameter in lambda configuration.

When outputSchema is set to undefined in a contract, lambda handler's output typing inferred by getLambdaHandler is Promise<undefined>.

export type OutputType<Contract extends ApiGatewayContract> =
  Contract['outputSchema'] extends JSONSchema
    ? FromSchema<Contract['outputSchema']>
    : undefined;

This causes a TypeScript error when creating a lambda that does not return anything because TypeScript expects a Promise<undefined> instead of a Promise<void>.

Setting outputSchema in the contract to undefined seems to be the proper configuration when a lambda does not return anything. Typing lambda handler's output as Promise<void> would solve this.

export type OutputType<Contract extends ApiGatewayContract> =
  Contract['outputSchema'] extends JSONSchema
    ? FromSchema<Contract['outputSchema']>
    : void;

Current deployment flow

In order to release a new version of Swarmion, we currently need to:

• ✋🏻 checkout locally on main
• ✋🏻 install node dependencies and run tests
• ✋🏻 run yarn release which will:
  • ✋🏻 prompt for the new version number
  • 🔨 bump versions in all our package.json and yarn.lock
  • 🔨 commit these changes
  • 🔨 add a git tag and push it
  • 🔨 publish packages
• ✋🏻 go to examples/starter
• ✋🏻 install node dependencies
• ✋🏻 run yarn upgrade for Swarmion dependencies
• ✋🏻 potentially take breaking changes into account
• ✋🏻 run the tests
• ✋🏻 commit and push these changes
• ✋🏻 create the Github release from the git tag created by Lerna

These are may steps 😱. We need to optimize this process in order to scale.

Constraints

• we don't want to release at every commit on main
• we need to be able to choose the version number
• with #175 , we want to use Swarmion starter with a version number. This means the tagging step must occur after upgrading Swarmion dependencies in the starter
• we might also want a simpler way to publish alpha releases (but it is not necessarily the same issue)

At the moment, although it is not typed as required, getFetchRequest requires a baseUrl in its options argument : providing an empty object for options results in the following error : Failed to construct 'URL': Invalid URL.

This is because getFetchRequest creates a new URL and provide it to the fetchFunction. In the absence of the baseUrl option, it should only call the fetchFunction with the contract path as first parameter (the fetchFunction will then be responsible of adding the baseUrl).

Also, the options argument is always required in getFetchRequest. It could be optional and set to an empty object by default if no option is required.

Since the awesome #112, we use serverless-helpers in the examples folder, but there is little interest in having it in the generated apps.

Instead, we should publish it on npm to bring new helpers to repos that are already using it.

We may have cache mismatches due to syncpack

Let's imagine that someone forgets to run syncpack format before pushing to PR. When running the build step, we run yarn install, which may edit the package.json. Then the cache key will be computed with the modified value.

When arriving in the test steps, the package.json will not have been changed and therefore the cache key will not match the one from the build step.

This may cause precious time to be lost.

In order to mitigate, we could add a syncpack (not syncpack format) command to our build step, this will validate that no package.json has been modified