Git Product home page Git Product logo

aws / aws-pdk Goto Github PK

View Code? Open in Web Editor NEW
309.0 10.0 63.0 382.26 MB

The AWS PDK provides building blocks for common patterns together with development tools to manage and build your projects.

Home Page: https://aws.github.io/aws-pdk/

License: Apache License 2.0

Shell 0.82% TypeScript 75.65% JavaScript 0.35% HTML 0.01% Mustache 19.17% Handlebars 3.56% Python 0.14% Dockerfile 0.06% Java 0.24%
aws bootstrap cdk iac monorepo nx projen smithy

aws-pdk's Introduction

Getting started

What is the AWS PDK?

The AWS Project Development Kit (AWS PDK) provides building blocks for common patterns together with development tools to manage and build your projects.

The AWS PDK lets you define your projects programatically via the expressive power of type safe constructs available in one of 3 languages (typescript, python or java). This approach yields many benefits, including:

  • Ability to set up new projects within seconds, with all boilerplate already pre-configured.
  • Receive updates to previously bootstrapped projects when new versions become available i.e: updated dependenies or lint configurations.
  • Build polyglot monorepos, with build caching, cross-language build dependencies, dependency visualization and much more.
  • Leverage codified patterns which vend project and infrastructure (CDK) code.

The AWS PDK is built on top of Projen and as such all constructs that you compose together need to be defined via a projenrc file.

Why use the AWS PDK?

It's much easier to show than explain! Here is some PDK code (within projenrc file) that creates a Polyglot monorepo, with a React Website pre-configured with Cognito Auth and pre-integrated with a Smithy Type Safe Api.

import { CloudscapeReactTsWebsiteProject } from "@aws/pdk/cloudscape-react-ts-website";
import { InfrastructureTsProject } from "@aws/pdk/infrastructure";
import { MonorepoTsProject } from "@aws/pdk/monorepo";
import {
    DocumentationFormat,
    Language,
    Library,
    ModelLanguage,
    TypeSafeApiProject,
} from "@aws/pdk/type-safe-api";
import { javascript } from "projen";

const monorepo = new MonorepoTsProject({
    name: "my-project",
    packageManager: javascript.NodePackageManager.PNPM,
    projenrcTs: true,
});

const api = new TypeSafeApiProject({
    parent: monorepo,
    outdir: "packages/api",
    name: "myapi",
    infrastructure: {
        language: Language.TYPESCRIPT,
    },
    model: {
        language: ModelLanguage.SMITHY,
        options: {
        smithy: {
            serviceName: {
            namespace: "com.aws",
            serviceName: "MyApi",
            },
        },
        },
    },
    runtime: {
        languages: [Language.TYPESCRIPT],
    },
    documentation: {
        formats: [DocumentationFormat.HTML_REDOC],
    },
    library: {
        libraries: [Library.TYPESCRIPT_REACT_QUERY_HOOKS],
    },
    handlers: {
        languages: [Language.TYPESCRIPT],
    },
});

const website = new CloudscapeReactTsWebsiteProject({
    parent: monorepo,
    outdir: "packages/website",
    name: "website",
    typeSafeApi: api,
});

new InfrastructureTsProject({
    parent: monorepo,
    outdir: "packages/infra",
    name: "infra",
    cloudscapeReactTsWebsite: website,
    typeSafeApi: api,
});

monorepo.synth();

This code (also available in Python and Java), produces all the source code, packages and infrastructure needed to deploy a fully-operable application in the AWS cloud. All that's left to do is build and deploy it!

From this ~70 lines of code above, the AWS PDK produces the following packages on your behalf:

  • monorepo: Root level project that manages interdependencies between projects within the Monorepo, provides build caching and dependency visualziation.
  • api/model: A project that allows you to define your API using Smithy (or OpenAPI) IDL.
  • api/generated/documentation: A project that automatically creates API documentation in a variety of formats.
  • api/generated/infrastructure: A project that automatically creates API infrastructure constructs in a type-safe manner.
  • api/generated/libraries: A project that automatically generates a react hooks library that can be used to call your API from a React based website.
  • api/generated/runtime: A project that contains server bindings for handlers to ensure type safety.
  • api/handlers: A project that automatically creates handler stubs, preconfigured with type-safety and a variety of value added features based on your defined API's.
  • website: A project which creates a React based website built using Cloudscape that comes pre-integrated with Cognito Auth and your created API. This provides you with the ability to call your API securely.
  • infra: A project which sets up all CDK related infrastructure needed to deploy your application. It also comes pre-configured to generate a diagram based on your CDK code everytime you build.

Bootstrapped Source

Generated Website

Generated Diagram

As you can see, the AWS PDK provides you with valuable time savings so you can focus on working on what matters most to your project.

Developing with the AWS PDK

Please refer to the full documentation website.

https://aws.github.io/aws-pdk

Contributing to the AWS PDK

https://aws.github.io/aws-pdk/contributing/index.html

License

This project is licensed under the Apache-2.0 License.

aws-pdk's People

Contributors

agdimech avatar bhrutledge avatar caopengau avatar cogwirrel avatar dboyd13 avatar dependabot[bot] avatar donatoaz avatar drskur avatar em-xqm avatar garysassano avatar himan085 avatar janeklb avatar jeremyjonas avatar jessieweiyi avatar jongwooo avatar joshuatoth avatar jstrunk avatar martenpayne avatar mirgj avatar mteichtahl avatar pcozzi avatar sperka avatar swap-aws avatar trivikr avatar valebedu avatar valter-silva-au avatar walmsles avatar wawajo avatar zetashift avatar zsstiers avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-pdk's Issues

[BUG] dependencies are not installed by npm

Describe the bug

Hi, I'm using PDK with npm and observing a weird behavior about npm install or npm update. Could you check this? The reproduction steps are below.

Expected Behavior

npm i aws-prototyping-sdk should install all the dependencies, and npm update won't remove any packages.

Current Behavior

Not all the required packages are installed on npm i aws-prototyping-sdk.

Also npm update removes several dependent packages of PDK, causing inconsistency between package.json and package-lock.json.

Reproduction Steps

Run the following commands in an empty directory:

npm init -y
npm i aws-prototyping-sdk
npm ci
# -> npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.

npm i
npm ci
# -> success

npm update
npm ci
# -> npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.

I confirmed the issue on both npm v8 and v9.

Possible Solution

No response

Additional Information/Context

npm ci is the command we recommend for customers to use to install dependencies, but currently customers sometimes see the above error, getting confused. (e.g. after running npm update, they get the error on npm ci.)

PDK version used

0.13.5

What languages are you seeing this issue on?

TypeScript

Environment details (OS name and version, etc.)

macOS 12.6.3

[BUG] Only one PDKPipeline can be created per account/region

Describe the bug

See: #235

The exportName for the CodeRepositoryGRCUrl is hardcoded and so conflicts with a second deployment of PDKPipeline.

Expected Behavior

Able to deploy multiple PDKPipeline constructs in the same account/region

Current Behavior

Export with name CodeRepositoryGRCUrl is already exported by stack PipelineStack. Rollback requested by user.

Reproduction Steps

Deploy more than one PDKPipeline, either in the same app or as separate apps.

Possible Solution

Include props.repositoryName in the export name since this is already a unique name.

Additional Information/Context

No response

PDK version used

0.12.4

What languages are you seeing this issue on?

Typescript, Java, Python

Environment details (OS name and version, etc.)

OSX

[BUG] CloudscapeReactTsWebsiteProject build breaks with ""

Describe the bug

Building a CloudscapeReactTsWebsiteProject created with the following code should succeed:

const website = new CloudscapeReactTsWebsiteProject({
  name: "website",
  defaultReleaseBranch: "mainline",
  parent: monorepo,
  outdir: "packages/website",
  deps: ["@aws-amplify/auth", "@aws-amplify/core"],
});

Expected Behavior

Build is successful

Current Behavior

> $ npx nx run-many --target=build --all                                                                                                                   [±mainline ●●]


    ✖  nx run website:build
yarn run v1.22.19
$ npx projen build
       👾 build » compile | react-scripts build
       Creating an optimized production build...
       Failed to compile.
       
       [eslint] 
       src/NavHeader.tsx
         Line 4:1:  '@aws-amplify/auth' should be listed in the project's dependencies. Run 'npm i -S @aws-amplify/auth' to add it  import/no-extraneous-dependencies
       
       Search for the keywords to learn more about each error.
       
       
       👾 Task "build » compile" failed when executing "react-scripts build" (cwd: /Users/donatoaz/src/aws/protos/rotoplas/src/frontend/packages/website)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Reproduction Steps

Create nx-monorepo project. Add CloudscapeReactTsWebsiteProject as described above. Run npx projen && npx nx run website:build.

Possible Solution

No response

Additional Information/Context

No response

PDK version used

^0.65.48

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

mac os, node js v14.19.0

[BUG] `NxMonorepoProject` disables github

Describe the bug

NxMonorepoProject disables GitHub integration

Expected Behavior

NxMonorepoProject shall not have an opinion on this, and should allow user to set this.

Current Behavior

Hard coded github: false.

Reproduction Steps

N/A

Possible Solution

  1. Allow passing props.
  2. Setup NxMonorepoProject to work with GitHub properly

Additional Information/Context

N/A

PDK version used

latest

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

macOS

[BUG] Build fail with new SmithyAPIGatewayTsProject

Describe the bug

Steps to reproduce:

  1. Gen project with: npx projen new --from aws-prototyping-sdk nx-monorepo
  2. Add a SmithyAPIGatewayTsProject to .projenrc.ts:
new SmithyApiGatewayTsProject({
  clientLanguages: [ClientLanguage.TYPESCRIPT],
  defaultReleaseBranch: "main",
  name: "api",
  parent: monorepo,
  outdir: "packages/api",
});
  1. Run npx projen && npx nx run-many --target=build --all

Outcome:

projen default command fails (ts-node --project tsconfig.dev.json .projenrc.ts) however, it looks like a subcommand failed instead:

# Command formated
Error: Command failed: ./gradlew \ 
  -p /prototype/packages/api/smithy-build generate \
  -Pconfig=/prototype/packages/api/smithy-build/smithy-build.json \
  -Pdiscover=/prototype/packages/api/model \
  -Poutput=/prototype/packages/api/smithy-build/output
FAILURE: Build failed with an exception.

* What went wrong:
Could not open cp_settings generic class cache for settings file 'prototype/packages/api/smithy-build/settings.gradle' (/Users/cpenta/.gradle/caches/7.5.1/scripts/1b8lfqt0kt3e5yt7t7grdr4rh).
> BUG! exception in phase 'semantic analysis' in source unit '_BuildScript_' Unsupported class file major version 63

Inside of the settings.gradle file is only the autogenerated content:

# settings.gradle
pluginManagement {
    plugins {
        id 'software.amazon.smithy' version '0.6.0'
    }
}

Welp...figured out the issue while describing it. Good rubber duck...Will comment with resolution and close

Expected Behavior

See resolution

Current Behavior

See resolution

Reproduction Steps

See resolution

Possible Solution

No response

Additional Information/Context

No response

PDK version used

0.11.3

What languages are you seeing this issue on?

No response

Environment details (OS name and version, etc.)

MacOS

[FEATURE] Improve local development and package validation by using local verdaccio

Describe the feature

We need a way to verify published package content works as expected.

Use Case

Just realized that cdk-graph and cdk-graph-plugin-diagram packages were completely broken due to "bundledDependencies" not getting bundled, which was caused by yarn workspace hoisting. When testing those packages I created create several example workspaces outside of PDK repo and used "yarn link" successfully. The issue was that the "bundledDependencies" where available since yarn link is on the source which include them.

If we had an easy way to publish the PDK monorepo locally (verdaccio) during development, then we could more easily test usage outside the repo against actual published artifacts.

Proposed Solution

Implement tasks / helpers in the root to spawn verdaccio server and publish package to.

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

PDK version used

0

What languages will this feature affect?

No response

Environment details (OS name and version, etc.)

n/a

[Documentation] Update jsdocs

Quite a few classes are lacking proper jsdocs. We should describe (in detail) how every public class, function and variable is to be used.

[FEAT] Add support for packaging successfully scanned code to S3

We should add a feature to the SonarScanner to support published successfully scanned code to S3.

  1. At the completion of the SonarScanner build job we should retrieve the resolvedSourceVersion via aws codebuild batch-get-builds --ids $SYNTH_BUILD_ID | jq -r '.builds[0].resolvedSourceVersion'
  2. zip up the src directory
  3. aws s3 cp src.zip s3://<some-publish-bucket>/<resolved-src-version>.zip

To support this, we will need to create an S3 bucket with encryption and grant write access (including kms encrypt) from the SonarScanner CodeBuild role.

[BUG] Missing 403 error response for Static Website

Describe the bug

The Python static_website construct doesn't route 403 AccessDenied errors to the root object. This is with a Nuxt SPA built via npm run generate prior to running cdk deploy.

Expected Behavior

Getting the Nuxt 404 page when accessing a route that doesn't exist.

Screen Shot 2022-06-28 at 11 46 23 AM

Current Behavior

Getting an AWS error:

<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>SD5ACEACKRM2GR8Q</RequestId>
<HostId>X0fWDXEGxlX/fS+eFoBiM/8yI4ZpZi7g5HWrwluVsb8h2Z2NtBbMLqHVdiGiPnsIUkajgARoS9Y=</HostId>
</Error>

Reproduction Steps

Here's the CDK stack that I created:

from pathlib import Path
from typing import Any

from aws_cdk import CfnOutput, Stack
from aws_prototyping_sdk import static_website
from constructs import Construct


class FrontendStack(Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs: Any) -> None:
        super().__init__(scope, construct_id, **kwargs)

        dist_path = Path(__file__).parent / ".." / ".." / "frontend" / "dist"

        website = static_website.StaticWebsite(
            self,
            "Frontend",
            website_content_path=str(dist_path.resolve()),
        )

        CfnOutput(
            self,
            "URL",
            value=f"https://{website.cloud_front_distribution.domain_name}",
        )

Possible Solution

Additional Information/Context

I added a 403 response manually via the CloudFront console, and got the expected behavior (i.e. the Nuxt 404 page).

Screen Shot 2022-06-28 at 11 41 42 AM

I was able to accomplish the same thing by overriding error_responses:

        website = static_website.StaticWebsite(
            self,
            "Frontend",
            website_content_path=str(dist_path.resolve()),
            distribution_props=cloudfront.DistributionProps(
                # HACK: Per the docs, default_behavior.origin is required, but ignored
                default_behavior=cloudfront.BehaviorOptions(
                    origin=static_website.StaticWebsiteOrigin()
                ),
                error_responses=[
                    # HACK: Handle AWS `AccessDenied` errors for missing Nuxt routes
                    cloudfront.ErrorResponse(
                        http_status=403,
                        response_http_status=200,
                        response_page_path="/index.html",
                    ),
                    # Copied from StaticWebsite to add an additional respose
                    cloudfront.ErrorResponse(
                        http_status=404,
                        response_http_status=200,
                        response_page_path="/index.html",
                    ),
                ],
            ),
        )

PDK version used

aws-prototyping-sdk.static-website==0.2.10

What languages are you seeing this issue on?

Python

Environment details (OS name and version, etc.)

macOS 12.4

[BUG] npx projen results in error related aws-cdk-lib

Describe the bug

An error related to aws-cdk-lib library dependency occurs when the following steps are performed.
Note that I haven't made any changes to .projenrc.ts file yet.

mkdir my-project && cd my-project
npx projen new --from aws-prototyping-sdk nx-monorepo

npx projen
👾 default | ts-node --project tsconfig.dev.json .projenrc.ts
Error: Cannot find module 'aws-cdk-lib'
Require stack:
- /Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/aws-prototyping-sdk/pipeline/lib/pdk-pipeline.js
- /Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/aws-prototyping-sdk/pipeline/lib/index.js
- /Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/aws-prototyping-sdk/pipeline/index.js
- /Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/aws-prototyping-sdk/index.js
- /Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/.projenrc.ts
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/@cspotcode/source-map-support/source-map-support.js:811:30)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/aws-prototyping-sdk/pipeline/src/pdk-pipeline.ts:17:1)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Object.require.extensions.<computed> [as .js] (/Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/ts-node/src/index.ts:1587:43)
    at Module.load (node:internal/modules/cjs/loader:981:32) {

It is questionable whether a dependency error occurs even though pure nx-monorepo project has been configured, and if necessary, it seems that the dependency must be declared in advance.

For temporal purpose, I solved the problem now by running npm i command as a workaround.

Expected Behavior

No error like this:

npx projen
👾 default | ts-node --project tsconfig.dev.json .projenrc.ts

Current Behavior

npx projen
👾 default | ts-node --project tsconfig.dev.json .projenrc.ts
Error: Cannot find module 'aws-cdk-lib'
Require stack:
- /Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/aws-prototyping-sdk/pipeline/lib/pdk-pipeline.js
- /Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/aws-prototyping-sdk/pipeline/lib/index.js
- /Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/aws-prototyping-sdk/pipeline/index.js
- /Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/aws-prototyping-sdk/index.js
- /Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/.projenrc.ts
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/@cspotcode/source-map-support/source-map-support.js:811:30)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/aws-prototyping-sdk/pipeline/src/pdk-pipeline.ts:17:1)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Object.require.extensions.<computed> [as .js] (/Users/kwonyul/MyWork/source/test-projen/pdk-test-issue/node_modules/ts-node/src/index.ts:1587:43)
    at Module.load (node:internal/modules/cjs/loader:981:32) {

Reproduction Steps

Every time in the following condition:

mkdir my-project && cd my-project
npx projen new --from aws-prototyping-sdk nx-monorepo

npx projen

Possible Solution

solve dependency

Additional Information/Context

no

PDK version used

latest 0.2.17

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

mac, node = v16.15.1

[BUG] OpenApiGatewayPythonProject Intermittent CodeBuild failure

Describe the bug

We are using OpenApiGatewayPythonProject and PDKPipeline project. There are some intermittent CodeBuild failures in the CodePipeline synth stage (around 1/3 builds failed). Although click the release change to rerun the pipeline may make the build pass, it is annoying that it may be failed again after rerun.

There are 2 type of intermittent errors. I think they may be related. Seems like all related to the installation of *api-python module.

  • Intermittent Error 1:
INSTALL Failed COMMAND_EXECUTION_ERROR: Error while executing command: yarn install --frozen-lockfile || npx projen && yarn install --frozen-lockfile. Reason: exit status 1
👾 install | pip install --editable .
Obtaining file:///codebuild/output/src937497960/src/packages/project_name_api/generated/python
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: urllib3>=1.15 in /codebuild/output/src937497960/src/.env/lib/python3.9/site-packages (from project-name-api-python==1.0.0) (1.26.12)
Requirement already satisfied: certifi in /codebuild/output/src937497960/src/.env/lib/python3.9/site-packages (from project-name-api-python==1.0.0) (2022.9.24)
Requirement already satisfied: python-dateutil in /codebuild/output/src937497960/src/.env/lib/python3.9/site-packages (from project-name-api-python==1.0.0) (2.8.2)
Requirement already satisfied: frozendict>=2.0.3 in /codebuild/output/src937497960/src/.env/lib/python3.9/site-packages (from project-name-api-python==1.0.0) (2.3.4)
Requirement already satisfied: six>=1.5 in /codebuild/output/src937497960/src/.env/lib/python3.9/site-packages (from python-dateutil->project-name-api-python==1.0.0) (1.16.0)
Installing collected packages: project-name-api-python
  Attempting uninstall: project-name-api-python
    Found existing installation: project-name-api-python 1.0.0
    Uninstalling project-name-api-python-1.0.0:
      Successfully uninstalled project-name-api-python-1.0.0
  Running setup.py develop for project-name-api-python
    error: subprocess-exited-with-error
    
    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [2 lines of output]
        running develop
        error: [Errno 2] No such file or directory: '/codebuild/output/src937497960/src/.env/lib/python3.9/site-packages/project-name-api.egg-link'
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
  Rolling back uninstall of project-name-api-python
  Moving to /codebuild/output/src937497960/src/.env/lib/python3.9/site-packages/project-name-api-python.egg-link
   from /tmp/pip-uninstall-nd55vfjv/project-name-api-python.egg-link
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [2 lines of output]
    running develop
    error: [Errno 2] No such file or directory: '/codebuild/output/src937497960/src/.env/lib/python3.9/site-packages/project-name-api.egg-link'
    [end of output]
  • Intermittent Error 2:
BUILD Failed COMMAND_EXECUTION_ERROR: Error while executing command: npx nx run-many --target=build --all. Reason: exit status 1
> nx run infra:build

·[1Gyarn run v1.22.19
·[1G$ npx projen build
👾 build » post-compile » synth:silent | cdk synth -q
Traceback (most recent call last):
  File "/codebuild/output/src425806827/src/packages/infra/infra/pipeline.py", line 4, in <module>
    from application_stage import ApplicationStage
  File "/codebuild/output/src425806827/src/packages/infra/infra/application_stage.py", line 3, in <module>
    from application_stack import ApplicationStack
  File "/codebuild/output/src425806827/src/packages/infra/infra/application_stack.py", line 10, in <module>
    from project_name_api import ProjectNameApi
  File "/codebuild/output/src425806827/src/packages/project_name_api/project_name_api/__init__.py", line 1, in <module>
    from .api.project_name_api import ProjectNameApi
  File "/codebuild/output/src425806827/src/packages/project_name_api/project_name_api/api/project_name_api.py", line 2, in <module>
    from project_name_api_python.apis.tags.default_api_operation_config import OperationConfig
ModuleNotFoundError: No module named 'project_name_api_python'

Subprocess exited with error 1
👾 Task "build » post-compile » synth:silent" failed when executing "cdk synth -q" (cwd: /codebuild/output/src425806827/src/packages/infra)
·[1Gerror Command failed with exit code 1.
·[1Ginfo Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

 
 >  NX   Running target "build" failed

   Failed tasks:
   
   - infra:build

Expected Behavior

It should have consistent build results.

Current Behavior

1/3 builds failed due to the 2 reasons above

Reproduction Steps

They are intermittent errors. Just try a few times in your CodePipeline. you may reproduce it.

Possible Solution

No response

Additional Information/Context

projenrc file

import { pipeline, nx_monorepo } from 'aws-prototyping-sdk';
import { ApprovalLevel } from 'projen/lib/awscdk';
import { VenvOptions } from 'projen/lib/python';
import { OpenApiGatewayPythonProject, ClientLanguage } from '@aws-prototyping-sdk/open-api-gateway';

const sharedEnv: VenvOptions = {
  envdir: '../../.env',
};

const monorepo = new nx_monorepo.NxMonorepoProject({
  defaultReleaseBranch: 'mainline',
  devDeps: ['[email protected]'],
  deps: [
    '@aws-prototyping-sdk/[email protected]', 
    '@aws-prototyping-sdk/[email protected]',
  ],
  gitignore: [".env", ".ipynb_checkpoints", "xgboost-model"],
  name: 'project_name',
  tsconfig: {
    compilerOptions: {},
    include: [
      "packages/**/*.ts",
      "packages/**/*.tsx",
    ]
  }
});

const apiProject = new OpenApiGatewayPythonProject({
  parent: monorepo,
  outdir: 'packages/project_name_api',
  name: 'project-name-api',
  moduleName: 'project_name_api',
  version: '1.0.0',
  authorName: '',
  authorEmail: '',
  clientLanguages: [ClientLanguage.TYPESCRIPT],
  venvOptions: sharedEnv,
  deps: [
    '[email protected]',
  ]
});

apiProject.tasks.tryFind('install')!.exec('pip install --editable .');

const pipelineProject = new pipeline.PDKPipelinePyProject({
  authorEmail: '',
  authorName: '',
  cdkVersion: '2.0.0',
  moduleName: 'infra',
  name: 'infra',
  version: '0.0.0',
  requireApproval: ApprovalLevel.NEVER,
  parent: monorepo,
  outdir: 'packages/infra',
  appEntrypoint: 'infra/pipeline.py',
  deps: [
    "[email protected]",
    "[email protected]", 
    "aws-cdk.aws-lambda-python-alpha",
    apiProject.name
  ],
  venvOptions: sharedEnv,
});

const pipelineProjectTestTask = pipelineProject.tasks.tryFind("test");
pipelineProjectTestTask?.reset();
pipelineProjectTestTask?.exec("pytest tests/");

monorepo.addImplicitDependency(pipelineProject, apiProject);

monorepo.synth();

PDK version used

0.11.1

What languages are you seeing this issue on?

Python

Environment details (OS name and version, etc.)

CodeBuild provisioned by PDKPipeline

fix: replace the verbose header text with shorthand SPDX-License-Identifier

Legacy verbose header

https://github.com/aws/aws-prototyping-sdk/blob/1082c41443976f55a5c7fe8dff6c4e5e25a180a4/header.js#L1-L15

Preferred SPDX header

/*! Copyright [Amazon.com](http://amazon.com/), Inc. or its affiliates. All Rights Reserved.
SPDX-License-Identifier: Apache-2.0 */

Additionally, use "block" rather than file for eslint header plugin options

https://github.com/aws/aws-prototyping-sdk/blob/1082c41443976f55a5c7fe8dff6c4e5e25a180a4/packages/identity/.eslintrc.json#L130-L132

"block" feature: https://github.com/Stuk/eslint-plugin-header#2-arguments

By using "block" we can prevent relative file issues with eslint, and since SPDX header is smaller it will not pollute the eslint config.

[FEATURE] Support smithy-vscode extension

Describe the feature

To create a projen-managed artifact that is generated when one uses a smithy-api-gateway-XXX-project in order to support smithy vscode extension and not to fail with validations that could be easily avoided.

Use Case

In the current form, smithy-vscode extension is constantly failing with the error of not recognizing aws.protocols#restJson1 trait:

Caused by: software.amazon.smithy.model.validation.ValidatedResultException: Result contained ERROR severity validation events: 
[ERROR] my.namespace#MyService: Unable to resolve trait `aws.protocols#restJson1`. If this is a custom trait, then it must be defined before it can be used in a model. | Model ....

This becomes annoying after some time.

Proposed Solution

As the README says, a smithy-build.json should be present in the root of the project.

Generating a smithy-build.json with the following content in the root of the project solves the issue mentioned above:

{
  "version": "2.0",
  "maven": {
    "dependencies": [
      "software.amazon.smithy:smithy-cli:1.24.0",
      "software.amazon.smithy:smithy-model:1.24.0",
      "software.amazon.smithy:smithy-openapi:1.24.0",
      "software.amazon.smithy:smithy-aws-traits:1.24.0"
    ],
    "repositories": [{ "url": "https://repo1.maven.org/maven2/" }]
  },
  "//": "~~ Generated by projen. To modify, edit .projenrc.js and run \"npx projen\"."
}

NOTE: the dependencies can be generated based on the smithy project's settings in the PDK project. More specifically, all the dependencies that are present in packages/project-name/smithy-build/build.gradle's dependencies section, can be added to the "root" smithy-build.json. This won't have any effect generating the API project.

Additionally, based on the smithy-vscode codebase, there is a way of defining a rootPath for the extension: source. It would be ideal to generate a .smithy/smithy-build.json file, but unfortunately at this point smithyLsp.rootPath in .vscode/settings.json results in Unknown Configuration Setting and the value is not picked up by the Smithy language server.

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

PDK version used

0.11.2

What languages will this feature affect?

Typescript

Environment details (OS name and version, etc.)

MacOS 12.5.1

[FEATURE] Smithy should provide a way to group API operators in generated documentation

Describe the feature

Today, when documentation is generated from smithy it doesn't respect/understand the needs of documentation generation tools. For example, redoc, requires an x-tag for operator grouping - see https://redocly.com/docs/api-reference-docs/specification-extensions/x-tags/

Use Case

support a mechanism via custom tags or other implementation so that documentation generated is presentable and usable to customers.

Proposed Solution

No response

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

PDK version used

0.12.24

What languages will this feature affect?

Typescript, Java, Python

Environment details (OS name and version, etc.)

all

[BUG] npm ERR! This command does not support workspaces

Describe the bug

Setup a nx-monorepo that uses a subproject of type AwsCdkConstructLibrary.

Running npm set //npm.pkg.github.com/:_authToken $GITHUB_PACKAGER_TOKEN && npm publish in a task results in the error:

� package-all » package:js | jsii-pacmak -v --target js
       [jsii-pacmak] [INFO] Found 1 modules to package
       [jsii-pacmak] [INFO] Packaging NPM bundles
       [jsii-pacmak] [INFO] Loading jsii assemblies and translations
       [jsii-pacmak] [INFO] Packaging 'js' for test_project
       [jsii-pacmak] [INFO] js finished
       [jsii-pacmak] [INFO] Packaged. load jsii (0.7s) | npm pack (0.4s) | js (0.0s) | cleanup (???)
      
� releaser » unbump | /home/ec2-user/.nvm/versions/node/v16.14.2/bin/node /home/ec2-user/environment/test_project/node_modules/projen/lib/release/reset-version.task.js

� releaser | npm set //npm.pkg.github.com/:_authToken $PACKAGER_TOKEN && npm publish
       npm ERR! code ENOWORKSPACES
       npm ERR! This command does not support workspaces.

Expected Behavior

npm command should work without error.

Current Behavior

npm ERR! code ENOWORKSPACES
npm ERR! This command does not support workspaces.

Reproduction Steps

Attached a test repo. node modules removed for size reasons.

  • Run yarn install
  • Run npx nx run-many --target=releaser --all

test_projen.zip

Possible Solution

No response

Additional Information/Context

npm -v
8.15.0

PDK version used

0.12.20

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

Linux x86_64

[FEATURE] Make Smithy Build/Model a first class package

Describe the feature

Can we pull Smithy Build and Model out of OpenAPIGateway and make it it's own project?

Use Case

Ive been using SmithyAPIGatewayTSProject to generate typescript API. Instead of generating code from an OpenAPI spec, I wanted to generate code from Smithy directly instead. The current solution has been:

  • Just generate and ignore/delete the generated code for OpenAPI
  • Use the generated SDKs instead

I keep running into a chicken and egg problem:

  • Opinionated CDK Construct uses generated SDKs
  • SDKs are generated within the same project

nx tries to find and build all subdirectories with a package.json. So what ends up happening is that the CDK construct tries to build before the model builds. You can get around this with some projen-fu by generated SDKs, manually running yarn install to install the net new dependencies the SDK needs, commenting out files, adding dependencies, uncommenting files, etc, etc etc.

It would be easier if the flow was:

packages/
|--model        # contains model and generated code
|--construct   # contains CDK constructs that depends on model
|--anything else down the road # depends on model

This way, all packages can depend on model, that way, whenever I run npx nx run-many --target build --all. nx will know to always build model first.

Proposed Solution

No response

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

PDK version used

0.12.4

What languages will this feature affect?

No response

Environment details (OS name and version, etc.)

MacOS

[TESTS] E2E Test Package

A e2e package should be created which extends Typescript project. This package should depend on aws-prototyping-sdk and run a series of e2e tests by running npx projen new --from aws-prototyping-sdk nx-monorepo --sample-language=<language>. It should completely simulate how a user would do this, meaning we should spin up a local NPM registry ala Verdaccio, point our npm registry to the local verdaccio instance and upload the latest dist/js/*.tar.gz from aws-prototyping-sdk. From here it is a matter or validating that the project which gets synthesized contains the correct files, builds, generates the right synthed files (can be done via snapshotting).

In summary:

  1. Create a new e2e-tests package using TypescriptProject.
  2. Add a devDep on [email protected]
  3. Write a script to spin up a local verdaccio instance and update the local registry to point to it (.npmrc or some other mechanism).
  4. prepend the startup of the verdacio server to the build task
  5. Implement E2E tests as Jest tests by running npx projen new --from aws-prototyping-sdk nx-monorepo --sample-language=<language>

Things to test for:

  • All languages should build successfully with npx nx run-many --target=build --all
  • Synthesized outputs are correct (with and without supplied context)

[BUG] API Gateway `ANY` type is not supported

Describe the bug

I am, unfortunately, porting over some code that uses API gateway's ANY type for HTTP methods as the actual Lambda does function level routing based on the HTTP method passed in.

Since the underlying OpenAPI generator does not support it Operations. This causes a silent build fail when specifying it in a smithy model

@http(method: "ANY", uri: "/configure")
operation Configure {
    input: ConfigureInput,
    output: ConfigureOutput,
    errors: [ValidationException]
}

Expected Behavior

Build or throw hard fail when @http(method: "ANY") is encountered.

Current Behavior

Silenty fails until typescript throws an error that it cannot find a configureHandler since one was not built.

Reproduction Steps

Should be any operation with @http(method: "ANY")

Possible Solution

Not sure if Smithy's typescript SSDK would solve this? For me, I'm simply going to port the code to use standard HTTP methods and routing at the API gateway level, but wanted to document this.

Additional Information/Context

No response

PDK version used

0.11.4

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

MacOS

[BUG] PDKPipeline occurs error on latest CDK

Describe the bug

When turn on crossAccountKeys, it occurs following error.

SSE-S3 is the only supported default bucket encryption for Server Access Logging target buckets

Expected Behavior

no error

Current Behavior

occurs error
SSE-S3 is the only supported default bucket encryption for Server Access Logging target buckets

Reproduction Steps

    this.pipeline = new PDKPipeline(this, 'ApplicationPipeline', {
      primarySynthDirectory: 'packages/infra/cdk.out',
      repositoryName: this.node.tryGetContext('repositoryName') || 'monorepo',
      publishAssetsInParallel: false,
      crossAccountKeys: true,
      synth: {},
      sonarCodeScannerConfig: this.node.tryGetContext('sonarqubeScannerConfig'),
    });

Possible Solution

I think it changed the default behavior in the cdk.
So we can create new bucket for server access logging.

Additional Information/Context

"dependencies": {
"aws-cdk-lib": "^2.59.0",
"cdk-nag": "^2.21.51",
"constructs": "^10.1.209"
}

PDK version used

0.12.34

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

Mac OS

[FEATURE] open-api-gateway - allow all operations to use the same lambda integration

Describe the feature

Today an Api requires a lambda function and associated integration for each operation. This proposed feature should allow a developer to send all operations to a single lambda integration.

Use Case

As a developer

  • I want to use a single lambda as a handler for all Api operations.
  • PDK should provision my single lambda
  • the provisioned lambda should provide me a simple way of integrating my modules for each operation
  • I want this functionality to work regardless of my lambda implementation language

Proposed Solution

export class SampleApi extends Api {
  constructor(scope: Construct, id: string) {
    super(scope, id, {
      defaultAuthorizer: Authorizers.iam(),
      integration:  Integrations.lambda(new NodejsFunction(scope, 'apiHandlerFunction')
    });
  }
}```

### Other Information

_No response_

### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### PDK version used

0.12.24

### What languages will this feature affect?

Typescript, Java, Python

### Environment details (OS name and version, etc.)

N/A

[FEATURE] Be able to lock the version of python open-api-gateway api project

Describe the feature

Currently, there is no way to lock the version of python @aws-prototyping-sdk/open-api-gateway deps in the API project itself since the project is automatically generated.

Even when the version of @aws-prototyping-sdk/open-api-gateway is locked in the monorepo definition, the version in the API project is not locked.

When there is any breaking change in the lib, the CI/CD pipeline will break since it is always installing the latest version if we cannot lock the version from requirements.txt or use tooling like pipenv.

Use Case

Now when there is any breaking change in the lib, the CI/CD pipeline will break since it is always installing the latest version if we cannot lock the version from requirements.txt or use tooling like pipenv.

Proposed Solution

The @aws-prototyping-sdk/open-api-gateway should have the same version as the monorepo project if it is locked so that we have consistent build result.

Other Information

This is when I tried to lock the version:

projen.rc file

const monorepo = new nx_monorepo.NxMonorepoProject({
  defaultReleaseBranch: 'mainline',
  devDeps: ['[email protected]'],
  deps: [
    '@aws-prototyping-sdk/[email protected]', 
    '@aws-prototyping-sdk/[email protected]'
  ],
  gitignore: [".env", ".ipynb_checkpoints"],
  name: '******',
  tsconfig: {
    compilerOptions: {},
    include: [
      "packages/**/*.ts",
      "packages/**/*.tsx",
    ]
  }
});

deps.json file of the monorepo

{
  "name": "@aws-prototyping-sdk/open-api-gateway",
  "version": "0.10.2",
  "type": "runtime"
},

deps.json file of the api project
{
  "name": "aws_prototyping_sdk.open_api_gateway",
  "type": "runtime"
},



### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### PDK version used

0.11.0

### What languages will this feature affect?

Python

### Environment details (OS name and version, etc.)

CodePipeline

[BUG] cdk-graph-plugin diagram release build only works on linux-x64

Describe the bug

cdk-graph-plugin-diagram is broken due to environment issues with the sharp dependencies.

The sharp package requires pre-built dependencies based on environment - which for the CI release build is linux-x64, resulting in broken functionality on macOs and other non-linux-x64 environments.

https://sharp.pixelplumbing.com/install#prebuilt-binaries

Expected Behavior

Sharp bundled dependency should work cross platform

Current Behavior

Sharp bundled dependency ONLY works on linux-x64 (which is CI release platform)

Reproduction Steps

Use the cdk-graph-diagram-plugin on macOs

Possible Solution

https://github.com/lovell/sharp-libvips

Additional Information/Context

No response

PDK version used

0.12.27

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

macos

[TRACKING] Nested stack runtime-config.json properties

Describe the bug

For my startup-ish prototype project based on CDK and Projen, I like to use Amplify AppSync. Kenneth Winner did a nice construct around it a while back https://www.npmjs.com/package/cdk-appsync-transformer . Unlucky the AppSync is wrapped in a nested stack. So when I take an output from that nested stack I can't use it for the runtime-config.json.

For completion, I don't use the StaticWebsite directly. I found that runtimeOptions part very inspiring so, I incorporated it into my project. But yeah it is fairly simple so you should have the same problem in your amazing construct.

Expected Behavior

No error is thrown.

Current Behavior

An error is thrown like:

failed: Error [ValidationError]: Template error: instance of Fn::GetAtt references undefined resource apiC8550315

Reproduction Steps

const nestedStack = new core.NestedStack(this, 'appsync-nested-stack');
const app = new appsync.GraphqlApi(nestedStack, 'api', { name: 'blub' });

const dashboard = new StaticWebsite(this, 'dashboard', {
  ...,
  runtimeOptions: {
    jsonPayload: {
      // appSyncGraphqlEndpoint: appSyncTransformer.appsyncAPI.graphqlUrl,
      appSyncGraphqlEndpoint: app.graphqlUrl,
    },
  },
});

Possible Solution

No response

Additional Information/Context

No response

PDK version used

0.0.0

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

MacOs

[FEATURE] [open-api-gateway] Smithy Support

Describe the feature

As an alternative to OpenAPI, it would be great if APIs could be defined with Smithy, a protocol-agnostic IDL.

Use Case

Defining APIs in an alternative, non-yaml format, less verbose than OpenAPI without the need to understand $ref and allOf etc.

Proposed Solution

New projen constructs SmithyApiGateway<Language>Project could be exposed by the open-api-gateway package which will generate the sample hello world api in Smithy IDL.

Smithy IDL can be converted to OpenAPI, so we would add this as a pre-step such that the generated clients, construct etc can be reused.

Smithy IDL allows for authorizers to be specified, so we would need to figure out a neat way to marry up those specified in the OpenApiGatewayLambdaApi CDK construct with those specified in the spec.

Other Information

Generated lambda handler wrappers would all still be generated and in theory would work with no change, however we might also wish to provide the option to use the Smithy Server SDK for Typescript as an alternative for writing type-safe lambda handlers in typescript.

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

PDK version used

0.8.x

What languages will this feature affect?

Typescript, Java, Python

Environment details (OS name and version, etc.)

All!

[BUG] KMS encryption of static-website bucket preventing cloudfront distribution.

Describe the bug

With the addition of a default KMS encryption on the website bucket, cloudfront is now unable to serve the files:

Expected Behavior

Cloudfront should be able to serve the website files through it's distribution.

Current Behavior

image

Reproduction Steps

Not really any sort of extra config outside of the default:

    new StaticWebsite(this, "StaticFrontend", {
      websiteContentPath: path.join(
        __dirname,
        "../../../frontend/build"
      ),
      runtimeOptions: {
        jsonPayload: {
          identityPoolId: userIdentity.identityPool.identityPoolId,
          userPoolId: userIdentity.userPool?.userPoolId,
          userPoolClientId: userIdentity.userPoolClient?.userPoolClientId,
          region: stack.region,
        },
      },
    });

The files are in s3

Possible Solution

It looks like a common fix for this is to provide a lambda@edge function to help with retrieving the files. Seems like overkill though...

Additional Information/Context

This is a fresh deployment with the latest version: 0.2.1

PDK version used

static-website: 0.2.1

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

OSX 12.3.1, deployed from local using CDK

[FEAT] Auto publishing packages to package managers

Being able to publish designated libs to package managers would be very useful.

For instance, if I had a monorepo as such:

A
 |- B (py)
 |- C (java)
 |- D (ts)
 |- E (ts)

If I wanted to say publish packages B, C and D to CodeArtifact we should support the following config on the PDKPipeline:

interface PublishConfig {
  packagesToPublish?: string[];
}

interface CodeArtifactPublishConfig extends PublishConfig {
   codeArtifactArn?: string; // If this is set, all other options are ignored
   authTokenSecretArn?: string; // defaults to a generated secret with the token pre-initialized
   authTokenRotation?: boolean; // defaults to true
   domainName: string;
   domainOwner?: string; //defaults to pipeline account
   repositoryName: string;
}

new pdk_pipeline.PDKPipeline(this, 'ApplicationPipeline', {
      primarySynthDirectory: 'packages/infra/cdk.out',
      repositoryName: 'monorepo',
      publishAssetsInParallel: false,
      crossAccountKeys: true,
      synth: {},
      publishConfig: {
          codeArtifactPublishConfig: {
              domainName: '<domain>',
              repositoryName: '<repo-name>',
              packagesToPublish: ['B', 'C', 'D']
          },
      }
    });

In this instance, the Pipeline is configured to publish artifacts to a a new CodeArtifact. In terms of how this would work, a high level algorithm is as follows:

  1. Scan the repo based on the packagesToPublish to identify candidate packages for publishing. (Not sure if we should fail silently or loudly if a package is listed for publishing which is not supported i.e: GO)
  2. Generate a .versionrc file in the root with a list of bumpFiles.
  3. bump the versions by using standard-version. We will need to potentially write custom updates for languages like python or java as we will need to update pom.xml and *.toml
  4. build all packages so that a dist dir is created in each package
  5. unbump all packages that were previously bumped
  6. run git diff --ignore-space-at-eol --exit-code (this makes sure src is identical to how it was before we bumped)
  7. Use npx publlib to publish each package

This can later be extended to have MavenPublishConfig, PyPiPublishConfig, etc.

[BUG] inter-dependencies between PDK packages will cause issues when migrated to stable

Describe the bug

inter-dependencies between PDK packages will cause issues when migrated to stable as they will expect an interface from say @aws-prototyping-sdk/foo by when packaged the interface will be exposed as aws-prototyping-sdk/foo which will lead to type errors;

Expected Behavior

inter-dependencies between PDK packages should be supported and when moved to stable, the correct types should be exposed.

Current Behavior

inter-dependencies between PDK packages will cause issues when migrated to stable as they will expect an interface from say @aws-prototyping-sdk/foo by when packaged the interface will be exposed as aws-prototyping-sdk/foo which will lead to type errors;

Reproduction Steps

N/A

Possible Solution

Re-write package imports for stable packages from @aws-prototyping-sdk to aws-prototyping-sdk.

Additional Information/Context

No response

PDK version used

0.2.8

What languages are you seeing this issue on?

Typescript, Java, Python

Environment details (OS name and version, etc.)

prod

[FEATURE] Smithy generated Server SDK (SSDK) for TS

Describe the feature

It looks like we generate types and values from OpenAPI tools. It would be nice to generate types, validators, etc using Smithy tools. Right now, it looks like most are unstable with the exception of typescript, which is in preview:

Use Case

From previous experiences, the code generation tends to be better coming from a Smithy Model vs going from Smithy -> OpenAPI -> OpenAPITools but not in any drastic way.

Proposed Solution

By default, a SmithyApiGatewayTsProject would use smithy code generation, but you can opt-in to OpenAPI generation

const api = new SmithyApiGatewayTsProject({
  clientLanguages: [ClientLanguage.TYPESCRIPT],
  defaultReleaseBranch: "main",
  name: "api",
  parent: monorepo,
  useOpenAPIGen: true, // <-- new flag
  outdir: "packages/api",
  serviceName: "aws.bumpercars#Bumpercars",
  deps: [
    "@aws-lambda-powertools/logger",
    "@aws-lambda-powertools/tracer",
    "@aws-lambda-powertools/metrics",
  ],
});

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

PDK version used

0.11.3

What languages will this feature affect?

Typescript

Environment details (OS name and version, etc.)

MacOS

[BUG] Doc build can still succeed even though there are partial failures

Describe the bug

It is possible for the docs build to succeed with partial failures as per: https://github.com/aws/aws-prototyping-sdk/actions/runs/2539820405

Expected Behavior

This should not be the case and the build should fail if any submodule fails to render docs.

Current Behavior

docs build succeeds even though there are partial failures.

Reproduction Steps

happens intermittently.

Possible Solution

N/A

Additional Information/Context

No response

PDK version used

0.2.8

What languages are you seeing this issue on?

Typescript, Java, Python

Environment details (OS name and version, etc.)

github

[BUG] AwsPrototypingChecks not compatible with cdk-nag v2.22.0 or above

Describe the bug

AwsPrototypingChecks is not compatible with cdk-nag v2.22.0 or above.

Expected Behavior

Expected to show security check results.

Current Behavior

The following error occurred and the security check results are not shown.

node_modules/cdk-nag/src/nag-pack.ts:120
      : params.rule.name;
                    ^
TypeError: Cannot read properties of undefined (reading 'name')
    at AwsPrototypingChecks.applyRule (node_modules/cdk-nag/src/nag-pack.ts:120:21)
    at node_modules/@aws-prototyping-sdk/pdk-nag/src/packs/aws-prototyping.ts:24:14
    at Array.forEach (<anonymous>)
    at AwsPrototypingChecks.visit (node_modules/@aws-prototyping-sdk/pdk-nag/src/packs/aws-prototyping.ts:23:20)
    at recurse (node_modules/aws-cdk-lib/core/lib/private/synthesis.js:1:1848)
    at recurse (node_modules/aws-cdk-lib/core/lib/private/synthesis.js:1:2176)
    at recurse (node_modules/aws-cdk-lib/core/lib/private/synthesis.js:1:2176)
    at recurse (node_modules/aws-cdk-lib/core/lib/private/synthesis.js:1:2176)
    at invokeAspects (node_modules/aws-cdk-lib/core/lib/private/synthesis.js:1:1474)
    at Object.synthesize (node_modules/aws-cdk-lib/core/lib/private/synthesis.js:1:542)

Reproduction Steps

  1. npm install @aws-prototyping-sdk/pdk-nag
  2. Add AwsPrototypingChecks usage to CDK application.
  3. cdk synth

Possible Solution

Possible cause

S3BucketServerSideEncryptionEnabled rule was removed from cdk-nag since v2.22.0.
https://github.com/cdklabs/cdk-nag/releases/tag/v2.22.0

Current workaround

Specify cdk-nag version 2.21.86 explicitly.

Additional Information/Context

No response

PDK version used

0.14.6

What languages are you seeing this issue on?

No response

Environment details (OS name and version, etc.)

macOS 13.2.1, node v16.19.0

[BUG] CDKGraph with CdkGraphDiagramPlugin not generating diagram

Describe the bug

Generating a graph with CdkGraphDiagramPlugin throws an error on the step build » post-compile » synth:silent | cdk synth -q.

Expected Behavior

Expected cdk.out directory to contain generated diagrams.

Current Behavior

Current error thrown:


 Instead change the require of index.js in .../node_modules/@aws-prototyping-sdk/cdk-graph-plugin-diagram/node_modules/cross-spawn/lib/util/resolveCommand.js to a dynamic import() which is available in all CommonJS modules.

Reproduction Steps

(async () => {
  const app = PDKNag.app();
  const graph = new CdkGraph(app, {
    plugins: [new CdkGraphDiagramPlugin()],
  });

  const pipelineStack = new PipelineStack(app, "PipelineStack", {
    env: {
      account: process.env.CDK_DEFAULT_ACCOUNT!,
      region: process.env.CDK_DEFAULT_REGION!,
    },
  });

  const devStage = new ApplicationStage(app, "Dev", {
    env: {
      account: process.env.CDK_DEFAULT_ACCOUNT!, // Replace with Dev account
      region: process.env.CDK_DEFAULT_REGION!, // Replace with Dev region
    },
  });

  pipelineStack.pipeline.addStage(devStage);

  const stagingStage = new ApplicationStage(app, "Stg", {
    env: {
      account: process.env.CDK_DEFAULT_ACCOUNT!, // Replace with Dev account
      region: process.env.CDK_DEFAULT_REGION!, // Replace with Dev region
    },
  });

  pipelineStack.pipeline.addStage(stagingStage, {
    pre: [
      new ManualApprovalStep("Pre-Staging Check", {
        comment: "I confirm that I am deploying to the staging environment.",
      }),
    ],
  });

  // Add additional stages here i.e. Prod

  pipelineStack.pipeline.buildPipeline(); // Needed for CDK Nag
  app.synth();
  await graph.report();
})().catch(err => console.error(err));

Additional Information/Context

Project is using CommonJS

PDK version used

1.31.0

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

MacOS Monterey 12.6.2 (21G320)

[FEATURE] Support Smithy IntelliJ plugin

Describe the feature

We should support the Smithy IntelliJ Plugin to improve the user experience when working with Smithy.

Use Case

The best I could set up in intellij was syntax highlighting by using the vs-code plugin as a textmate bundle, but this doesn't have proper auto-complete and error checking, making the developer experience worse than it could be.

Proposed Solution

No response

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

PDK version used

0.13.4

What languages will this feature affect?

Typescript, Java, Python

Environment details (OS name and version, etc.)

OSX

[DOCS] cdk-graph-plugin-diagram documentation could have quickstart documentation

Describe the issue

Hey there,

I'm new to typescript. I followed the guidance in https://github.com/aws/aws-prototyping-sdk/blob/mainline/README.md, then moved to https://github.com/aws/aws-prototyping-sdk/tree/mainline/packages/cdk-graph-plugin-diagram, as it was my primary purpose to experiment making CDK based graphs.

Issues I encountered:

  • I don't actually know how to run my typescript file. I chose visual studio code.
  • What is IIFE?
  • Can you explain to me how to point to a CDK folder/file, and load it? Quickstart line // ... add stacks, etc is not helpful to the noob.
  • There's a lot of implicit knowledge here that I lack. What do I need to do to get this to run if I am starting from scratch? Reference other guides where necessary.

Links

https://github.com/aws/aws-prototyping-sdk/blob/mainline/README.md

Feature: support PDKProject nx configuration and improve nx caching

Currently nx input/output configuration is only configurable for all projects with a default set which is managed by the pdk-monorepo-project itself. However many packages variant/additional "input" and/or "output" paths that if defined would benefit from more targeted caching control.

Current implementation

https://github.com/aws/aws-prototyping-sdk/blob/ac6c3aff0e92e6b9f9a264120a17f91f4646339c/private/projects/pdk-monorepo-project.ts#L193-L212

Example of package that would benefit

packages/aws-arch

  • Input

    • /scripts
    • /src
    • !/src/generated (exclude this)
  • Output

    • /assets
    • /dist
    • /lib
    • /src/generated

image


Additionally looks like test-reports should be added to default Outputs.

Also worth discussion, should we split out "build" and "test" target input / output configurations? I know that build currently calls test, but would consider separating these to benefit from nx caching. To this extends, what about separating out pre-compile, compile, etc.... so they can all be cached. This would require ensuring that nx run is utilized as well throughout to enable caching behaivor.

[BUG] NX Monorepo errors with @types/babel__traverse

Describe the bug

Linking the description and resolution for the bug from projen as I ran into this and they helped debug it.

projen/projen#2264

Expected Behavior

No errors during compile.

Current Behavior

� compile | jsii --silence-warnings=reserved-word
[2022-11-29T17:24:08.213] [ERROR] jsii/compiler - Compilation errors prevented the JSII assembly from being created
../../../node_modules/@types/babel__traverse/index.d.ts:68:50 - error TS1005: ']' expected.

68 export type ArrayKeys<T> = keyof { [P in keyof T as T[P] extends any[] ? P : never]: P };
                                                    ~~
../../../node_modules/@types/babel__traverse/index.d.ts:68:53 - error TS1005: ';' expected.

68 export type ArrayKeys<T> = keyof { [P in keyof T as T[P] extends any[] ? P : never]: P };
                                                       ~

Reproduction Steps

npm run compile

Possible Solution

Pinning @types/babel__traverse to 7.18.2

Additional Information/Context

No response

PDK version used

0.12.20

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

Linux x86_64

[BUG] additionalInterceptors is not iterable

Describe the bug

I'm following the steps here, and after deploying my stack and trying to run awscurl my lambda is failing with:

ERROR	Invoke Error 	
{
    "errorType": "TypeError",
    "errorMessage": "additionalInterceptors is not iterable",
    "stack": [
        "TypeError: additionalInterceptors is not iterable",
        "    at Runtime.handler (/var/task/index.js:504:42)",
        "    at Runtime.handleOnceNonStreaming (/var/runtime/Runtime.js:74:25)"
    ]
}

All additionalInterceptors references are generated. It is not something I added explicitly in my code.

My lambda implementation:

export const handler = sayHelloHandler(async ({ input }) => {
  return {
    statusCode: 200,
    headers: {
      "Access-Control-Allow-Origin": "*",
      "Access-Control-Allow-Headers": "*",
    },
    body: {
      message: `Hello ${input.requestParameters.name}!`,
    },
  };
});

Expected Behavior

Lambda should not with generated boilerplate code.

Current Behavior

Fail with additionalInterceptors is not iterable

Reproduction Steps

  1. Set up a new npx projen new --from aws-prototyping-sdk nx-monorepo
  2. Add API and infra
import {
  ClientLanguage,
  DocumentationFormat,
  SmithyApiGatewayTsProject,
} from "@aws-prototyping-sdk/open-api-gateway";
import { nx_monorepo } from "aws-prototyping-sdk";
import { AwsCdkTypeScriptApp } from "projen/lib/awscdk";

const monorepo = new nx_monorepo.NxMonorepoProject({
  defaultReleaseBranch: "main",
  devDeps: ["aws-prototyping-sdk", "@aws-prototyping-sdk/open-api-gateway"],
  name: "project",
});

const api = new SmithyApiGatewayTsProject({
  clientLanguages: [ClientLanguage.TYPESCRIPT],
  defaultReleaseBranch: "main",
  documentationFormats: [DocumentationFormat.HTML2],
  name: "api",
  parent: monorepo,
  outdir: "packages/api",
  serviceName: {
    namespace: "com.project",
    serviceName: "ApiService",
  },
});

new AwsCdkTypeScriptApp({
  defaultReleaseBranch: "main",
  parent: monorepo,
  outdir: "packages/infra",
  name: "infra",
  cdkVersion: "2.1.0",
  deps: [api.package.packageName],
});


monorepo.synth();
  1. Update infra to include SampleApi
new SampleApi(this, "sample-api");
  1. awscurl awscurl --service execute-api --region us-east-1 https://[ID].execute-api.us-east-1.amazonaws.com/prod/hello\?name\=boooo

Possible Solution

No response

Additional Information/Context

No response

PDK version used

0.12.33

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

macOS Monterey Version 12.5

[BUG] `NxMonorepoProject` enables prettier

Describe the bug

NxMonorepoProject force-enables prettier.

https://github.com/aws/aws-prototyping-sdk/blob/a458fae9ce3841b945746f5f3f58fc3877787255/packages/nx-monorepo/src/nx-monorepo.ts#L216

Expected Behavior

Allow package consumer to set their own value.

Current Behavior

The value is hardcoded.

Reproduction Steps

https://github.com/aws/aws-prototyping-sdk/blob/a458fae9ce3841b945746f5f3f58fc3877787255/packages/nx-monorepo/src/nx-monorepo.ts#L216

Possible Solution

Do not force-set the value.

Additional Information/Context

I don't use prettier, and don't want the extra deps or files around.

PDK version used

latest

What languages are you seeing this issue on?

Typescript

Environment details (OS name and version, etc.)

macOS

[BUG] Cannot use api test console when using lambda integration

Describe the bug

Since https://github.com/aws/aws-prototyping-sdk/blob/mainline/packages/open-api-gateway/src/construct/integrations/integrations.ts#L260 grants access only to the deployment stage, the api gateway test console (which uses a stage named "test-invoke-stage") fails invoking any lambda with an error Execution failed due to configuration error: Invalid permissions on Lambda function

I suggest either adding this stage to the lambda resource policy generated, or using a "*" for stage name in the same way api gateway does when associating a lambda with a method

Expected Behavior

I can invoke a lambda integration through the AWS Api gateway test console

Current Behavior

The lambda fails with "Invalid permissions on Lambda function"

Reproduction Steps

Generate an api gateway with a lambda integration using the @aws-prototyping-sdk/open-api-gateway module

Possible Solution

I submitted a PR changing from the specific stage to a star permission for the lambda integration

Additional Information/Context

No response

PDK version used

0.11.4

What languages are you seeing this issue on?

No response

Environment details (OS name and version, etc.)

Any

[FEATURE] make affected.defaultBase configurable in nx.json

Describe the feature

Right now affected.defaultBase in the generated nx.json is hardcoded to mainline:

https://github.com/aws/aws-prototyping-sdk/blob/4881808bdccb2f3e53df5fc96acc186612634bed/packages/nx-monorepo/src/nx-monorepo.ts#L300-L302

It would be nice if this could be configured in the projen config file.

Use Case

When not using a mainline branch, some nx commands will fail. For example, npx nx print-affected fails with

Error: Command failed: git merge-base --fork-point "mainline" "HEAD"
fatal: No such ref: 'mainline'

Proposed Solution

The branch could be specified as a new parameter in the projen file, or it could be set to options.defaultReleaseBranch (I'm new to nx and not sure which is more appropriate).

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

PDK version used

v0.11.1

What languages will this feature affect?

No response

Environment details (OS name and version, etc.)

Mac OS 12.3.1

[BUG] support lookups for CDK Graph resources defined outside of stack

Describe the bug

When trying to use CdkGraph on a stack that references a Lambda external to current stack (i.e. Function.fromName() ) it errors out.

Expected Behavior

For lookups to be handled and resources defined outside of stack to be displayed in diagram.

Current Behavior

Throws error

Error: LogicalId defined outside of stack: ...

Reproduction Steps

Include a lambda function from lookup in stack to be graphed.

Possible Solution

No response

Additional Information/Context

No response

PDK version used

0.13.0

What languages are you seeing this issue on?

No response

Environment details (OS name and version, etc.)

macOS 12.6.1

Nag error - AwsPrototyping-VPCDefaultSecurityGroupClosed

Discussed in #256

Originally posted by drskur December 16, 2022

Node Version

v16.17.0

Deps Version

  • "aws-prototyping-sdk": "^0.12.32"
  • "aws-cdk-lib": "^2.53.0"
  • "cdk-nag": "^2.21.4"
  • "constructs": "^10.1.162"

The following error occurred while using the PDK.

[Error at /Dev/ProductSearchStack/Vpc/ProductVpc/Resource] AwsPrototyping-VPCDefaultSecurityGroupClosed: The VPC's default security group allows inbound or outbound traffic.

This error does not occur if AwsPrototypingChecks rule is not used.

const app = PDKNag.app();

It is difficult, because there is no properties to edit default security group, on the Vpc Construct.

Here is my code.

const app = PDKNag.app({
  nagPacks: [new AwsPrototypingChecks()],
});
this.vpc = new Vpc(this, "ProductVpc", {
  vpcName: "product-search",
  subnetConfiguration: [
    {
      cidrMask: 24,
      name: "ingress",
      subnetType: SubnetType.PUBLIC,
      mapPublicIpOnLaunch: false,
    },
    {
      cidrMask: 28,
      name: "rds",
      subnetType: SubnetType.PRIVATE_ISOLATED,
    },
    {
      cidrMask: 28,
      name: "application",
      subnetType: SubnetType.PRIVATE_WITH_EGRESS,
    },
  ],
  flowLogs: {
    cloudwatch: {
      destination: FlowLogDestination.toCloudWatchLogs(cloudWatchLogs),
      trafficType: FlowLogTrafficType.ALL,
    },
  },
});

NxMonorepo scripts/tasks should be for workspace operations

Currently nx-monorepo project uses the default project tasks (build/precompile/compile/watch/etc) which don't have much value for the root workspace. These should be using workspace operations utilizing npx nx run-many format.

https://github.com/aws/aws-prototyping-sdk/blob/f4f5401f37aa0a81918114bf4c62560c4636e5b0/package.json#L3-L20

Expected:

  • build: npx nx run-many --target=build --all --parallel=4 --output-style stream --nx-bail
  • test: npx nx run-many --target=test --all --parallel=4 --output-style stream --nx-bail
  • ...

Once this is completed, the github workflows could use the rather than having inline for consistency and maintainability.

[FEAT] PR Build Checker

We should expose a capability in the PDKPipeline to run automated builds when a PR is opened or changed.

Features to implement:

  • Option to allow the PR Build Checker to approve PR's (This would mean the Build Job would call the approve API against the PR)
  • Should provide updates via comments on the PR in terms of starting, failed, succeeded with a link to logs.

[FEAT] Explicit gating for SonarScanner as an option

Currently the SonarScanner runs asynchronously via Eventbridge. We should support an option in the PDKPipeline to have the SonarScanner act as an approval gate.

For a pipeline perspective, this would mean a new CodeBuild task will be executed after the Synth action within the Build Stage.

[BUG] Smithy Service name must be defined ahead of time

Describe the bug

When you generate a sample project, then manually overwrite the smithy sample generated. npx projen will not rebuild the smithy model.

Expected Behavior

Able to overwrite the generated smithy model file and rebuild to get new types.

Current Behavior

npx projen will not recognize input change and will not build out new model

Reproduction Steps

1.) generate smithy open api
2.) manually change namespace, service, and operations
3.) Run npx projen

Possible Solution

This was fixed by deleted the generated API, then typing the service name ahead of time:

const api = new SmithyApiGatewayTsProject({
  clientLanguages: [ClientLanguage.TYPESCRIPT],
  defaultReleaseBranch: "main",
  name: "api",
  parent: monorepo,
  outdir: "packages/api",
  serviceName: "aws.bumpercars#Bumpercars", <-- add this, then npx projen
  deps: [
    "@aws-lambda-powertools/logger",
    "@aws-lambda-powertools/tracer",
    "@aws-lambda-powertools/metrics",
  ],
});

Additional Information/Context

No response

PDK version used

0.11.3

What languages are you seeing this issue on?

No response

Environment details (OS name and version, etc.)

MacOS

[FEATURE] Support zero-config KMS encryption for static-website.

Describe the feature

Cloudfront OAI does not currently support KMS encryption on buckets and as such a user needs to use S3_MANAGED encryption or follow this guide: https://aws.amazon.com/blogs/networking-and-content-delivery/serving-sse-kms-encrypted-content-from-s3-using-cloudfront/.

Use Case

Highly regulated customers which mandate using customer managed keys i.e. banks.

Proposed Solution

A Lambda@Edge function will need to be associated to the distribution which will sign the request on OAI's behalf as per: https://aws.amazon.com/blogs/networking-and-content-delivery/serving-sse-kms-encrypted-content-from-s3-using-cloudfront/.

This lambda must exist in us-east-1 and as such a custom resource will need to be created such that it can be created agnostic of the host region. This lambda will need to intercept origin requests to S3, sign the request using it's own credentials (s3 ro access + kms decrypt).

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

PDK version used

N/A

What languages will this feature affect?

Typescript, Java, Python

Environment details (OS name and version, etc.)

N/A

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.