cdklabs / cdk-from-cfn Goto Github PK

A cloudformation json -> cdk typescript transpiler.

License: Apache License 2.0

Rust 97.72% TypeScript 0.37% Go 0.54% Java 0.51% Python 0.36% C# 0.51%

cdk-from-cfn's Introduction

cdk_from_cfn

In a world where people want to use the full extent of the cdk, there was no product that would transform all your JSON/YAML into beautiful typescript...until now.

cdk_from_cfn will take your JSON/YAML and output the equivalent typescript.

User Guide

$ cargo build --release
$ ./target/release/cdk_from_cfn [INPUT] [OUTPUT]

INPUT is the input file path (STDIN by default).
OUTPUT is the output file path; if not specified, output will be printed on your command line (STDOUT by default).

Cargo Features

Name	Enabled by default	Description
`typescript`	✔️	Enables support for TypeScript output
`golang`	✔️	Enables support for Go output
`java`	✔️	Enables support for Java output
`Python`	✔️	Enables support for Python output
`csharp`	✔️	Enables support for C# output

You can enable experimental languages (not enabled by default) by enabling the relevant feature:

$ cargo build --release --features=<feature-name>
Finished release [optimized] target(s) in 0.17s

If you want to disable on-by-default languages, you can pass --no-default-features:

$ cargo build --release --no-default-features --features=golang
Finished release [optimized] target(s) in 0.17s

Implemented

Remaining

There are known unsupported features. Working on them in priority order:

Create policy
ssm metadata references
secretsmanager references

cdk-from-cfn's People

Contributors

Stargazers

Watchers

Forkers

roblaks jfkisafk sandeepklr vtanguturi adwhalen kexinlu fouwels msjawad hbobertz wongni therealamazonkendra sattigar winteryukky chamcca pr0methean

cdk-from-cfn's Issues

Parsing issue for intrinsic function Fn::Sub.

If the value for the intrinsic function Fn::Sub does not contain a variable to be replaced, starting with "${" ending with "}", then parsing fails.

"Description": { "Fn::Sub": "Some value without a variable to be replaced" }

Error occurs at this line of code: delimited(tag("${"), take_until("}"), take(1usize))

fn inner_resolver(str: &str) -> IResult<&str, SubValue> {
    let ir = alt((
        map(
            delimited(tag("${"), take_until("}"), take(1usize)),
            |var: &str| SubValue::Variable(var.to_string()),
        ),
        map(take_until("${"), |static_str: &str| {
            SubValue::String(static_str.to_string())
        }),
    ))(str);

    let (remaining, res) = ir?;
    IResult::Ok((remaining, res))
}

Custom resources aren't resolving

If you use a custom resource, it will explode:

Error: TransmuteError: invalid resource type name: Custom::<NAME_OF_CUSTOM_RESOURCE>

YAML template conversion fails when intrinsic functions using ! are present

Any place using intrinsic functions containing ! instead of Fn:: will fail when converting YAML. The code sample below will fail while the second sample below will succeed. When using JSON format for the template, there is no issue which rules out a template issue.

IsUsEast1: !Equals 
  - !Ref 'AWS::Region'
  - us-east-1

IsUsEast1: 
    Fn::Equals:
    - Fn::Ref 'AWS::Region'
    - us-east-1

go: synthesizing a template with CFN properties of type json fails

a template including this:

"MyRDMessageQueueGroup" : {
      "Type" : "AWS::IAM::Group",
      "Properties" : {
        "Policies": [ {
          "PolicyName": "MyQueueGroupPolicy",
          "PolicyDocument": {
            "Statement":[ {
              "Effect":"Allow",
 	      "Action":[ "sqs:DeleteMessage", "sqs:ReceiveMessage" ],
              "Resource":[ { "Fn::GetAtt" : ["MyQueue1", "Arn"]}, { "Fn::GetAtt" : ["MyQueue2", "Arn"]} ]
            } ]
          }
        } ]
      }
    }

Will fail to synthesize to go with the error:

thread 'main' panicked at 'internal error: entered unreachable code: object with simple structure (Json)', src/synthesizer/golang/mod.rs:637:29

This is because the type of a PolicyDocument is json (docs), and the go synthesizer doesn't currently handle this case https://github.com/iph/noctilucent/blob/e75176e31e4af8b157d28b825ea9f995e4c7bcae/src/synthesizer/golang/mod.rs#L631.

Minor improvement - remove many non-null assertions

It's not good to use the non-null assertion operator if we can avoid it, and we can fix the way defaults are applied to props to make this not needed.

We can change the following output

public constructor(scope: cdk.App, id: string, props: NoctStackProps) {
    super(scope, id, props);

    // Applying default props
    props = {
      ...props,
      myProperty: props.myProperty ?? '',

To the below output

public constructor(scope: cdk.App, id: string, originalProps: NoctStackProps) {
    super(scope, id, originalProps);

    // Applying default props
    const props = {
      ...originalProps,
      myProperty: originalProps.myProperty ?? '',

And now props has non nullable types and does not need non-null assertions everywhere.

Support for AWS::Serverlesss::* resources referencing non existent library "aws-cdk-lib/aws-serverless"

Generated application using CDK Migrate from a cloudfromation template containing AWS::Serverlesss::* resources such as AWS::Serverlesss::Function fail after migration during synth as the output uses “import * as serverless from 'aws-cdk-lib/aws-serverless';” but there is no such library. Possibly solutions are either to create such a library (discussed Async @TheRealAmazonKendra as we may be able to help but such resolution would be in a different repo) or to use the workaround below for this tool.

During cdk synth

TSError: ⨯ Unable to compile TypeScript:
lib/testcdkmigrate-stack.ts:3:29 - error TS2307: Cannot find module 'aws-cdk-lib/aws-serverless' or its corresponding type declarations.
3 import * as serverless from 'aws-cdk-lib/aws-serverless';

Workaround change is to manually edit the CDK to use the SAM libarary
Change

import * as serverless from 'aws-cdk-lib/aws-serverless';

import * as serverless from 'aws-cdk-lib/aws-sam';

wrong output - boolean properties

props.myProperty! === 'true';

should be

props.myProperty! === true;

Intrinsic function Fn::GetAzs being translated incorrectly.

Intrinsic function Fn::GetAzs is being translated wrongly as cdk.Fn.getAZs. It should be cdk.Fn.getAzs as per the library.

Quoted numbers in yaml sometimes cause an error

There's inconsistent handling of quoted numbers in yaml CloudFormation templates. Consider this example:

AWSTemplateFormatVersion: 2010-09-09
Parameters:
  Param1:
    Type: Number
    Default: !!int "0"  # This line does not cause an error.
Conditions:
  Condition1:
    "Fn::Equals":
    - "Fn::Select":
      - !!int "0"  # This line causes an error unless you remove the quotes around the zero.
      - "Fn::Split":
        - "delimiter"
        - "string"
    - ""
Resources:

The first instance of !!int "0" is accepted by noctilucent, but the second instance causes this error:

Error: Conditions.Condition1.Fn::Equals[0].Fn::Select[0]: invalid type: string "0", expected usize at line 10 column 9

Vague error message related to resource property

This input causes an error. Note the use of SourceSecurityGroupId in a SecurityGroupEgress. CloudFormation accepts this template and creates a stack, but Noctilucent produces a really vague error message.

AWSTemplateFormatVersion: 2010-09-09
Resources:
  SG1:
    Type: 'AWS::EC2::SecurityGroup'
    Properties:
      GroupDescription: 'SG1'
      VpcId: !Ref TestVPC
  SG2:
    Type: 'AWS::EC2::SecurityGroup'
    Properties:
      VpcId: !Ref TestVPC
      GroupDescription: 'SG2'
      SecurityGroupEgress:
        IpProtocol: 'TCP'
        FromPort: 10000
        ToPort: 10000
        SourceSecurityGroupId: !Ref SG1
  TestVPC:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: '10.0.0.0/16'

The output looks like this:

thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/ir/resources/mod.rs:129:68
stack backtrace:
   0:        0x105483b87 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hae51cb91d407e2ef
   1:        0x1054195db - core::fmt::write::h746bc0969202388b
   2:        0x10545ee3c - std::io::Write::write_fmt::h4098c2c7437a0bd7
   3:        0x105487704 - std::sys_common::backtrace::print::h8360bf0158e89b36
   4:        0x1054874c9 - std::panicking::default_hook::{{closure}}::hedf04c568eb6e0bc
   5:        0x105487a67 - std::panicking::rust_panic_with_hook::h22edd02828aff274
   6:        0x105487878 - std::panicking::begin_panic_handler::{{closure}}::h6895f44a9b7e2caa
   7:        0x105487809 - std::sys_common::backtrace::__rust_end_short_backtrace::h013519e9b77978c5
   8:        0x1054877f2 - _rust_begin_unwind
   9:        0x10549ea63 - core::panicking::panic_fmt::h2a51b60c12d11322
  10:        0x10549eb47 - core::panicking::panic::h00b4c20719c0abc2
  11:        0x105437484 - noctilucent::ir::resources::ResourceTranslator::translate::hf043457459ff11fa
  12:        0x10542f4e7 - noctilucent::ir::resources::ResourceInstruction::from::ha9a604782ce9b876
  13:        0x10542a9e2 - noctilucent::ir::CloudformationProgramIr::from::hbee5e1fbc7775a35
  14:        0x1053d706e - noctilucent::main::hde7d4cf2812ec0cf
  15:        0x1053ae516 - std::sys_common::backtrace::__rust_begin_short_backtrace::hcdb722f1a5cc110d
  16:        0x1053d8449 - _main

There are actually two issues here. The first is that the template causes an error. The second is that the error message doesn't give the user any information they can use to locate the problem in the input. There's no line number, property name, resource name, etc.

wrong output - updatePolicy casing

    lambdaFunctionAliaslive.cfnOptions.updatePolicy = {
      CodeDeployLambdaAliasUpdate: {
        ApplicationName: serverlessDeploymentApplication.ref,
        DeploymentGroupName: lambdaFunctionDeploymentGroup.ref,
      },
    };

Looks like it should be camelCased:

    lambdaFunctionAliaslive.cfnOptions.updatePolicy = {
      codeDeployLambdaAliasUpdate: {
        applicationName: serverlessDeploymentApplication.ref,
        deploymentGroupName: lambdaFunctionDeploymentGroup.ref,
      },
    };

Also lambdaFunctionDeploymentGroup was output after this, causing "Variable 'lambdaFunctionDeploymentGroup' is used before being assigned."

cdk migrate cli UX issue

When not supplying stack name an error with unclear messaging is shown. Expectation state the missing parameters

$ ./node_modules/cdk/bin/cdk migrate --from-path template.yaml  --output-path cdkstack/
  ❌  Migrate failed for undefined: stack generation failed due to error 
'The text and separator arguments should be of type string'

supplying --stack-name test as well resolves the issue but it was unclear that this was needed from the error.

Fn::Import causes logical error in emission

Repro -- valid template

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Parameters": {
        "RefKey": {
            "Type": "String",
        }
    },
    "Resources": {
        "WaitHandle": {
            "Type": "AWS::CloudFormation::WaitConditionHandle",
            "Timeout": {
                "Fn::Import": {
                    "Ref": "RefKey"
                }
            }
        },
    }
}

Expected Result

Exporting of a template with fn import.

Actual Result

Logical error, it seems Noctilucent truncated valid data:

import * as cdk from 'aws-cdk-lib';
import * as cloudformation from 'aws-cdk-lib/aws-cloudformation';

export interface NoctStackProps extends cdk.StackProps {
  /**
   */
  readonly refKey: string;
}

export class NoctStack extends cdk.Stack {
  public constructor(scope: cdk.App, id: string, props: NoctStackProps) {
    super(scope, id, props);

    // Resources
    const waitHandle = new cloudformation.CfnWaitConditionHandle(this, 'WaitHandle', {
    });
  }
}

Does not support using a single value instead of list for Transform

CDK migrate does not support using a single value instead of list in at least this case (maybe others? not checked).

In a template where the transform value is specified as a single value rather than list which is common in examples/docs e.g. https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-specification-template-anatomy.html)
i.e. this line

Transform: AWS::Serverless-2016-10-31

Results in this Error when using CDK migrate

This is an experimental feature. We make no guarantees about the outcome or stability of the functionality.
 ❌  Migrate failed for testcdkmigrate: stack generation failed due to error 'Transform: invalid type: string "AWS::Serverless-2016-10-31", expected a sequence at line 1 column 12'

Changing the template to use a list instead works as a workaround

Transform: 
  - AWS::Serverless-2016-10-31

Expectations are both formats work.

java: use java conventions instead of CloudFormation ones

A lot of the Java output still uses CloudFormation conventions instead of Java-isms.

Recommend adding license headers to source files

Having license headers on files is a best practice, providing clarity when those files are disconnected from the top level LICENSE file.

For this project I believe the default license header would be:

// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0 OR MIT

csharp: support for synthesizing apps in C#

Status: Work in Progress

To implement

Challenges unique to C#

The casing for using statements is not consistent. Acronyms are all caps (Amazon.CDK.IAM) but names of services are capitalized (Amazon.CDK.Lambda). To know which is which, we need to fetch this from somewhere.
Defining the type for each input prop
Defining the type for each output prop
Type names for composite nested properties need to be explicitly included - i.e. the WebsiteConfiguration property must be declared with new CfnBucket.WebsiteConfigurationProperty(C# doc link)

Optimizations

Only create local variables for resources that will be referenced again later
Extract shared code snippets to helper functions. (I didn't do this yet, because too much abstraction makes it harder to read and understand. So I'm not sure it's worth it to avoid a little bit of repetition)

all languages: type not properly translated into booleans

When a template has the following:

"Type": "String",
 "Default": "false",
 "AllowedValues": ["false", "true"]

The CDK translates this into a boolean but cdk-from-cfn translates it into a string across all languages. This is causing every language to fail for the parameters block in this template

typescript: CfnOutputs aren't correctly populated

We are only assigning them to class fields at present but they should all be emitted as CfnOutputs.

cdk synth error on generated code

I started with this template: https://github.com/awslabs/aws-cloudformation-templates/blob/master/aws/solutions/LambdaStaticIP/lambda-static.cfn.yaml
Downloaded it locally and run "cdk migrate --stack-name "evgeny" --language "typescript" --from-path lambda-static.cfn.yaml"
The CDK app was generated successfully.
Run "cdk synth" generates "Error: There is already a Construct with name 'PublicSubnetOne' in EvgenyStack [evgeny]"
cdk version: 2.106.1 (build a2e5f65)

Single Dependency not resolving

Repro

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Conditions": {
        "EmitAccountId": {
            "Fn::Equals": [
                "true",
                "false"
            ]
        }
    },
    "Resources": {
        "WaitHandle": {
            "Type": "AWS::CloudFormation::WaitConditionHandle"
        },
    },
    "Outputs": {
        "Condition": "EmitAccountId",
        "AccountId": {
            "Value": {
                "Ref": "AWS::AccountId"
            }
        }
    }
}

Expected Results

To see an emitted template

Actual Results

Error: Outputs.Condition: invalid type: string "EmitAccountId", expected struct Output at line 17 column 22

Support exporting as a Construct vs a Stack

At the moment, noctilucent only exports as a stack. However, our use-case is a hybrid CDK + JSON stack , where we load the JSON using CfnInclude, and then retrieve a CDK reference to the resource using Cfn.getResource. It'd be great to have a command-line option to instead export as a Construct for this use-case.

README shows wrong command

README says ./target/release/cdk_from_cfn [INPUT] [OUTPUT]

But it should be ./target/release/cdk-from-cfn [INPUT] [OUTPUT]

Also great project!

Support for AWS::Serverlesss::* resources referencing non existent library "aws-cdk-lib/aws-serverless"

During cdk synth

TSError: ⨯ Unable to compile TypeScript:
lib/testcdkmigrate-stack.ts:3:29 - error TS2307: Cannot find module 'aws-cdk-lib/aws-serverless' or its corresponding type declarations.
3 import * as serverless from 'aws-cdk-lib/aws-serverless';

Workaround change is to manually edit the CDK to use the SAM libarary
Change

import * as serverless from 'aws-cdk-lib/aws-serverless';

import * as serverless from 'aws-cdk-lib/aws-sam';

Add support for --verbose in cdk-from-cfn and aws-cdk

cdk-from-cfn currently does not surface much helpful error messaging for debugging purposes. One example of which being #187. We need to add additional error logging, and debug logging across cdk-from-cfn and surface it to the user depending on a --verbose flag. We also need to surface this in our transmute function so that we can pass a --verbose flag from the cdk cli to cdk-from-cfn

ops: setup publishing

We need GitHub actions to automatically publish new versions so they can be consumed by the cdk cli.

support for Fn::ForEach

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-foreach.html

Multiline strings aren't correctly formatted in python

The template

  LambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: StaticExternalIp
      Description: Lambda Function with code to call an external service
      Code:
        ZipFile: !Sub |
          const https = require('https');

          exports.handler = async (event) => {
            const url = 'https://api.ipify.org';
            let data = "";

            return new Promise((resolve) => {
              https.get(url, res => {

              res.on('data', chunk => { data += chunk }) 

              res.on('end', () => {
                resolve(data);
                })
              }) 
            })
          };

is formatted into a string that does not synth.

python: lambda imports fail

lambda seems to be a protected term in python because import aws_cdk.lambda as lambda cannot find the import. The error resolved when this is changed to import aws_cdk.aws_lambda as aws_lambda. We may want to consider prefixing all our imports with aws_ in case there are other cases of this we have not yet discovered.

CDK migrate