Git Product home page Git Product logo

state-machine's Introduction

Workflow Studio compatible State Machine

View on Construct Hub

This is a Workflow Studio compatible AWS Step Function state machine construct.

The goal of this construct is to make it easy to build and maintain your state machines using the Workflow Studio but still leverage the AWS CDK as the source of truth for the state machine.

Read more about it here.

How to Use This Construct

Start by designing your initial state machine using the Workflow Studio. When done with your first draft, copy and paste the ASL definition to a local file.

Create a new instance of this construct, handing it a fully parsed version of the ASL. Then add overridden values. The fields in the overrides field should match the States field of the ASL.

Version Usage

The AWS CDK StateMachine construct introduced a change in version 2.85.0 that deprecated an earlier usage of 'definition' by this construct. This construct has been updated to use the new 'definitionBody' field.

If you are using a version of the CDK before version 2.85.0, you should use version 0.0.28 of this construct.

If you are using a version fo the CDK great or equal to 2.85.0, you should use version 0.0.29+ of this construct.

Projen component

There is a projen component included in this library which will help you in using the construct. It works similar to the auto-discovery feature. To use it, first add the component to your projen project:

// ...
const { StepFunctionsAutoDiscover } = require('@matthewbonig/state-machine');

const project = new awscdk.AwsCdkTypeScriptApp({
  // ...,
  deps: [
    // ...,
    '@matthewbonig/state-machine',
  ]
});

new StepFunctionsAutoDiscover(project);

Now projen will look for any files with a suffix .workflow.json and generate new files beside the .json:

  • A typed overrides interface which is based on your workflow.
  • A construct derived from StateMachine that uses this override.

Instead of using the StateMachine construct directly you can now use the generated one:

.
├── MyFancyThing.workflow.json
└── MyFancyThing-statemachine.ts
export class SomeStack extends Stack {
  constructor(scope: Construct, id: string, props: StackProps) {
    super(scope, id, props);
    const handler = new NodejsFunction(this, 'MyHandler');
    new SomeFancyThingStateMachine(this, 'MyFancyWorkflow', {
      overrides: {
        'My First State': {
          Parameters: {
            FunctionName: handler.functionName
          }
        }
      }
    })
  }
}

⚠️ The interfaces and constructs generated here are NOT jsii compliant (they use Partials and Omits) and cannot be compiled by jsii into other languages. If you plan to distribute any libraries you cannot use this.

Alternative Extensions

There is an optional parameter, extension that you can pass to have it search for alternative extensions. AWS recommends that ASL definition files have a .asl.json extension, which will be picked up by some IDE tools. This extension was recommended after initial development of this component. Therefore, the default is to use the original extension. But, you can override this by passing a different extension to the AutoDiscover's constructor options. There are two constants defined, JSON_STEPFUNCTION_EXT and AWS_RECOMMENDED_JSON_EXT that you can use.

// ...
const { StepFunctionsAutoDiscover, AWS_RECOMMENDED_JSON_EXT } = require('@matthewbonig/state-machine');

const project = new awscdk.AwsCdkTypeScriptApp({
  // ...,
  deps: [
    // ...,
    '@matthewbonig/state-machine',
  ]
});

new StepFunctionsAutoDiscover(project, { extension: AWS_RECOMMENDED_JSON_EXT });

Yaml files

Yaml files are supported as well. You can provide an extension to the AutoDiscover component to have it search for yaml files. If the file has 'yaml' or 'yml' anywhere in the name it will be parsed as yaml. If not, it will be parsed as json.

// ...
const { StepFunctionsAutoDiscover } = require('@matthewbonig/state-machine');

const project = new awscdk.AwsCdkTypeScriptApp({
  // ...,
  deps: [
    // ...,
    '@matthewbonig/state-machine',
  ]
});

new StepFunctionsAutoDiscover(project, { extension: '.yaml.asl' });

Examples

const secret = new Secret(stack, 'Secret', {});
new StateMachine(stack, 'Test', {
  stateMachineName: 'A nice state machine',
  definition: JSON.parse(fs.readFileSync(path.join(__dirname, 'sample.json'), 'utf8').toString()),
  overrides: {
    'Read database credentials secret': {
      Parameters: {
        SecretId: secret.secretArn,
      },
    },
  },
});

You can also override nested states in arrays, for example:

new StateMachine(stack, 'Test', {
    stateMachineName: 'A-nice-state-machine',
    overrides: {
      Branches: [{
        // pass an empty object too offset overrides
      }, {
        StartAt: 'StartInstances',
        States: {
          StartInstances: {
            Parameters: {
              InstanceIds: ['INSTANCE_ID'],
            },
          },
        },
      }],
    },
    stateMachineType: StateMachineType.STANDARD,
    definition: {
      States: {
        Branches: [
          {
            StartAt: 'ResumeCluster',
            States: {
              'Redshift Pass': {
                Type: 'Pass',
                End: true,
              },
            },
          },
          {
            StartAt: 'StartInstances',
            States: {
              'StartInstances': {
                Type: 'Task',
                Parameters: {
                  InstanceIds: [
                    'MyData',
                  ],
                },
                Resource: 'arn:aws:states:::aws-sdk:ec2:startInstances',
                Next: 'DescribeInstanceStatus',
              },
              'DescribeInstanceStatus': {
                Type: 'Task',
                Next: 'EC2 Pass',
                Parameters: {
                  InstanceIds: [
                    'MyData',
                  ],
                },
                Resource: 'arn:aws:states:::aws-sdk:ec2:describeInstanceStatus',
              },
              'EC2 Pass': {
                Type: 'Pass',
                End: true,
              },
            },
          },
        ],
      },
    },
  });

For Python, be sure to use a context manager when opening your JSON file.

  • You do not need to str() the dictionary object you supply as your definition prop.
  • Elements of your override path do need to be strings.
secret = Secret(stack, 'Secret')

with open('sample.json', 'r+', encoding='utf-8') as sample:
    sample_dict = json.load(sample)

state_machine = StateMachine(
    self,
    'Test',
    definition = sample_dict,
    overrides = {
    "Read database credentials secret": {
      "Parameters": {
        "SecretId": secret.secret_arn,
      },
    },
  })

In this example, the ASL has a state called 'Read database credentials secret' and the SecretId parameter is overridden with a CDK generated value. Future changes can be done by editing, debugging, and testing the state machine directly in the Workflow Studio. Once everything is working properly, copy and paste the ASL back to your local file.

Issues

Please open any issues you have on Github.

Contributing

Please submit PRs from forked repositories if you'd like to contribute.

state-machine's People

Contributors

a-bigelow avatar dmytrokosiachenko avatar mbonig avatar rubenfonseca avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

state-machine's Issues

projen mistakenly as (non dev)dependency

Hi there

We thought about using your construct as it seemed to fit our needs (for replacing account related hardcoded ids in our step function), thanks for creating it :)

However, currently projen is listed as dependency in

"projen": "^0.71.34"

Probably thats only added by mistake as consumers should'nt need this dependency when using this construct.

Set projen peerDeps using ">="

I'm seeing these warnings when using this construct in a projen managed project:

warning " > @matthewbonig/[email protected]" has incorrect peer dependency "projen@^0.65.74".

Also, I get type mismatches when trying to use the StepFunctionsAutoDiscover integration.

I think the peerDep for projen should be specified like projen@>=0.65.74 to allow newer minor versions of projen.

peerDeps: ['projen', 'constructs'],

File extension change request

For convenient usage in Visual Studio Code there is a suggestion from AWS to use *.json.asl file extension. Currently with *.workflow.json extension Visual Studio Code can not define it automatically correctly as ASL file.

RFC: Provide an alternate method for overriding ASL objects

The current method of positionally overriding parameters within ASL gets pretty confusing with larger ASL documents.

It would be awesome to have 1-2 alternate methods of overriding, e.g.

Recursive find and replace

Give the overrides prop a list of replacement objects, sorta like this:

{ "string" : "MyFunctionNamePlaceholder" , "replaceWith" : myFunction.functionName }

The construct then recurses through the ASL and replaces all occurrences of "MyFunctionNamePlaceholder" with myFunction.functionName

JSON Path overrides

Give the overrides prop a list of JSON path elements, and the value to replace them with

{ "path" : "$.States.Branches[0].Parameters.FunctionName" , "replaceWith" : myFunction.functionName }

Let's discuss. I'd like to help implement whatever change we decide on.

Update to not use deprecated prop `StateMachineProps#definition`

When using this dependency for our stepfunctions we are seeing this error:

[WARNING] aws-cdk-lib.aws_stepfunctions.StateMachineProps#definition is deprecated.
  use definitionBody: DefinitionBody.fromChainable()
  This API will be removed in the next major release.

Any chance of updating to not use this deprecated prop?

Thanks!

Attempting to override Array elements causes new elements to be created

I'm trying to override some parameters in multiple members of a Branches array. Attempting to override the parameters by simply providing providing my own array is causing duplicate members to be created, rather than overriding the specified values on the existing array members.

Given this ASL array:

ASL Array
     "Branches": [
       {
         "StartAt": "ResumeCluster",
         "States": {
           "ResumeCluster": {
             "Type": "Task",
             "Parameters": {
               "ClusterIdentifier": "MyData"
             },
             "Resource": "arn:aws:states:::aws-sdk:redshift:resumeCluster",
             "Next": "DescribeClusters"
           },
           "DescribeClusters": {
             "Type": "Task",
             "Parameters": {
               "ClusterIdentifier": ""
             },
             "Resource": "arn:aws:states:::aws-sdk:redshift:describeClusters",
             "Next": "Evaluate Cluster Status"
           },
           "Evaluate Cluster Status": {
             "Type": "Choice",
             "Choices": [
               {
                 "Variable": "$.Clusters[0].ClusterStatus",
                 "StringEquals": "available",
                 "Next": "Redshift Pass"
               }
             ],
             "Default": "Redshift Wait"
           },
           "Redshift Pass": {
             "Type": "Pass",
             "End": true
           },
           "Redshift Wait": {
             "Type": "Wait",
             "Seconds": 5,
             "Next": "DescribeClusters"
           }
         }
       },
       {
         "StartAt": "StartInstances",
         "States": {
           "StartInstances": {
             "Type": "Task",
             "Parameters": {
               "InstanceIds": [
                 "MyData"
               ]
             },
             "Resource": "arn:aws:states:::aws-sdk:ec2:startInstances",
             "Next": "DescribeInstanceStatus"
           },
           "DescribeInstanceStatus": {
             "Type": "Task",
             "Next": "Evaluate Instance Status",
             "Parameters": {
               "InstanceIds": [
                 "MyData"
               ]
             },
             "Resource": "arn:aws:states:::aws-sdk:ec2:describeInstanceStatus"
           },
           "Evaluate Instance Status": {
             "Type": "Choice",
             "Choices": [
               {
                 "And": [
                   {
                     "Variable": "$.InstanceStatuses[0].InstanceState.Name",
                     "StringEquals": "running"
                   },
                   {
                     "Variable": "$.InstanceStatuses[0].SystemStatus.Details[0].Status",
                     "StringEquals": "passed"
                   },
                   {
                     "Variable": "$.InstanceStatuses[0].InstanceStatus.Details[0].Status",
                     "StringEquals": "passed"
                   }
                 ],
                 "Next": "EC2 Pass"
               }
             ],
             "Default": "EC2 Wait"
           },
           "EC2 Pass": {
             "Type": "Pass",
             "End": true
           },
           "EC2 Wait": {
             "Type": "Wait",
             "Seconds": 5,
             "Next": "DescribeInstanceStatus"
           }
         }
       }
     ]

And this override:
(Stripped to just what's relevant)

Override
                    "Branches": [{
                        "StartAt": "ResumeCluster",
                        "States": {
                            "ResumeCluster": {
                                "Parameters": {
                                    "ClusterIdentifier": "CLUSTER_NAME"  # TODO Sub this out
                                }
                            },
                            "DescribeClusters": {
                                "Parameters": {
                                    "ClusterIdentifier": "CLUSTER_NAME"  # TODO Sub this out
                                }
                            },
                        }
                    }, {
                        "StartAt": "StartInstances",
                        "States": {
                            "StartInstances": {
                                "Parameters": {
                                    "InstanceIds": ["INSTANCE_ID"]  # TODO Sub this out
                                }
                            },
                            "DescribeInstanceStatus": {
                                "Parameters": {
                                    "InstanceIds": ["INSTANCE_ID"]  # TODO Sub this out
                                }
                            }
                        }
                    }]

I'm left with this JSON in the state language definition. You can see clearly that it's appended the overrides as additional members rather than overriding the existing ones:

Resulting ASL
"Branches": [
        {
          "StartAt": "ResumeCluster",
          "States": {
            "ResumeCluster": {
              "Type": "Task",
              "Parameters": {
                "ClusterIdentifier": "MyData"
              },
              "Resource": "arn:aws:states:::aws-sdk:redshift:resumeCluster",
              "Next": "DescribeClusters"
            },
            "DescribeClusters": {
              "Type": "Task",
              "Parameters": {
                "ClusterIdentifier": ""
              },
              "Resource": "arn:aws:states:::aws-sdk:redshift:describeClusters",
              "Next": "Evaluate Cluster Status"
            },
            "Evaluate Cluster Status": {
              "Type": "Choice",
              "Choices": [
                {
                  "Variable": "$.Clusters[0].ClusterStatus",
                  "StringEquals": "available",
                  "Next": "Redshift Pass"
                }
              ],
              "Default": "Redshift Wait"
            },
            "Redshift Pass": {
              "Type": "Pass",
              "End": true
            },
            "Redshift Wait": {
              "Type": "Wait",
              "Seconds": 5,
              "Next": "DescribeClusters"
            }
          }
        },
        {
          "StartAt": "StartInstances",
          "States": {
            "StartInstances": {
              "Type": "Task",
              "Parameters": {
                "InstanceIds": [
                  "MyData"
                ]
              },
              "Resource": "arn:aws:states:::aws-sdk:ec2:startInstances",
              "Next": "DescribeInstanceStatus"
            },
            "DescribeInstanceStatus": {
              "Type": "Task",
              "Next": "Evaluate Instance Status",
              "Parameters": {
                "InstanceIds": [
                  "MyData"
                ]
              },
              "Resource": "arn:aws:states:::aws-sdk:ec2:describeInstanceStatus"
            },
            "Evaluate Instance Status": {
              "Type": "Choice",
              "Choices": [
                {
                  "And": [
                    {
                      "Variable": "$.InstanceStatuses[0].InstanceState.Name",
                      "StringEquals": "running"
                    },
                    {
                      "Variable": "$.InstanceStatuses[0].SystemStatus.Details[0].Status",
                      "StringEquals": "passed"
                    },
                    {
                      "Variable": "$.InstanceStatuses[0].InstanceStatus.Details[0].Status",
                      "StringEquals": "passed"
                    }
                  ],
                  "Next": "EC2 Pass"
                }
              ],
              "Default": "EC2 Wait"
            },
            "EC2 Pass": {
              "Type": "Pass",
              "End": true
            },
            "EC2 Wait": {
              "Type": "Wait",
              "Seconds": 5,
              "Next": "DescribeInstanceStatus"
            }
          }
        },
        {
          "StartAt": "ResumeCluster",
          "States": {
            "ResumeCluster": {
              "Parameters": {
                "ClusterIdentifier": "CLUSTER_NAME"
              }
            },
            "DescribeClusters": {
              "Parameters": {
                "ClusterIdentifier": "CLUSTER_NAME"
              }
            }
          }
        },
        {
          "StartAt": "StartInstances",
          "States": {
            "StartInstances": {
              "Parameters": {
                "InstanceIds": [
                  "INSTANCE_ID"
                ]
              }
            },
            "DescribeInstanceStatus": {
              "Parameters": {
                "InstanceIds": [
                  "INSTANCE_ID"
                ]
              }
            }
          }
        }
      ]

The expected behavior would be for the construct to drill into the existing array members in-order (since I have no way to target an array element via JMES or anything like that) and perform the overrides.

Is yaml supported really needed?

What is the use case behind yaml support?

Asking because the template file exposed by the Step-Functions UI is JSON.

Because I am usually a bit picky about my dependencies and the used js-yaml library doesn't seem to be maintained any more (no update since 3 years).

So I personally would vote against yaml support, unless there is a specific use case.
If you want to support yaml it would be cool if you can replace js-yaml with the better maintained library called yaml

(didn't want to come across as rude or similar, just curious 😄 )

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.