Next steps for the stepfunction event logger, in decreasing order of priority (and complexity):

• Add CDK code here, to spin up necessary Postgres infrastructure if the user selects the POSTGRES datastore options
• Add python code here to format and send stepfunction history items to Postgres datastore
• Add unit tests + integration tests for the lambda function
• Implement user-configurable DynamoDB indexes and capacity scaling (to allow user finer grain control over the most cost intensive aspects of the seed)

npm run docgen generates per package docs. It would be nice to somehow automatically combine these docs into a website for cdk-seed.

It would be nice if there was a template for creating new packages, and even better, a template that would prepopulate the package with the basic needs

We should add basic TS/JS code linting and formatting to the project

npm run package builds distribution artifacts for NPM and PyPI, so we should add some scripting that publishes those build artifacts.

Open questions:

• What account would be used for publishing to each package manager?
• When/how would publishing happen?
  • Seems like having it run as part of CI would be preferable
  • Only publish if the version number has changed?

CI builds should:

• Run tests
• ?

TL;DR:

Do you constantly find yourself running cdk destroy and finding that you're left with 100s of LogGroups orphaned? Fed up of polluting your CloudWatch with the LogGroups of Lambdas long dead? I may have the solution for you!

Create your LogGroups named as they would be by CloudWatch! This will override the out-of-the-box ones and you can set retention & removal_policy!

An Example

I've found in the past that creating and destroying multiple stacks in a day, for multiple days, for multiple weeks results in 100s if not 1000s of orphaned LogGroups. This is annoying as it's noisy and you spend longer looking for the one LogGroup you need.

After some discussions with @JakeHendy, it was noted that creating a LogGroup explicitly, named the same as it would be implicitly, would allow us to:

• Set a custom retention policy for logs
• Set a custom removal policy for LogGroups

You can see an example below:

from aws_cdk import (
    aws_lambda,
    aws_lambda_python,
    aws_logs,
    core,
)

lambda_function = aws_lambda_python.PythonFunction(
    self,
    entry="function_entry",
    handler="handler",
    runtime=aws_lambda.Runtime.PYTHON_3_8,
    memory_size=128,
    timeout=core.Duration.minutes(5),
)

aws_logs.LogGroup(
    self,
    log_group_name=f"/aws/lambda/{lambda_function.function_name}",
    removal_policy=core.RemovalPolicy.DESTROY,
    retention=aws_logs.RetentionDays.ONE_DAY,
)

This will then result in you being able to do cdk deploy and cdk destroy and no LogGroups being available under /aws/lambda/your_function_name* 🎉

A Gotcha

One issue I am still currently struggling with (and am losing hope on) is finding an easy way to not get orphaned LogGroups for CustomResource Lambda functions.

I have implemented the same pattern for explicit LogGroup creation for a Lambda-backed CustomResource and I always end up with an orphaned LogGroup... taunting me. 🤨 👎

I believe that this is due to how CloudFormation handles the deletion of the CustomResource, it probably looks like:

• cdk destroy
• Explicit LogGroup destroyed
• CustomResource invoked with destroy
  • Lambda invoked, which creates a new (out of our domain) LogGroup
• CustomResource destroyed
• Lambda destroyed
• cdk destroy ends
• Orphaned LogGroup taunts me 🧟‍♂️

I've posted in https://cdk.dev/ and as of yet, have no real luck. I've tried bouncing around with dependencies similar to how @alukach is doing it in the ss-orders project (Making a Construct explicitly depend on another) but that's not been too successful 😭

Jake suggested creating a Lambda that listens for DeleteStack in CloudTrail then deletes all LogGroups, but that's a Lambda for the sake of handling potentially one orphaned LogGroup, so in terms of effort vs. return, it's not a winner.

Happy for folks to jump in and give suggestions on how we could cover this gotcha off!

TL;DR:

Failures that occur outside of the handler of a Custom Resource result in long periods of inactivity when invoking CDK commands, they're also never raised as actual failures. This is a PITA.

So recently whilst working on HLS I was making some 🎸 Custom Resources 🤘

I'd wrapped my logic in beautiful try/except blocks and I'd handled the CFN Callbacks so that my Custom Resource called back to the mothership 👽 🛸 to tell CFN what was happening. This is used by Custom Resources to tell CloudFormation (and CDK) whether a resources creation/update/delete has been successful or not.

BUT

When I ran cdk deploy, my deployment was seemingly stuck on creating the Custom Resource forever. Upon further inspection, I could see in the logs of the handler that it was erroring out as soon as it was invoked - strange, this should be caught and Cloud Formation should be informed of the failure and begin the rollback logic.

So, I've got a Stack stuck deploying, my first thought? Delete the thing. So I deleted the stack... and it got stuck deleting the Custom Resource forever 🙃 .

u wot 🤨

So, this was confusing at first but then I took a look at the error messages in CloudWatch. Let's say I had a index.py like:

import cfnresponse
import my_cool_module

def handler(event, context):
    try:
        my_cool_module.do_something()
        cfnresponse.send_success() # This isn't real but you get the idea
    except my_cool_module.a_not_so_cool_exception as ex:
        print(ex)
        cfnresponse.send_failure(ex)

My importing of my_cool_module was erroring, not anything in my handler function. Because of this, I was never reaching any of my callback code, which meant that as far as CDK/CloudFormation were concerned, my Custom Resource was doing its thing and it'd hear from it eventually.

Because these callbacks are required for any CDK action, they'd result in infinitely (1+ hours) running deploys/updates/destroys which really wastes time.

You might ask, did you not test your code locally @ciaranevans?! - Well, I did. It worked beautifully because of how it was interpreting the import statement... not so correct when actually on its own in a Lambda 😭

So what should we do?

I suppose the easiest and grossest way could be:

import cfnresponse
try:
    import my_cool_module
except:
    cfnresponse.send_failure()

def handler(event, context):
    try:
        my_cool_module.do_something()
        cfnresponse.send_success() # This isn't real but you get the idea
    except my_cool_module.a_not_so_cool_exception as ex:
        print(ex)
        cfnresponse.send_failure(ex)

I don't like try/catch or conditional imports. So if someone has a better idea or knows of how we could gracefully handle this kind of issue, I'm all ears!

I imagine most languages will suffer this kind of situation, or at least have it a situation that's possible - is there a way for CDK to treat any failure that's not explicitly handled as a failure for CloudFormation? 🤷

• Mind dump over.

cc. @developmentseed/earthdata-infrastructure

A construct to apply a Permission Boundary, as described in aws/aws-cdk#3242. We've been using the following in projects that need deployment to GCC:

from typing import Union

import jsii
from aws_cdk import aws_iam, core
from jsii._reference_map import _refs
from jsii._utils import Singleton


@jsii.implements(core.IAspect)
class PermissionBoundaryAspect:
    """
    This aspect finds all aws_iam.Role objects in a node (ie. CDK stack) and
    sets permission boundary to the given ARN.
    https://github.com/aws/aws-cdk/issues/3242#issuecomment-553815373
    """

    def __init__(self, permission_boundary: Union[aws_iam.ManagedPolicy, str]) -> None:
        """
        :param permission_boundary: Either aws_iam.ManagedPolicy object or
        managed policy's ARN string
        """
        self.permission_boundary = permission_boundary

    def visit(self, construct_ref: core.IConstruct) -> None:
        """
        construct_ref only contains a string reference to an object. To get the
        actual object, we need to resolve it using JSII mapping.
        :param construct_ref: ObjRef object with string reference to the actual object.
        :return: None
        """
        if isinstance(construct_ref, jsii._kernel.ObjRef) and hasattr(
            construct_ref, "ref"
        ):
            kernel = Singleton._instances[
                jsii._kernel.Kernel
            ]  # The same object is available as: jsii.kernel
            resolve = _refs.resolve(kernel, construct_ref)
        else:
            resolve = construct_ref

        def _walk(obj):
            if isinstance(obj, aws_iam.Role):
                cfn_role = obj.node.find_child("Resource")
                policy_arn = (
                    self.permission_boundary
                    if isinstance(self.permission_boundary, str)
                    else self.permission_boundary.managed_policy_arn
                )
                cfn_role.add_property_override("PermissionsBoundary", policy_arn)
            else:
                if hasattr(obj, "permissions_node"):
                    for c in obj.permissions_node.children:
                        _walk(c)
                if hasattr(obj, "node") and obj.node.children:
                    for c in obj.node.children:
                        _walk(c)

        _walk(resolve)