Git Product home page Git Product logo

org-formation-reference's Introduction

IMPORTANT NOTE

This reference architecture and also the steps outlined in this review are still under development and wont work as-is.

Please do

  • watch this project if you would like to get updates on the process.
  • contribute best practices / ideas / documentation / code through issues or PRs <3

thanks!

Introduction

This is a reference architecture to get started quickly with using org-formation as well as provide examples of best practices and demonstrates the capabilities. The reference architecture is built with the following core principles:

  • Batteries included: it should deploy with as little dependencies as possible
  • Always relevant: every organization will get value out of every part of the reference architecture. This means two things
    • it should not contain services that might not be used. Examples: AWS ECS, AWS Step Functions
    • it should not be too opinionated about choosing an AWS service as the implementation for a function that can also be sourced externally. Examples: AWS SecurityHub, AWS CloudWatch logs and metrics

To use this reference architecture, follow the Getting Started. If you want to deploy additional stacks using org-formation on top of this reference architecture, then it is advised to do that using delegated builds. Within the org-formation GitHub there will be example stacks configured as delegated builds that are usable, but are not eligeble to be part of the reference architecture because they either depend on an external system (not batteries included) or not always relevent (uses an AWS service that does something that is not always relevant). Examples are:

  • Break glass notifications based on events
  • DNS and Domain management

Getting Started

To create an AWS Organization based on this reference architecture, managed by org-formation follow these steps end to end.

  1. Create the AWS Management Account
  2. Create the AWS Organization
  3. Configure AWS SSO
  4. Clone and modify this repository
  5. Initialize org-formation
  6. Clone your new org-formation repository

Prerequisites

  1. A valid credit card
  2. A working phone number
  3. Four unique email addresses within your domain (management-root, compliance-root, log-archive-root and orgbuild-root)

Hint: if you are using Google as an email provider, you can use [email protected] to create unique addresses that all arrive in the same email box

1. Create the AWS Management Account

Create an AWS Account. This will be the management account of your AWS Organization

  1. Navigate here to sign up to an AWS account. Or sign-up on the page here
  2. Gather the following data that you will need in step Clone and modify this repository
Parameter Where to find it Example
{{management-account-id}} Top right -> your account -> My Account 341696816352

2. Create the AWS Organization

  1. Create an AWS Organization
  2. Enable all features
  3. Enable all policy types

3. Configure AWS SSO

You will use AWS SSO to obtain access to the AWS Accounts after we are done with the initial setup. IAM Users including the root user should not be used as a security best practice for human access.

  1. Enable AWS SSO
  2. Create 4 groups
    • Admin
    • Developer
    • Auditor
    • Supporter
  3. Create 1 user
    • Set a password
    • Enroll 2FA
    • Add user to the Administrator group
  4. Gather the following data you will need in steps Clone and modify this repository and Clone your new org-formation repository
Parameter Description Example
SSO start URL Go to AWS SSO -> Settings -> User Portal -> User Portal URL https://dgega332fa.awsapps.com/start
{{sso-instance-arn}} Go to AWS SSO -> Settings -> ARN arn:aws:sso:::instance/ssoins-6987b39db64e1ecd
{{sso-admin-group-id}} Go to AWS SSO -> Groups -> Administrator -> Details -> Group ID 99672ac0cf-8495fd69-c57e-4214-88a4-b9f41eed0d32
{{sso-auditor-group-id}} Go to AWS SSO -> Groups -> Auditor -> Details -> Group ID 99672ac0cf-8495fd69-c57e-4214-88a4-b9f41eed0d32
{{sso-developer-group-id}} Go to AWS SSO -> Groups -> Developer -> Details -> Group ID 99672ac0cf-8495fd69-c57e-4214-88a4-b9f41eed0d32
{{sso-supporter-group-id}} Go to AWS SSO -> Groups -> Supporter -> Details -> Group ID 99672ac0cf-8495fd69-c57e-4214-88a4-b9f41eed0d32

Configuring AWS SSO manually using AWS SSO itself as an identity provider is the quickest and easiest way to get started (and is batteries included). At the time of writing AWS SSO does not support automating this via an API, as soon as it does, this guide will be updated accordingly. If you want to configure an external Identity Provider, then start here.

4. Clone and modify this repository

  1. Clone this repository locally
  2. Search and replace the following values globally in all files
Parameter Description Source Example
{{management-account-id}} 12 digit identifier of the management account Create the AWS Management Account 341696816352
{{state-bucket-name}} S3 bucket where the IaC state will be stored choose organization-formation-state-341696816352-prd
{{organization-name}} Alias of the management account choose bee-awesome
{{primary-aws-region}} The primary AWS region to deploy to choose us-east-1
{{management-root-email-address}} Email address used to register the management account Create the AWS Management Account [email protected]
{{security-root-email-address}} Email address for the security account Prerequisites [email protected]
{{log-archive-root-email-address}} Email address for the log-archive account Prerequisites [email protected]
{{budget-alarm-email-address}} Email address to receive budget alarms Prerequisites [email protected]
{{orgbuild-root-email-address}} Email address for the org build account Prerequisites [email protected]
{{sso-instance-arn}} AWS SSO instance ARN Configure AWS SSO arn:aws:sso:::instance/ssoins-6987b39db64e1ecd
{{sso-admin-group-id}} Principal ID from Identity Provider's group used by administrators Configure AWS SSO 99672ac0cf-8495fd69-c57e-4214-88a4-b9f41eed0d32
{{sso-auditor-group-id}} Principal ID from Identity Provider's group used by auditors Configure AWS SSO 99672ac0cf-8495fd69-c57e-4214-88a4-b9f41eed0d32
{{sso-developer-group-id}} Principal ID from Identity Provider's group used by auditors Configure AWS SSO 99672ac0cf-8495fd69-c57e-4214-88a4-b9f41eed0d32
{{sso-supporter-group-id}} Principal ID from Identity Provider's group used by supporters Configure AWS SSO 99672ac0cf-8495fd69-c57e-4214-88a4-b9f41eed0d32

5. Initialize org-formation

In this step, you run OrgFormation locally using the credentials of the root user of the management account. This is the only time you will use the root user of any account for any purpose and the root user will be closed off as the last step in this guide.

Note: you might need to run the OrgFormation commands more than once.


  1. Install OrgFormation on your local machine
npm install -g aws-organization-formation
I already have an existing AWS Organization

If you already have an existing organization, then the following command generates an organization.yml file for you based on the current structure of your organization.

npx org-formation init organization.yml --region _Primary Region_ --cross-account-role-name OrganizationFormationBuildAccessRole --print-stack --verbose

You can then merge that with the ./src/organization.yml file in the way you like and continue with this guide.


  1. Apply the organization.yml file to AWS Organizations. This means creating accounts and OUs and ordering them. This will also create the OrgBuild account where the org-formation build pipeline will reside.
npx org-formation update ./src/organization.yml --verbose
  1. Create the role that the org-formation uses inside of the Management Account
aws cloudformation create-stack --stack-name org-formation-role --template-body file://src/templates/000-org-build/role.yml --capabilities CAPABILITY_NAMED_IAM
  1. Zip this local repository into 000-org-build to be used as the initial commit for the OrgBuild CodeCommit repository. From the top level of this repository, execute:
zip ./src/templates/000-org-build/initial-commit src/**/*.* .gitignore .org-formationrc *.yml *.json README.md
  1. Deploy the stacks in 000-org-build. This creates the build roles, state bucket and file and the OrgFormation build pipeline in the OrgBuild account. It also uploads this entire local repository as an initial commit to the Git repository that the build pipeline will then execute.
npx org-formation perform-tasks ./src/templates/000-org-build/_tasks.yml --organization-file ./src/organization.yml --max-concurrent-stacks 50 --max-concurrent-tasks 1 --print-stack --verbose

When you have finished the setup, we will need to delete the S3 bucket containing the state in the management account. This was created in step 2 because at that time the OrgBuild account, was assumed to not exist yet.

6. Clone your new org-formation repository

Here you will configure command line access to the CodeCommit repository that will trigger the Build Pipeline in the OrgBuild Account. It requires you to configure the aws cli to use SSO and the clone the repository using HTTPS git-remote-codecommit (GRC)

  1. Configure AWS CLI with AWS SSO by following this guide

You will need to provide the following details:

Parameter Description Example
SSO start URL The landing page URL to be found in the AWS SSO of the management account https://dgega332fa.awsapps.com/start
SSO region The default region us-east-1
SSO account id Select the OrgBuild account from the drop-down 222140350420
SSO role name Select a role with write permission the drop-down Administrator
CLI default client Region The default region us-east-1
CLI default output format Whatever format you prefer yaml
CLI profile name Name of the profile, choose wisely ba-orgbuild-admin
  1. Configure support for CodeCommit git-remote-codecommit

  2. Using your IDP (either AWS SSO itself or an external IDP), log into the OrgBuild account, navigate to AWS CodeCommit, find the repository URL and clone

It looks something like this

git clone codecommit::eu-central-1://<AWS_CLI_PROFILE_NAME>@organization-formation
  1. You can now make changes, commit and push and that will be effectuated by the build pipeline in the OrgBuild Account

org-formation-reference's People

Contributors

apenney avatar boomfish avatar dbaynard avatar eduardomourar avatar keithduncan avatar leantorres73 avatar olafconijn avatar rene84 avatar skwashd avatar sshvetsov avatar stefan-karlsson avatar zaro0508 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

org-formation-reference's Issues

Initialize org-formation with root user

Step 5 of the README says that:

In this step, you run OrgFormation locally using the credentials of the root user of the management account

however running the update command gives me this result:

> org-formation update ./src/organization.yml --verbose "--profile" "dangerous"

WARN: Hi there!
WARN: You just ran into an error when assuming the role OrganizationFormationBuildAccessRole in account x.
WARN: Possibly, this is due a breaking change in org-formation v0.9.15.
WARN: From v0.9.15 onwards the org-formation cli will assume a role in every account it deploys tasks to.
WARN: This will make permission management and SCPs to deny / allow org-formation tasks easier.
WARN: More information: https://github.com/org-formation/org-formation-cli/tree/master/docs/0.9.15-permission-change.md
WARN: Thanks!
WARN: ======================================
ERROR: error: AccessDenied, aws-request-id: x
ERROR: Roles may not be assumed by root accounts.

(I think the warning is irrelevant in this case.)

I believe I could workaround this by creating an IAM account and specifying it as one of the assumeRolePrincipals for the next step:

aws cloudformation create-stack --stack-name org-formation-role --template-body file://src/templates/000-org-build/role.yml

Is this what I should be doing? Or have I likely got something else wrong?

050-event-routing: name change?

currently 050 is called event-routing. the contents is a sample on further automating the account creation process. i wouldn't look for that example within this folder.

should we rename to 050-account-creation?

Reference architecture should be deployable over SAR

would be most convenient to deploy the reference architecture using the Serverless Application Repository (from within the management account).

Limitations in CloudFormation would prevent you from executing code (cant do: org-formation init)... but if you do something like set up a codecommit/codebuild/codepipeline, point it at an 'initial-commit.zip' it will run automatically.

this is a bit of a hack but imho really worth the while.

implementing this reference architecture would then be as easy as:

  1. create account
  2. deploy using SAR
  3. done

Make this template support jinja templating

in order to

  1. automate the installation of this reference architecture
  2. integation with the org-formation-cli tool

I would be beneficial for this repo to support a formal templating language (e.g. Jinja).
The org-formation-cli can then use the repo-name as an option when performing init-pipeline in a similar way sls uses a --template-url (in our case maybe --template-repo?).

the replacement keys would need to be known in both the cli and this repo.

Include a delegated build pipeline in the same account

It's good practice to separate build pipelines and repositories to small units, while limiting the amount of dependencies. This enables different groups of ppl (teams) to work more independently of each other.

The prescribed way of org-formation of doing this is to use delegated builds. Might as well add a delegated build (repo and pipeline) into the reference architecture so users of the reference architecture have a starting point and example of how to set up a delegated build.

This could initially be done within the same OrgBuild account, with documentation of how to set it up in other accounts and guidance on whether or not to creating delegated builds within delegated builds is a good idea

Unable to init from cli

I've followed the instructions until https://github.com/org-formation/org-formation-reference#5-initialize-org-formation and first it ran into a Error: Profile default not found error. I assumed I needed to put the root crendentials in, but that doesn't work:

(ins)hendry-tw-mbp~/orgtest/org-formation-reference$ npx org-formation update ./src/organization.yml --verbose
WARN: ======================================
WARN: Hi there!
WARN: You just ran into an error when assuming the role OrganizationFormationBuildAccessRole in account 381831929214.
WARN: Possibly, this is due a breaking change in org-formation v0.9.15.
WARN: From v0.9.15 onwards the org-formation cli will assume a role in every account it deploys tasks to.
WARN: This will make permission management and SCPs to deny / allow org-formation tasks easier.
WARN: More information: https://github.com/org-formation/org-formation-cli/tree/master/docs/0.9.15-permission-change.md
WARN: Thanks!
WARN: ======================================
ERROR: error: AccessDenied, aws-request-id: 018ac5f7-258a-4f88-aa23-76145560f36b
ERROR: Roles may not be assumed by root accounts.

And then I created a user with Administrator access rights and that also did not work:

(ins)hendry-tw-mbp~/orgtest/org-formation-reference$ vim .env 
(ins)hendry-tw-mbp~/orgtest/org-formation-reference$ source .env 
(ins)hendry-tw-mbp~/orgtest/org-formation-reference$ npx org-formation update ./src/organization.yml --verbose
WARN: ======================================
WARN: Hi there!
WARN: You just ran into an error when assuming the role OrganizationFormationBuildAccessRole in account 381831929214.
WARN: Possibly, this is due a breaking change in org-formation v0.9.15.
WARN: From v0.9.15 onwards the org-formation cli will assume a role in every account it deploys tasks to.
WARN: This will make permission management and SCPs to deny / allow org-formation tasks easier.
WARN: More information: https://github.com/org-formation/org-formation-cli/tree/master/docs/0.9.15-permission-change.md
WARN: Thanks!
WARN: ======================================
ERROR: error: AccessDenied, aws-request-id: 90d5db12-a048-4ffe-a999-73ad3c398472
ERROR: User: arn:aws:iam::381831929214:user/admintest is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::381831929214:role/OrganizationFormationBuildAccessRole
(ins)hendry-tw-mbp~/orgtest/org-formation-reference$ 

What am I missing please?

Can't create deny-unsupported-regions SCP

Hi,

I'm hoping someone can help me. I'm running into an issue where I can't create the SCP to block regions, when it runs it looks like:

INFO: Executing: update-stacks templates/010-scps/deny-unsupported-regions.yml monad-deny-unsupported-regions.{"result":{"state":"FAILURE","reason":{"$metadata":{"httpStatusCode":200,"requestId":"1abcd8fa-5dd8-4340-8106-1ea449e194f6","attempts":
1,"totalRetryDelay":0},"Stacks":[{"StackId":"arn:aws:cloudformation:us-east-1:637423365128:stack/monad-deny-unsupported-regions/8060052
0-d0f7-11ee-9b0b-0e4b40acefbf","StackName":"monad-deny-unsupported-regions","Parameters":[{"ParameterKey":"targetIds","ParameterValue":
"r-yjlu"},{"ParameterKey":"supportedRegions","ParameterValue":"us-west-1,us-west-2,us-east-1,us-east-2"}],"CreationTime":"2024-02-21T20
:26:32.467Z","DeletionTime":"2024-02-21T20:26:37.173Z","RollbackConfiguration":{},"StackStatus":"ROLLBACK_COMPLETE","DisableRollback":f
alse,"NotificationARNs":[],"Capabilities":["CAPABILITY_NAMED_IAM","CAPABILITY_IAM","CAPABILITY_AUTO_EXPAND"],"Tags":[],"EnableTerminati
onProtection":false,"DriftInformation":{"StackDriftStatus":"NOT_CHECKED"}}]}}} (637423365128 = ManagementAccount)
ERROR: Resource Scp failed because Internal Failure.

I see the same thing in the console, with a failure and:

The following resource(s) failed to create: [Scp]. Rollback requested by user.

I tried looking through CloudTrail but I see nothing useful in there. I then tried making an SCP by hand (well, the policy) with the same regions just to make sure there wasn't anything obvious. That worked.

The params for this template look like:

supportedRegions: us-west-1,us-west-2,us-east-1,us-east-2
targetIds: r-yjlu

My manually created SCP was just the policy part, I didn't try to attach it to the target. Not sure if that's part of the issue. I'm not really sure what to troubleshoot next, I'm feeling a bit lost on this one!

080-aws-config-inventory: decide on ConfigTopic

currently 080-aws-config-inventory contains a ConfigTopic. if i understand correctly, this is a Topic that will be deployed to all accounts and can be used to subscribe Config Findings (or is it inventory delivery??).

either case:

  • if config findings -> this should be in the security account (not all accounts) and not be part of the baseline reference.
  • if inventory delivery -> i would leave it out of the reference arch.

consistent binding names

i noticed that 060-cloudtrails has a binding for the S3 bucket called StorageBinding. 070-guard-duty calls this an ArchiveBinding. should this be harmonized?

020-secure-defaults: include NoDefaultVpc resource

I would like to suggest to add the NoDefaultVpc resource to the secure default.
The reason should probably be explained in the readme: the fewer resources you have in your AWS the less you have to worry about from a security perspective.

Only possible downside: it might break the build if the default vpc is in use. should we then add failuretolerance for this task?

behavior:

  • new accounts -> default vpc gets removed from all regions
  • existing accounts -> default vpc gets removed if not in use. when in use build logs errors and continues

000-org-build: have OrganizationFormationInitialCommit download initial-commit over http

would be great to have the OrganizationFormationInitialCommit task download the initial commit from a central repo. this will require an issue to be made over at org-formation-cli to allow for https:// or s3:// sources?

consider to change attributes to SourcePath/TargetPath?

OrganizationFormationInitialCommit:
  Type: copy-to-s3
  DependsOn: OrganizationFormationBuckets
  LocalPath: 'initial-commit.zip'
  RemotePath: !Sub 's3://organization-formation-deployments-${accountId}-prd/initial-commit.zip'
  OrganizationBinding:
    Account: !Ref accountId
    Region: !Ref primaryRegion

020-secure-defaults: Error validating schemaHandlerPackage

When doing the integration of the secure defaults, I am getting the following error message on the register types I am integrating:

ERROR: Workload EbsEncryptionDefaultsRP in 123456789021/us-east-2 updated failed. reason: Error validating schemaHandlerPackage. Check the permissions on the bucket and object in your account and try again. (123456789021 = LogArchiveAccount)

This happens with the types that I am integrating, which are:

  • EbsEncryptionDefaultsRP
  • S3PublicAccessBlockRP
  • NoDefaultVpcRP

I am doing the integration according to the reference. Additionally I was checking the providers (which seems to be a bit outdated) and I don't see anything different.

Am I missing something?

SCP template: required key [Content] not found , extraneous key [PolicyDocument] is not permitted.

The template here

Does not appear to match up with the use of the Community::Organizations::Policy

The following error will be the result

ERROR: Resource Scp failed because Properties validation failed for resource Scp with message:
#: required key [Content] not found
#: extraneous key [PolicyDocument] is not permitted.

000-org-build: should OrgPipelineRole be toned down?

OrgPipelineRole should be able to do with less permissions.

The permissions assigned are the default permissions for a CodePipeline as created by AWS (IIRC).
Could be a valid decision to leave as is so that other common pipeline stages can be added.
If so: add a link to this issue as a means to document?

Had to complete steps out of order

Did not see a discussion board, but I needed to complete step 3, before I could complete step 2. The reason for this is that you can not assume a role before the role is created.

Error when configuring SSO

Hi there.

I am trying to follow the step by step instructions in the Readme, but unfortunately, I get an error when trying to configure the groups for SSO:

Screenshot 2021-02-03 at 09 53 02

Any ideas?

Split ComplianceAccount to SecurityAccount and LogArchiveAccount

Currenlty we have a ComplianceAccount in the repo.
this account will, however, host two different types of resources:

  • log archival for Config, CloudTrail, etc.
  • the delegated administration controls for GuardDuty, SecurityHub.

I would propose to split this in two:

  • LogArchiveAccount for log archival. Access would be very incidental, any access to this account could break glass.
  • SecurityAccount for it-sec and compliance tools. Access would be part of a security officers day-to-day routine

add transit gateway to reference project

I think setting up and configuration of the AWS transit gateway is similar to AWS cloudtrail, guardduty and config. It's one of those resources that requires lots of cross account orchestration. It would be super helpful if this reference project contained an example of setting up the transit gateway in a multi account configuration.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.