Taking EC2 Volume daily backup using Lambda and scheduled using EventBridge Rule.
A Lambda function is scheduled to run every night at 1:00 AM to take backups of the EBS Volumes and retains them for 7 days. The lambda does the following:
1.Gets the EBS volumes with tag as BackupVolume:Yes 2.Deletes any snaphot which are older than 7 days. 3.Creates the snapshots with Name tag as date time in the format YY-mm-DD HH24:MM:SS and a tag with name as CreatedTimestamp:UnixTimestamp
mindmap
root((1 - AWS CloudFormation ))
2
AWS Lambda
3
AWS IAM
4
AWS CloudWatch
5
AWS Key Management Service
7
AWS EventBridge
- This repository is configured to deploy the stack in Development, Staging and Production AWS Accounts. To use the pipeline you need to have three AWS Accounts created in an AWS Org under a Management Account (which is the best practice). The Org structure will be as follows:
Root
├─ Management
├─ Development
├─ Test
└─ Production
-
Create KMS Key in each of the AWS Accounts which will be used to encrypt the resources.
-
Create an OpenID Connect Identity Provider
-
Create an IAM Role for OIDC and use the sample Trust Policy in each of the three AWS accounts
{
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::<Account Id>:oidc-provider/token.actions.githubusercontent.com"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringLike": {
"token.actions.githubusercontent.com:sub": [
"repo:<GitHub User>/<GitHub Repository>:",
]
}
}
}
]
}
- Create an IAM Policy to allow CloudFormation access and attach it to the OIDC Role, using the sample policy document:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowStackPermission",
"Effect": "Allow",
"Action": [
"cloudformation:CreateStack",
"cloudformation:DeleteStack",
"cloudformation:CreateChangeSet",
"cloudformation:DescribeChangeSet",
"cloudformation:ExecuteChangeSet",
"cloudformation:DeleteChangeSet",
"cloudformation:DescribeStacks",
"cloudformation:DescribeStackEvents"
],
"Resource": "*"
},
{
"Sid": "AllowKMSKeyAccess",
"Effect": "Allow",
"Action": "kms:CreateGrant",
"Resource": "arn:aws:kms:<AWS Region>:<AWS Account Id>:key/<KMS Key Id>"
}
]
}
- Create an IAM Policy to allow creation and deletion of resources and attach it to the OIDC Role, using the following sample policy document:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowS3Access",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:CreateBucket",
"s3:PutObjectTagging",
"s3:PutBucketPublicAccessBlock",
"s3:PutBucketAcl",
"s3:DeleteObject",
"s3:DeleteBucket",
"s3:ListBucket",
"s3:PutBucketTagging",
"s3:GetBucketTagging",
"s3:GetBucketPolicy",
"s3:GetBucketAcl",
"s3:GetObjectAcl",
"s3:GetObjectVersionAcl",
"s3:PutObjectAcl",
"s3:PutObjectVersionAcl",
"s3:GetBucketCORS",
"s3:PutBucketCORS",
"s3:GetBucketWebsite",
"s3:GetBucketVersioning",
"s3:GetAccelerateConfiguration",
"s3:GetBucketRequestPayment",
"s3:GetBucketLogging",
"s3:GetLifecycleConfiguration",
"s3:GetReplicationConfiguration",
"s3:GetEncryptionConfiguration",
"s3:GetBucketObjectLockConfiguration",
"s3:GetBucketOwnershipControls",
"s3:GetObjectTagging",
"s3:GetBucketNotification",
"s3:PutEncryptionConfiguration",
"s3:PutBucketOwnershipControls",
"s3:PutBucketNotification",
"s3:ListBucketVersions",
"s3:PutObjectTagging"
],
"Resource": "*"
},
{
"Sid": "IAMAccess",
"Effect": "Allow",
"Action": [
"iam:GetRole",
"iam:CreateRole",
"iam:DeleteRolePolicy",
"iam:PutRolePolicy",
"iam:DeleteRole",
"iam:PassRole",
"iam:TagRole",
"iam:CreatePolicy",
"iam:ListRolePolicies",
"iam:GetPolicy",
"iam:ListAttachedRolePolicies",
"iam:GetPolicyVersion",
"iam:ListInstanceProfilesForRole",
"iam:AttachRolePolicy",
"iam:DetachRolePolicy",
"iam:ListPolicyVersions",
"iam:DeletePolicy",
"iam:UntagRole",
"iam:GetRolePolicy"
],
"Resource": "*"
},
{
"Sid": "LambdaCreateAccess",
"Effect": "Allow",
"Action": [
"lambda:GetFunction",
"lambda:DeleteFunction",
"lambda:CreateFunction",
"lambda:TagResource",
"lambda:InvokeFunction",
"lambda:PutFunctionConcurrency",
"lambda:CreateFunction",
"lambda:ListVersionsByFunction",
"lambda:GetFunctionCodeSigningConfig",
"lambda:PutFunctionEventInvokeConfig",
"lambda:AddPermission",
"lambda:GetFunctionEventInvokeConfig",
"lambda:GetPolicy",
"lambda:DeleteFunctionEventInvokeConfig",
"lambda:RemovePermission",
"lambda:ListTags"
],
"Resource": "*"
},
{
"Sid": "SQSCreateAccess",
"Effect": "Allow",
"Action": [
"sqs:GetQueueAttributes",
"sqs:ListQueueTags",
"sqs:CreateQueue",
"sqs:TagQueue",
"sqs:SetQueueAttributes",
"sqs:SendMessage",
"sqs:DeleteQueue"
],
"Resource": "*"
},
{
"Sid": "SNSCreateAccess",
"Effect": "Allow",
"Action": [
"SNS:GetTopicAttributes",
"SNS:ListTagsForResource",
"SNS:GetSubscriptionAttributes",
"SNS:CreateTopic",
"SNS:TagResource",
"SNS:SetTopicAttributes",
"SNS:DeleteTopic",
"SNS:Subscribe",
"SNS:Unsubscribe"
],
"Resource": "*"
},
{
"Sid": "DynamoDBAccess",
"Effect": "Allow",
"Action": [
"dynamodb:DescribeTable",
"dynamodb:DescribeContinuousBackups",
"dynamodb:DescribeTimeToLive",
"dynamodb:ListTagsOfResource",
"dynamodb:CreateTable",
"dynamodb:TagResource",
"dynamodb:DeleteTable"
],
"Resource": "*"
},
{
"Sid": "KMSKeyAccess",
"Effect": "Allow",
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:DescribeKey"
],
"Resource": [
"arn:aws:kms:<AWS Region>:<AWS Account Id>:key/<KMS Key Id>"
]
},
{
"Sid": "CloudWatchAccess",
"Effect": "Allow",
"Action": [
"cloudwatch:PutMetricAlarm",
"cloudwatch:DescribeAlarms",
"cloudwatch:ListTagsForResource",
"cloudwatch:DeleteAlarms",
"cloudwatch:TagResource"
],
"Resource": "*"
},
{
"Sid": "StateMachineAccess",
"Effect": "Allow",
"Action": [
"states:CreateStateMachine",
"states:TagResource",
"states:DescribeStateMachine",
"states:DeleteStateMachine"
],
"Resource": "*"
},
{
"Sid": "CWLogGroupAccess",
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:TagResource",
"logs:PutRetentionPolicy",
"logs:DescribeLogGroups",
"logs:DeleteLogGroup"
],
"Resource": "*"
},
{
"Sid": "EventBridgeAccess",
"Effect": "Allow",
"Action": [
"events:TagResource",
"events:CreateEventBus",
"events:DeleteEventBus",
"events:PutRule",
"events:DescribeRule",
"events:PutTargets",
"events:RemoveTargets",
"events:DeleteRule"
],
"Resource": "*"
}
]
}
- Clone the repository.
- Create a S3 bucket to used a code repository.
- Modify the params/cfn-parameter.json with you Key Id
{
"stack-prefix": "ec2-vol-backup",
"stack-suffix": "stack",
"template-path": "/cft/ec2-volume-bkp-lambda-cft.yaml",
"project-name": "erketu",
"parameters": {
"devl": [
{
"Environment": "devl"
}
],
"test": [
{
"Environment": "test"
}
],
"prod": [
{
"Environment": "prod"
}
]
}
}
-
Create three repository environments in GitHub (devl, test, prod)
-
Create the following GitHub repository Secrets:
| Secret Name | Secret Value |
|---|---|
| AWS_REGION | us-east-1 |
| DEVL_AWS_KMS_KEY_ARN | arn:aws:kms:<AWS Region>:<Development Account Id>:key/<KMS Key Id in Development> |
| TEST_AWS_KMS_KEY_ARN | arn:aws:kms:<AWS Region>:<Test Account Id>:key/<KMS Key Id in Test> |
| PROD_AWS_KMS_KEY_ARN | arn:aws:kms:<AWS Region>:<Production Account Id>:key/<KMS Key Id in Production> |
| DEVL_AWS_ROLE_ARN | arn:aws:iam::<Development Account Id>:role/<OIDC IAM Role Name> |
| TEST_AWS_ROLE_ARN | arn:aws:iam::<Test Account Id>:role/<OIDC IAM Role Name> |
| PROD_AWS_ROLE_ARN | arn:aws:iam::<Production Account Id>:role/<OIDC IAM Role Name> |
| DEVL_CODE_REPOSITORY_S3_BUCKET | <Repository S3 Bucket in Development> |
| TEST_CODE_REPOSITORY_S3_BUCKET | <Repository S3 Bucket in Test> |
| PROD_CODE_REPOSITORY_S3_BUCKET | <Repository S3 Bucket in Production> |
- Create Create a feature branch and push the code.
- The CI/CD pipeline will create a build and then will deploy the stack to devlopment.
- Once the Stage and Prod deployment are approved (If you have configured with protection rule ) the stack will be reployed in the respective environments
📧 Subhamay Bhattacharyya - [[email protected]]
Contributors names and contact info
Subhamay Bhattacharyya - [[email protected]]
- 0.1
- Initial Release
This project is licensed under Subhamay Bhattacharyya. All Rights Reserved.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent