Deploy GO application to EKS using Terraform and Gitlab
Deploy app & infra from local system.
Pre-requiste:
- AWS Account
- Create IAM user with administrator access policy and download its access key and secret key.
- Docker, Kubectl, Terraform & Terragrunt installed on local system.
- Install
base-iam-authenticator
on your local system
Setup Infrastructure:
- Setup AWS CLI on local system with access key and secret key
- Go to
infrastructure
directory and configureprofile
andregion
local variables interragrunt.hcl
file. - Initialize terragrunt setup with command
terragrunt init
(This initiates terraform S3 & dynamo db backend creation) - Once
terragrunt init
command is executed without any error then runterragrunt plan
- Finally run
terragrunt apply
command to deploy infrastructure. (This will create vpc, security groups and eks cluster in AWS account)
Once the infra is ready you can proceed to creating and deploying application.
First connect to EKS cluster using command
aws eks --region us-west-2 update-kubeconfig --name test-cluster-QVFPpWtx --profile default-borngreat-aws-account
then follow below steps to build and deploy application.
Setup/Deploy Go App:
- Build Docker image and push it to ECR repo (ECR repo link can be found in outputs of terragrunt)
aws ecr get-login-password --profile default-borngreat-aws-account | docker login --username AWS --password-stdin 944365771232.dkr.ecr.us-west-2.amazonaws.com/go-app-repo
cd docker-gs-ping
docker build -t 944365771232.dkr.ecr.us-west-2.amazonaws.com/go-app-repo:latest .
docker push 944365771232.dkr.ecr.us-west-2.amazonaws.com/go-app-repo:latest
- Deploy application and service using helm
cd helm-charts
helm install sample-go-app sample-go-app
Note: For first time helm project you can create sample project/files using command helm create sample-go-app
.
Create Gitlab pipeline
- Create gitlab repository
- Setup CI/CD vars in gitlab account
- Go to gitlab repository > Settings > CI/CD > Variables
- Setup variables for AWS Access
AWS_ACCESS_KEY_ID
,AWS_SECRET_ACCESS_KEY
andAWS_DEFAULT_REGION
. - Setup ECR Repo URL variables
ECR_REPOSITORY
- Setup variables for AWS Access
- Update
CLUSTER_NAME
variable in.gitlab-ci.yml
file on local system. - Upload/push all files and folder to Gitlab along with
.gitlab-ci.yml
file to the gitlab repository created this will automatically create a pipeline to build and deploy application to eks cluster.
Cleanup everything:
- First clean up all Kubernetes deployments/services
- go to
helm-charts
folder and runhelm uninstall sample-go-app
- Cleanup infrastrucutre
- Once deployments and services are cleanup then delete infrastructure using following commands:
terragrunt init
terragrunt destroy
- CI/CD pipeline can be pepared for both infra and application where infra will have a ci/cd pipeline with an upstream pipeline which handles deployment of latest application image
- CI/CD pipeline for infra aswell as application couple together to give a single interface and overview of whole release pipeline.
- MR based CI/CD pipeline triggers on Gitlab
- Role based authentication for terragrunt & CI/CD pipelines
- Kubernetes dashboard or any monitoring solution for resource optimization
- Multi AZ EKS setup - highly available across a given region
- Clean and unified overview of whole CI/CD pipeline on Gitlab
- Auto-scalable & Auto rollback deployments
- Infrastructure state maintained via Terraform remote state on S3 & Dynamo DB
- Application Docker image is stored on ECR
- Docker in Docker based CI/CD pipelines for lightweight and quick builds aswell as deployments.
- High maintainence - one has to always update and upgrade terrafrom and terragrunt versions
- Solution tightly coupled with AWS and Gitlab (Prone to downtimes based on provider status)
- High costs (you pay for Gitlab services and AWS services)
Incase you face error: You must be logged in to the server (Unauthorized)
This happens because the user you are using to connect to the cluster is not whitelisted in configmaps to allow access. Please add the IAM users to kubernet config map using below commands.
- See config map using:
kubectl -n kube-system get configmap aws-auth -o yaml
- Edit config map using:
kubectl -n kube-system edit configmap aws-auth
Add following section after mapRoles
mapUsers: |
- userarn: arn:aws:iam::XXXXXX:user/test_user
username: test_user
groups:
- system:masters
This will let external IAM users or roles access cluster from anywhere.