The tetris_project_doc from karthi770

SUMMARY:

The aim of this project is to set a CICD pipeline that uses GitOps for the continuous delivery.

Jenkins Server

In this project we shall first start with the terraform codes which creates the roles and the EC2 instance:

# provider.tf
terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~>5.0"
    }
  }
}
provider "aws" {
  region = "us-east-1"
}

# main.tf
resource "aws_iam_role" "jenkins_role" {
  name = "Jenkins-terraform"
  # Terraform's "jsonencode" function converts a
  # Terraform expression result to valid JSON syntax.
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Sid    = ""
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      },
    ]
  })
}

resource "aws_iam_role_policy_attachment" "test-attach" {
  role       = aws_iam_role.jenkins_role.name
  policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess"
}

resource "aws_iam_instance_profile" "jenkins_profile" {
  name = "Jenkins-terraform"
  role = aws_iam_role.jenkins_role.name
}

resource "aws_security_group" "jenkins-sg" {
  name        = "Jenkins-security-group"
  description = "Open 22,80,443,8080,9000,3000"
  vpc_id = "vpc-047714d71a6ec55f4"
  # ingress rule to allow all the ports to open, we use for loop here

  ingress = [
    for port in [22,80,443,8080,9000,3000] : {
        description       = "TLS from VPC"
        from_port         = port
        to_port           = port
        protocol          = "tcp"
        cidr_blocks       = ["0.0.0.0/0"]
        ipv6_cidr_blocks  = []
        self = false
        prefix_list_ids = []
        security_groups= []
    }
  ]

  egress {
        from_port = 0 //opening all ports
        to_port  = 0
        protocol = "-1"
        cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_instance" "web" {
  ami           = "ami-0c101f26f147fa7fd"
  instance_type = "t2.large"
  key_name = "jenkins"
  vpc_security_group_ids = [aws_security_group.jenkins-sg.id]
  subnet_id = "subnet-0c492c119d2652991"
  user_data = templatefile("./install_jenkins.sh", {})
  iam_instance_profile = aws_iam_instance_profile.jenkins_profile.name

  tags = {
    Name = "Jenkins-argo"
  }

  root_block_device {
    volume_size = 30
  }
}

The install.sh file will have all the shell script commands to install jenkins, sonar, docker, trivy, terraform, kubectl and aws cli.

#!/bin/bash
sudo apt update -y
wget -O - https://packages.adoptium.net/artifactory/api/gpg/key/public | tee /etc/apt/keyrings/adoptium.asc
echo "deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | tee /etc/apt/sources.list.d/adoptium.list
sudo apt update -y
sudo apt install temurin-17-jdk -y
/usr/bin/java --version
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee /usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update -y
sudo apt-get install jenkins -y
sudo systemctl start jenkins
sudo systemctl status jenkins

#install docker
sudo apt-get update
sudo apt-get install docker.io -y
sudo usermod -aG docker ubuntu  
newgrp docker
sudo chmod 777 /var/run/docker.sock
docker run -d --name sonar -p 9000:9000 sonarqube:lts-community

# install trivy
sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y

#install terraform
sudo apt install wget -y
wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform

#install Kubectl on Jenkins
sudo apt update
sudo apt install curl -y
curl -LO https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client

#install Aws cli 
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo apt-get install unzip -y
unzip awscliv2.zip
sudo ./aws/install

#backend.tf
terraform {
  backend "s3" {
    bucket = "tetris-terraform-bucket"
    key = "jenkins-terraform.tfstate"
    region = "us-east-1"
  }
}

We need to create a bucket named "tetris-terraform-bucket" to store the statefile. The S3 bucket should be in place before executing this code.

The above codes created the EC2 instance, installed all the dependencies and the security group.

EC2 instance.

Security Group

S3 Backend

This command will reveal the password for the jenkins and this should be passed in the ec2 terminal. sudo cat /var/lib/jenkins/secrets/initialAdminPassword Both Sonar qube and Jenkins can run as admin. Sonar password - karthi770 Both sonar and Jenkins are UP and running.

Setting up Jenkins:

In Jenkins go to → Dashboard → Manage Jenkins → Plugins → Available plugins Install → Terraform. Now go to → Tools → we can see the terraform tab → Inside the install directory paste the path of terraform directory from the EC2 instance which terraform this command will give the path. Usually its /usr/bin/terraform but we have give only /usr/bin/

Building Pipeline:

pipeline {
    agent any

    stages {
        stage('Checkout') {
            steps {
                git branch: 'main', url: 'https://github.com/karthi770/Tertris_Game.git'
            }
        }
        stage('terraform init') {
            steps {
                dir('Eks-terraform'){
                    sh 'terraform init'
                }
            }
        }
        stage('terraform validate') {
            steps {
                dir('Eks-terraform'){
                    sh 'terraform validate'
                }
            }
        }
        stage('terraform plan') {
            steps {
                dir('Eks-terraform'){
                    sh 'terraform plan'
                }
            }
        }
        stage('terraform Apply/destroy') {
            steps {
                dir('Eks-terraform'){
                    sh 'terraform ${action} --auto-approve'
                }
            }
        }
    }
}

Now the option is changed to Build with Parameters, we have a choice to select the action.

Install the following plugins: > Temurin (for java),
> Sonarqube, > Node.js, > Owasp, > Docker

All the above tools must be configured:

Sonar qube will be up and running when we first start the EC2, once we stop and start the EC2, sonar is unreachable, since it is running as a docker image. Use these command to re-start the docker. docker ps -a this will show all the containers the is even stopped. docker start [container id] this will again start sonar qube.

Jenkins must access Sonar for code review, so a token is generated to give jenkins the access to sonar qube.
Give an id or name and save the token. Now we need to integrate Sonar and jenkins, go to manage Jenkins → select Systems:

Important

The public IP will change every time when we start and stop the instance.

Create a webhook in Sonarqube: Adimistration → Configuration → web hooks

Building Pipeline - 2:

pipeline {
    agent any
    
    tools{
        jdk 'jdk-17'
        nodejs 'node16'
    }
    environment {
        SCANNER_HOME=tool 'sonar-scanner'
    }

    stages {
        stage('Check Out') {
            steps {
                git branch: 'main', url: 'https://github.com/karthi770/Tertris_Game.git'
            }
        }
        stage('SonarQube analysis') {
            steps {
                withSonarQubeEnv('sonar-server'){
                    sh '''$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=TetrisV1 \
                    -Dsonar.projectKey=TetrisV1'''
                }
            }
        }
        // stage('Quality Gate') {
        //     steps {
        //         script {
        //         waitForQualityGate abortPipeline: false, credentialsId: 'sonar-token'
        //         }
        //     }
        // }
        stage('NPM') {
            steps {
                sh 'npm install'
            }
        }
        stage('TRIVY FS') {
            steps {
                sh 'trivy fs . > trivyfs.txt'
            }
        }
        stage('OWASP FS SCAN') {
            steps {
               dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'owap-check'
               dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
            }
        }
        stage('Docker build and push') {
            steps {
               script {
                   withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
                       sh '''
                       docker build -t tetrisv1 .
                       docker tag tetrisv1 karthi770/tetrisv1:latest
                       docker push karthi770/tetrisv1:latest
                       '''
                   }
               }
            }
        }
        stage('TETRIS IMAGE') {
            steps {
               sh 'trivy image karthi770/tetrisv1:latest > trivyimage.txt'
            }
        }
    }
}

While building the pipeline, we need the setup the sonar qube, go to sonar → click on projects → click on manually → create project with some name
Now the manually button has become locally. Click on that. Click on the use existing token option and enter the token generated previously in Sonarqube.

sonar-scanner
-Dsonar.projectKey=TetrisV1
-Dsonar.sources=.
-Dsonar.host.url=http://3.85.84.42:9000
-Dsonar.login=squ_ca4df954b64f07ab38623e24b94a863358284f0e

Quality gate pipeline Syntax:

Add Docker credentials: Provide the dockerhub userid and password To generate script for the docker:

Pipe line 3 - TETRIS V2 (Manifest and Triggers) Github token generation → Github account settings → Developer settings → Personal Access Token (Classic) → Generate token

pipeline {
     angent any
     environment {
        GIT_REPO_NAME = "Tertris_Manifest"
        GIT_USER_NAME = "karthi770"
    }
     stages {
        stage('Checkout Code') {
          steps {
            git branch: 'main', url: 'https://github.com/karthi770/Tertris_Manifest.git'
          }
        }
    
        stage('Update Deployment File') {
          steps {
            script {
              withCredentials([string(credentialsId: 'github', variable: 'GITHUB_TOKEN')]) {
                // Determine the image name dynamically based on your versioning strategy
                NEW_IMAGE_NAME = "karthi770/tetrisv1:latest"
    
                // Replace the image name in the deployment.yaml file
                sh "sed -i 's|image: .*|image: $NEW_IMAGE_NAME|' deployment.yml"
    
                // Git commands to stage, commit, and push the changes
                sh 'git add deployment.yml'
                sh "git commit -m 'Update deployment image to $NEW_IMAGE_NAME'"
                sh "git push https://${GITHUB_TOKEN}@github.com/${GIT_USER_NAME}/${GIT_REPO_NAME} HEAD:main"
              }
            }
          }
        }
     }
 }

ArgoCD installation: https://archive.eksworkshop.com/intermediate/290_argocd/install/

aws eks update-kubeconfig --name eks_cluster_01 --region us-east-1

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/v2.4.7/manifests/install.yaml

kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'

Load balancer is created. Wait for 2 minutes for the load balancer to be up and running.

sudo apt install jq -y

export ARGOCD_SERVER=`kubectl get svc argocd-server -n argocd -o json | jq --raw-output '.status.loadBalancer.ingress[0].hostname'`

echo $ARGOCD_SERVER

This will provide the link for the load balancer. ae2deb96b6dee42e6b5a54943e6030a5-1289219213.us-east-1.elb.amazonaws.com

Argo CD login

echo $ARGO_PWD

B9n0Yr9jKcbFJk8c

Click on New App and give the details below:

Load Balancer is updated with 2 more load balancers. a9a63be1ce1d646d992de2ef38693b1f-1954791266.us-east-1.elb.amazonaws.com

Updating Version - 2

pipeline {
    agent any
    
    tools{
        jdk 'jdk-17'
        nodejs 'node16'
    }
    environment {
        SCANNER_HOME=tool 'sonar-scanner'
    }

    stages {
        stage('Check Out') {
            steps {
                git branch: 'main', url: 'https://github.com/karthi770/Tertris_V2.git'
            }
        }
        stage('SonarQube analysis') {
            steps {
                withSonarQubeEnv('sonar-server'){
                    sh '''$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=TetrisV2 \
                    -Dsonar.projectKey=TetrisV2'''
                }
            }
        }
        // stage('Quality Gate') {
        //     steps {
        //         script {
        //         waitForQualityGate abortPipeline: false, credentialsId: 'sonar-token'
        //         }
        //     }
        // }
        stage('NPM') {
            steps {
                sh 'npm install'
            }
        }
        stage('TRIVY FS') {
            steps {
                sh 'trivy fs . > trivyfs.txt'
            }
        }
        stage('OWASP FS SCAN') {
            steps {
               dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'owap-check'
               dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
            }
        }
        stage('Docker build and push') {
            steps {
               script {
                   withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
                       sh '''
                       docker build -t tetrisv1 .
                       docker tag tetrisv1 karthi770/tetrisv2:latest
                       docker push karthi770/tetrisv2:latest
                       '''
                   }
               }
            }
        }
        stage('TETRIS IMAGE') {
            steps {
               sh 'trivy image karthi770/tetrisv2:latest > trivyimage.txt'
            }
        }
    }
}

 pipeline {
    agent any
    
    tools{
        jdk 'jdk-17'
        nodejs 'node16'
    }
    environment {
        SCANNER_HOME=tool 'sonar-scanner'
    }

    stages {
        stage('Check Out') {
            steps {
                git branch: 'main', url: 'https://github.com/karthi770/Tertris_V2.git'
            }
        }
        stage('SonarQube analysis') {
            steps {
                withSonarQubeEnv('sonar-server'){
                    sh '''$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=TetrisV2 \
                    -Dsonar.projectKey=TetrisV2'''
                }
            }
        }
        // stage('Quality Gate') {
        //     steps {
        //         script {
        //         waitForQualityGate abortPipeline: false, credentialsId: 'sonar-token'
        //         }
        //     }
        // }
        stage('NPM') {
            steps {
                sh 'npm install'
            }
        }
        stage('TRIVY FS') {
            steps {
                sh 'trivy fs . > trivyfs.txt'
            }
        }
        stage('OWASP FS SCAN') {
            steps {
               dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'owap-check'
               dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
            }
        }
        stage('Docker build and push') {
            steps {
               script {
                   withDockerRegistry(credentialsId: 'docker', toolName: 'docker') {
                       sh '''
                       docker build -t tetrisv2 -f ./Tetris-V2/Dockerfile .
                       docker tag tetrisv1 karthi770/tetrisv2:latest
                       docker push karthi770/tetrisv2:latest
                       '''
                   }
               }
            }
        }
        stage('TETRIS IMAGE') {
            steps {
               sh 'trivy image karthi770/tetrisv2:latest > trivyimage.txt'
            }
        }
        stage ('trigger manifest pipeline'){
            steps{
                build job: 'TETRIS V2 (Manifest and Triggers)', wait: true
            }
        }
    }
}

karthi770 / tetris_project_doc Goto Github PK

tetris_project_doc's Introduction

SUMMARY:

Jenkins Server

Setting up Jenkins:

tetris_project_doc's People

Contributors

Stargazers

Watchers

Forkers

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent