Git Product home page Git Product logo

csye7125-fall2023-group05 / webapp Goto Github PK

View Code? Open in Web Editor NEW
0.0 0.0 2.0 611 KB

Production-ready REST API that creates healthcheck data for websites in Postgresql along with creating a custom resource to monitor the health data of these websites periodically using a CronJob

License: MIT License

JavaScript 99.42% Dockerfile 0.58%
ci-cd github-actions jenkins jenkins-pipeline jenkinsfile nodejs postgresql rest-api restful-api restful-webservices semantic-release sequelize-orm winston-logger

webapp's Introduction

πŸ•ΈοΈ webapp

RESTful Backend API service for a fullstack web application.

πŸ“¦ Prerequisites

To install and run the app locally, you need to have the following installed on your local system:

  • git (configured with ssh) [link]
  • node v.16.17.0 and above [link]
  • Postman to demo hit the APIs [link]

‡️ Installation

Prerequisite: You need to have ssh configured on your local system to clone this project using ssh.

  • Clone the server side API service using the following command:
git clone [email protected]:cyse7125-fall2023-group05/webapp.git

The above command will clone the organization repository. Incase you want to clone the forked repository, use the following command:

git clone [email protected]:sydrawat01/webapp.git
  • You'll need to install the dependencies as well:
  npm i

πŸ› οΈ Development

Make sure to have a valid .env file before running the following commands. You can view the .env.example file for reference.

To run the server in dev mode, run the following command:

  npm run start:dev

This serves the app on http://localhost:3000 unless you specify a PORT number in the .env file.

🚏 API Endpoints

This is a simple RESTful API that checks if there is a successful connection to the database.

πŸš‘ Health

  • GET /healthz : Get the health of the API
    • Response: 200 OK

πŸ” Unsuccessful Connection to Database

  • GET /healthz : Get the health of the API
    • Response: 503 Service Unavailable

⛔️ Request Method Not Allowed

  • PUT /healthz : Get the health of the API
    • Response: 405 Method Not Allowed
  • POST /healthz : Get the health of the API
    • Response: 405 Method Not Allowed
  • PATCH /healthz : Get the health of the API
    • Response: 405 Method Not Allowed
  • DELETE /healthz : Get the health of the API
    • Response: 405 Method Not Allowed

❌ Random Endpoints

  • GET /* : Random API endpoint
    • Response: 404 Not Found
  • PUT /* : Random API endpoint
    • Response: 404 Not Found
  • POST /* : Random API endpoint
    • Response: 404 Not Found
  • PATCH /* : Random API endpoint
    • Response: 404 Not Found
  • DELETE /* : Random API endpoint
    • Response: 404 Not Found

πŸ§ͺ Testing

To run the test suite with unit/integration testing, use the following commands:

  • To run the test suite in interactive mode:
  npm run test:dev
  • To run the test suite without interactive mode:
  npm run test

πŸš€ Production

To build the app and run the app in production mode, use the following command:

  npm run build

🐳 Containerize the application

We will use Docker to containerize our application.

‡️ Install Docker Engine

Head to https://docs.docker.com/engine/install/ to install the Docker service for the required OS.

If you're using a MacOS based workstation, use this link and follow the instructions mentioned to install Docker Desktop for MacOS (Intel/Apple Silicon).

NOTE: Beginning with Docker Desktop 4.3.0, Docker has removed the hard requirement to install Rosetta 2. There are a few optional command line tools that still require Rosetta 2 when using Darwin/AMD64. See known issues. However, to get the best experience, we recommend that you install Rosetta 2. To install Rosetta 2 manually from the command line, run the following command:

softwareupdate --install-rosetta

To verify the installation, run:

docker version

Finally, create a docker hub account and login to your account through your terminal:

docker login -u <username>

πŸ”¨ Working with Docker

To build a docker image, we need to first be in the root folder of the application where our Dockerfile and .dockerignore files are present. This provides docker context as to what to build and from where. The .dockerignore file is used to ignore files and folder from the build image. This reduces image overhead by not including things that are not required to build and run the image.

A basic Dockerfile format looks like this:

FROM node:lts-alpine

WORKDIR /usr/src/webapp

COPY package*.json ./

RUN npm install

COPY . .

EXPOSE 3000

CMD ["npm", "run", "start:dev"]

More details on what each command does, is available in the official Docker documentation.

Once we have all the required docker files, we are now ready to build a docker image.

  • To build images specific to your system hardware architecture, use the build command:

    # -t flag for tag name, -f flag for Dockerfile name
    # --no-cache tells docker to ignore any cached layers during the build process
    # the final `.` is used to represent the build-context
    docker build --no-cache -t <dockerhub_username>/<image_name>:<tag> -f <Dockerfile-name> .
  • To push the images to docker hub:

    # this pushes the docker images to the default docker hub.
    docker image push --all-tags <dockerhub_username>/<image_name>
  • To tag and push the docker image to another registry host (other than dockerhub):

    # tag the image
    docker -t <image_name>:<tag> <registry_host>/<username>/<image_name>:<tag>
    # push to registry host
    docker push <registry_host>/<username>/<image_name>:<tag>
  • To run a container locally based on a docker image, use the following command:

    # -ti flag will attach to the container in interactive mode
    # -p flag is used to expose ports from container to local machine
    # --rm flag is used to delete the container once docker is closed
    docker run -ti --rm --name <container_name> -p <local_port>:<container_port> [registry_name]/<username>/<image_name>:<tag>
  • Clean all docker images, resources, containers and build cache using:

    # `docker system df` shows total space docker objects currently use
    docker system prune -a -f
  • To build images for various architectures (multi-platform build), use the buildx CLI tool provided by Docker:

    docker buildx create --use
    docker buildx ls
    docker buildx build --platform=linux/amd64,linux/arm64 -f <Dockerfile_name> --no-cache -t [registry_host]/<username>/<image_name>:<tag> -t [registry_name]<username>/<image_name>:<tag> .

Here are some other useful commands for working with Docker:

  • Docker logs: docker logs <ID or container_name>
  • Docker interactive shell: docker exec -ti <ID or container_name> /bin/bash
  • Docker stats: docker stats <ID or container_name>
  • Docker inspect: docker inspect <ID or container_name>
  • Docker pull: docker pull <image_name>:<tag>
  • Docker login (similar for other registry hosts):
    • Dockerhub: docker login
    • Quay.io: docker login quay.io
  • Limit CPU and Memory: docker run --cpus=".1" -m "8m

    b, k, m, g indicate bytes, kilobytes, megabytes, or gigabytes.

⎈ Kubernetes and power tools

We will need kubernetes on our workstation to create clusters where we can run deployments to our application.

NOTE: The steps mentioned below are only applicable to MacOS, for other distros, please use the documentation to guide you through the installation process.

  • Install minikube and kind

    brew install minikube # for single cluster setups
    brew install kind # for multi-cluster setups
    # verify installation
    minikube version
    kind version
  • Install kubectl and set alias

    # check if you have `kubectl` since it comes pre-installed with `Docker`
    kubectl version
    # if not installed, use:
    brew install kubernetes-cli

    To set an alias, edit your .zshrc file to include an alias for kubectl.

    # manage `.zsh_aliases separately`
    alias k="kubectl"
  • Install power tools for k8s: k9s, kubectxandkubens`

    We will use these tools extensively when working with k8s clusters.

    # install k9s
    brew install k9s
    # validate k9s version
    k9s version
    # `kubectx` installs `kubens` as well
    brew install kubectx
  • Install tmux, a powerful terminal window multiplexer tool

    brew install tmux

TIP: Checkout sydrawat01/dotfiles for customization options for your terminal.

βš™οΈ Working with k8s

To create a cluster for our RESTful API, we will use the containerized image create and uploaded to a docker image hub, which will be used in our Deployment resource. Additionally, we will also need to create a Service resource that will expose our app through a LoadBalancer via a targetPort.

A ConfigMap and a Secret resource is required to configure the environment variables for the webapp application and flyway database migration configurations.

  • To create a k8s cluster, we will first need to start minikube:

    minikube start
  • Create a namespace for our webapp deployment:

    k create namespace <ns-name>
    # to delete a namespace:
    k delete namespaces <ns-name>
  • Switch namespace from default to your <ns-name> namespace (from the previous step):

    kubens <ns-name>
  • Create the cluster with all the resources:

    k apply -f <path/to/>k8s/
  • Monitor the cluster using k9s:

    k9s
  • To delete the cluster and its resources:

    k delete -f <path/to/>k8s/
  • To stop minikube and delete all resources create by it, use:

    minikube stop
    minikube delete --all
    # you will still need to manually delete the minikube docker image!

πŸ₯· Author

Siddharth Rawat Karan Wadhwa Rishab Agarwal

πŸ“œ License

License: MIT

webapp's People

Contributors

karanwadhwa avatar semantic-release-bot avatar sydrawat01 avatar rishabneu avatar

webapp's Issues

πŸ“ Update docs

  • Include steps to install dependencies and run the REST API locally
  • Include steps on how to configure Dockerfile to run the REST API on a docker container which connects to the local postgresql service running on the local workstation.

NOTE: DBHOST = host.docker.internal is required when running the API docker container locally, which connects to the local postgresql service on PORT 5432.

🐞 Update HTTP errors for 404/405

The following HTTP error codes should be handled for the /healthz endpoint:

  • 405 Method Not Allowed should only be thrown when the requesting method is anything other than the GET method.
  • 400 Bad Request should be thrown in all other cases except when the /healthz endpoint successfully connects to the database in the back-end (which eventually returns 200 OK).
  • 503 Service Unavailable in case the app cannot connect to the database on the back-end.
  • Handle payload body with proper error codes (ideally, 400 Bad Request).

Currently the app throws a 405 Method Not Allowed for all requests that are not /healthz (along with properly working for methods other than GET).

🌡 Create k8s deployment for webapp and migration

  • k8s deployment which will create a replicaSet with minimum 1 replicas.
  • The replicaSet consists of a container which will run our webapp REST API service and an initContainer which will run our database migration scripts using flyway.
  • The initContainer runs before the webapp (container) is up and running.
  • Use k8s ConfigMaps and Secrets to configure the environment variables for the various containers and initContainer within the replicaSet.

πŸ—„οΈ Postgres db to be run via Docker container

Use the docker compose tool to create a Posgres service that will connect with our application and migration deployments.

ref:

services:
  postgres:
    hostname: postgres_local
    image: postgres
    ports:
      - "5432:5432"
    networks:
      - csye7125
    environment:
      - POSTGRES_USER=csye7125
      - PGUSER=csye7125
      - POSTGRES_PASSWORD=password
      - POSTGRES_DB=csye7125
      - PGDATA=/var/lib/postgresql/data
      - TZ=GMT
      - PGTZ=GMT
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready" , "-U", "csye7125", "-d", "csye7125" ]
      interval: 5s
      timeout: 5s
      retries: 10
      start_period: 10s

networks:
  csye7125:
    driver: bridge

πŸ—„οΈ health check to include db connection

The database connection should return 200 OK or 503 Unavailable depending on what is the status of the database service. In our case, we're using postgres, so we will have to test the connection accordingly.

Postgres works on PORT: 5432

🟒 CD pipeline for webapp: helm release install/upgrade on GKE cluster

  • Code changes are committed to GitHub. Jenkins can either monitor GitHub for changes or it can be configured to be notified by GitHub about the new commit.
  • Clone the repository with the latest code and build the container. Tag containers with git commit hash.
  • Push the container to your container registry.
  • Download the latest release of your Helm Chart from the repository.
  • Check if a (helm) release for the application exists on the cluster.
  • If a release does not exist, use the helm install command to install a release.
  • If a release exists, use the helm upgrade command to upgrade the release.
  • Helm release must use the new container built as part of this pipeline.

βž• Add more values to k8s util spec

Based on the payload body and the ConfigMap which the custom CronJob k8s controller will create, add the following fields to the spec section where the post request creates the Custom Resource for the controller to monitor:

  • name
  • rename retries to num_retries [to comply with the payload spec]
  • uptime_sla
  • response_time_sla
  • check_interval_in_seconds

Other payload objects (mentioned below) are subject to be added only if the controller will monitor those values in the ConfigMap:

  • is_paused
  • use_ssl

Things to be removed from spec:

  • res_code : since this is the response we get from the fetch() API call to get the status of the website in the producer app

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.