Gatekeeper is an Open Source platform for deploying various Data Engineering tools on AWS. It supports ease of use by giving the user a CLI which guides them through enabling kubernetes pods of their chosen technologies

Requirements
Terraform : https://learn.hashicorp.com/terraform/getting-started/install.html
AWS-CLI : https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
AWS-IAM : https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html
Go : https://golang.org/doc/install/source
AWS Account with EKS permissions.
Everything should be run from a bash shell.

Usage:

git clone https://github.com/cgoolsby/gateKeeper

cd gateKeeper

go run ./main.go

This produces a variables_override.tf in your main directory. In case you want to skip to this step in the future.

cp variables_override.tf terraForm/

cd terraForm

terraform init

terraform apply

type yes and go get a coffee.
Voila, you have the infrastructure of your pipeline.


Some Basic Commands to help you get started

kubectl get pods -o wide          ## a display of your currently running pods

kubectl get services -o wide      ## a display of your currently running services

kubectl exec -it $PODNAME -- bash  ## open a bash shell in a specific pod

------Be Aware that if you need more dependencies, you may need to edit kubernetes files. The easiest way to do this is to find a docker image which has the tech stack you need ----

In general, editing the yml files of your respective technologies should be straightforward. I HIGHLY recommend making a backup of each file before you begin.

MONITORING: Kubernetes Native Monitoring as well as Prometheus/Grafana are supported. Run Dashboard.sh for Kubernetes and/or Prometheus.sh for Prometheus/Grafana. For Prometheus, you will need to clone the prometheus operator from github to your desktop.

bash terraForm/Dashboard.sh #Kubernetes Dashboard

cd ~/Desktop/
https://github.com/coreos/prometheus-operator.git
cd -
bash terraform/Promethus.sh

TODOs: Implement sed feature to control # of master/worker nodes, and cpumin/maxing
Implement opening additional ports as per user specifications
Implement Web Server Toggle
Implement Prometheus Monitoring Toggle (Per Tech? or Just Overall?)