Git Product home page Git Product logo

gke-glusterfs-heketi's Introduction

Hyper-converged GlusterFS and Heketi Dynamic Volume Provisioning on Google Container Engine (GKE)

Original project is DEAD.

The updates provide current working scripts tested on Kubernetes 1.17, and also replace the scaling out with multi zone support, as it was originally creating all resources in a single zone.

Open Items To Fix

The script 'scripts/job/02-set-kernel-modules.sh' needs to be converted to DaemonSet instead of running commands directly on the kubernetes nodes. This will not survive upgrades, reconfiguring, or new nodes.

Usage

  1. Edit config to match your GKE project/zone
  2. Source helpers
  3. Generate k8s (run gke_glusterfs_heketi_generate_k8s)
  4. Build docker image (run gke_glusterfs_heketi_build_image)
  5. Push docker image (run gke_glusterfs_heketi_push_image)
  6. Create a cluster (run gke_glusterfs_heketi_create_cluster if you want)
  7. Configure cluster permissions (RBAC) (run gke_glusterfs_heketi_configure_rbac)
  8. Deploy Job within the cluster (run gke_glusterfs_heketi_deploy_glusterfs_heketi)
  9. Wait for it to finish (tail the logs if you want with: gke_glusterfs_heketi_tail_job_logs) NOTE: This takes forever. The script that runs gk-deploy -g to deploy glusterfs runs a job and the script has to wait for the job to time out right now before proceeding to create the necessary firewall rules and storage class.

Testing it worked

You can deploy the example k8s (mariadb statefulset) to test that everything works.

  1. kubectl apply -f k8s-example

If the mariadb pod gets stuck in "pending", you may need to recreate the storage class (I'm not sure why this is a bug).

This can be taken care of with gke_glusterfs_heketi_if_storage_class_not_found_during_k8s_example_run_me

Tear down / Clean up

  1. Run gke_glusterfs_heketi_delete_cluster_and_disks

Installation flow

NOTE: All of this is automated. This is included purely for documentation purposes.

  1. Create a cluster with at least 3 nodes
  2. Create persistent disks and attach to the nodes
  3. Load necessary kernel modules for GlusterFS, install glusterfs-client on host machines
  4. Generate storage network topology
  5. Create necessary firewall rules
  6. Run gk-deploy -g to deploy the glusterfs daemonset and heketi
  7. Change heketi service from ClusterIP to NodePort (will i/o timeout otherwise during persistent volume claim)
  8. Update firewall rules to allow new heketi node port
  9. Deploy heketi/glusterfs storage class using <any node ip>:<heketi nodeport>

gke-glusterfs-heketi's People

Contributors

jefferypalmer avatar stevenaldinger avatar wizard1024 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.