Git Product home page Git Product logo

ndn-k8s's Introduction

Named Data Networking on Kubernetes

This is a collection of tools for orchestrating Named-Data Networking(NDN) comtainers on Kubernetes.

Instructions for creating a StorageClass and PVC are included.

Installation

First, install dependencies:

  • kubectl
  • Helm(optional)

Make sure kubectl is configured properly with the Kubernetes cluster of your choosing.

Install Helm(optional)

Install Helm:

cd PATH

wget https://get.helm.sh/helm-v3.6.0-linux-amd64.tar.gz

tar -xvf helm-v3.6.0-linux-amd64.tar.gz

sudo cp linux-amd64/helm /usr/local/bin

Add the stable repo:

helm repo add stable https://charts.helm.sh/stable

Update Helm's repositories(similar to apt-get update):

helm repo update

The K8s cluster MUST have either a valid PVC or storage class. If a valid PVC does not exist, here are some example instructions to set up a NFS storage class:

Create NFS StorageClass and PVC (optional)

Install Helm

Add the "stable" repository:

helm repo add stable https://kubernetes-charts.storage.googleapis.com

Update Helm's repositories(similar to apt-get update):

helm repo update

Next, install a NFS provisioner onto the K8s cluster to permit dynamic provisoning for 50Gb(example) of persistent data:

helm install kf stable/nfs-server-provisioner \

--set=persistence.enabled=true,persistence.storageClass=standard,persistence.size=50Gi

Check that the nfs storage class exists:

kubectl get sc

To deploy the PVC(if needed), run kubectl create -f pvc.yaml

Copy Execution Scripts to PVC

The standard deployment types use predefined scripts. These scripts need to be moved to the PVC so the deployed containers can execute them. To do so:

./kube-load.sh <PVC_NAME> genomics/scripts /workspace/ndn

Method 1: kubectl

Using Helm is recommended, but here are instructions to deploy an NDN consumer using kubectl.

Configuration

Edit the .yaml files. Make sure that the names and resource requests are correct and compatible.

Make sure the PVC defined in deployment.yaml is valid.

Deployment

To deploy NDN, run kubectl create -f endpoint.yaml

Delete

To delete the deployment/pvc, simply run kubectl delete -f endpoint.yaml and kubectl delete -f pvc.yaml

Method 2: Helm(recommended)

Using the included Helm chart is recommended because of the configurability and versatility of the various deployment types.

4 basic deployments exist currently:

  • Manual(do anything interactively)
  • Producer(publish files)
  • Consumer(pull files from a producer)
  • Ingress(multi-container producer service) Work in progress

Configuration

Edit helm/values.yaml. Make sure that the names and resource requests are correct and compatible. There are example arguments in the default file, and preconfigured examples named helm/values.yaml.<TYPE>

Make sure the PVC is valid.

Also make sure any files used in arguments exist on the PVC.

The 1GB and 5GB example files can be downloaded here.

Deployment

To deploy NDN, run helm install <DEPLOYMENT_NAME> helm/

Delete

To delete the deployment, run helm uninstall <DEPLOYMENT_NAME>

Usage

Use kubectl get pods to be sure your container is running properly.

To get an interactive terminal as root, use kubectl exec -it <POD_NAME> -- /bin/bash

Publish

Example instructions to publish a file:

Download the file to persistent storage:

wget -O /workspace/1GB https://ftp.ncbi.nlm.nih.gov/1GB

Publish the file under the path of your choosing:

ndnputchunks /k8s/1GB < /workspace/1GB

It will be published within the cluster with the hostname <DEPLOYMENT_NAME>, which is configured in helm/values.yaml.

For larger files, the cache size must be increased in nfd.conf and NFD must be restarted

Pull Published Files

Example instructions to pull a file:

Add the face: nfdc face create tcp4://<PUBLISHER_HOST>

Add the route: nfdc route add /k8s tcp4://<PUBLISHER_HOST>

Pull the file: ndncatchuncks /k8s/1GB > /tmp/1GB.tmp

This example uses TCP to pull the published file 1GB with path /k8s/1GB.

Restart NFD

To apply new NFD configurations, you must restart NFD, which is running within the container by default:

killall nfd or pkill nfd

This will interrupt any NDN services on the container.

After edits are made, start NFD again with /usr/local/bin/nfd -c $CONFIG > $LOG_FILE 2>&1

The files at $CONFIG and $LOG_FILE are not persistent. Moving custom config files to peristent storage is advised.

ndn-k8s's People

Contributors

cbmckni avatar

Stargazers

Pratik Pathak avatar Dirk Kutscher avatar Sata avatar Abdel-moniem E. REZK avatar

Watchers

Susmit avatar  avatar David Reddick avatar

Forkers

feltus

ndn-k8s's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.