Git Product home page Git Product logo

argo-ml-demo's Introduction

Argo-ML-Demo

An Argo Workflow that takes in a photo, scans it for faces and produces a 3D obj rendering for each face detected. Our project uses Ageitgey's face_recognition to detect individual faces within a photo, YadiaraF's PRNet to create a 3D Face Reconstruction, and various other bash commands inside of Ubuntu and Debian containers for most of the other pods. This project uses various new Argo Workflow features such as conditional deployments, parallelism, custom resources, and loops. Our project also utilizes kubeflow and tensorflow in order to run PRNet. PRNet uses CUDA which requires access to a GPU (We use the NVIDIA Tesla P100). In order to run our workflow at high performance we used a GKE cluster.

Getting Started

Prerequisites

In order to run our project you must have the following installed.

  • Argo
argo version
argo: v2.1.1
  BuildDate: 2018-05-29T20:38:37Z
  GitCommit: ac241c95c13f08e868cd6f5ee32c9ce273e239ff
  GitTreeState: clean
  GitTag: v2.1.1
  GoVersion: go1.9.3
  Compiler: gc
  Platform: darwin/amd64
  • Kubectl
kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T22:29:25Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10+", GitVersion:"v1.10.5-gke.3", GitCommit:"6265b9797fc8680c8395abeab12c1e3bad14069a", GitTreeState:"clean", BuildDate:"2018-07-19T23:02:51Z", GoVersion:"go1.9.3b4", Compiler:"gc", Platform:"linux/amd64"}
  • Ksonnet
ks version
ksonnet version: 0.12.0-rc1
jsonnet version: v0.10.0
client-go version:
  • Kubeflow

Instructions for deploying kubeflow can be found here

  • Kubernetes Cluster w/ NVIDIA GPU's enabled

For GKE users, GPU deployment can be found here

For other users, make sure you have the NVIDIA driver set up in your cluster. The following command will install a Daemeonset with the driver which is needed in order to utilize the limits: nvidia.com/gpu: 1 line in the yaml.

kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/container-engine-accelerators/stable/nvidia-driver-installer/cos/daemonset-preloaded.yaml

Custom images

This project uses some custom docker images in the workflow. The face recognition project can be found here

Give Workflow RBAC privileges

In order for a workflow to create custom resources (e.g. TensorflowJob), a cluster-admin role should be used. The following will grant the default service account in the default namespace cluster-admin: kubectl create rolebinding default-admin --clusterrole=cluster-admin --serviceaccount=default:default

Deploy Our Project

git clone https://github.com/argoproj/argo-ml-demo.git
cd argo-ml-demo
# image must be publicly available on the internet
argo submit demo.yaml -p image-url="yourimagehere.jpg"

Authors

Acknowledgments

Special thanks to:

argo-ml-demo's People

Contributors

dthomson25 avatar edansneh avatar jessesuen avatar

Stargazers

 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.