Git Product home page Git Product logo

platform-architecture's Introduction

FABRIC: AWS K8

For Users:

  1. Create a Project
  2. Create a data-repo for the Project
  3. Create analysis pipeline
  4. Preview outputs
  5. Consume output using:
  • Web Application
  • API
  • BI Tool

For Developers:

###Infrastrcture

  1. "AWS K8" infrastructure defined here is deployed on domain inkeystone.com
  2. You will need access to private keys if you wish to update deployed state - first go to the master branch and follow that readme.
  3. This deploys Kubernetes (k8) cluster (3 nodes: 1 master, 2 service) on top of a secure networking architecture
  4. This Infrastructure is managed by Terraform scripts and the current state is stored in s3://inkeystone-com-state:
  • Networking: /labroots.tfstate
  • K8: /labroots-kubernetes.tfstate
  • Data: /labroots.inkeystone.com/
  1. Access the Admin Dashbaord for the k8 cluster @ api.labroots.inkeystone.com

View Infrastructure Diagram

###Deployed Services (on the platform)

  1. Pachyderm: Data Lake with Versioning & Language Agnostic Analysis Pipelines
  • State is stored in s3://inkeystone-com-pachd
  1. Kubeflow (Model Management)
  2. Dremio (Data Virtualization)

Dashboard UI

platform-architecture's People

Contributors

mosha101 avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.