Git Product home page Git Product logo

osseus's Introduction

Osseus

Github license

Osseus is full-stack web application for generating configurable plugin templates to be used in a wide variety of cloud-based applications. The user is able to select from available plugins provided by CN-Infra and generate working Go code to be immediately usable in an application.

๐Ÿš€ Quickstart

Requirements:

Prereq. Version
NodeJS 10.15^
Go 1.10^
Docker 18.09^

First, clone the repo:

git clone https://github.com/ligato/osseus
cd /osseus

Install dependencies

# Install dependencies for the agent
dep ensure

# Install dependencies for the client
cd client/
npm install

Run each component in a separate terminal

# Run Etcd
sudo docker run -p 12379:12379 --name etcd --rm \ 
    quay.io/coreos/etcd:v3.3.8 /usr/local/bin/etcd \ 
    -advertise-client-urls http://0.0.0.0:12379 \ 
    -listen-client-urls http://0.0.0.0:12379

# Run agent
cd cmd/agent/
go run main.go

# Run client
cd client/
npm start

๐Ÿ“– Documentation

Detailed documentation for each component of the application can be found here.

๐Ÿ”ง Architecture

Osseus was built using the CN-Infra framework, which provides component/library support and plugin lifecycle management to our application. Our development strategy was to split its production into separate efforts: building a user interface (UI), REST API and a code generator. The UI provides a way for the user to configure an agent setup consisting of chosen plugins. The REST API facilitates the communication between the UI and our data store (Etcd), which takes the user selected plugins and stores them under a certain key. The Generator then produces skeleton code for the userโ€™s configuration and stores this back in Etcd where the UI webhook can retrieve this configuration in a tar file format.

The architecture of the Osseus web application is shown below:

Osseus Architecture

We use React & SASS for our frontend, which is a component-based JavaScript library and a feature-rich CSS extension language. Go was chosen as our backend language due to the consistency of developing with CN-Infra, where we are able to use packages that are built for ease-of-use in the design of our generator and restapi plugins. Lastly, ETCD allows for multiversion persistent key-value storage and was chosen for its integration with various plugins/libraries through CN-Infra.

๐Ÿ’ก Contributing

Contributions to Osseus are welcome. We use the standard pull request model. You can either pick an open issue and assign it to yourself or open a new issue and discuss your feature.

The tool used for managing third-party dependencies is dep. After adding or updating a dependency in Gopkg.toml run dep ensure to download specified dependencies into the vendor folder.

๐Ÿ“ License

Licensed under the Apache License

osseus's People

Contributors

anthonydevelops avatar sctsui avatar kevin-velasquez avatar brecode avatar jmedved avatar

Stargazers

Niranjan Anandkumar avatar  avatar  avatar  avatar  avatar

Watchers

James Cloos avatar Ed Warnicke avatar  avatar  avatar  avatar  avatar Junmin Pan avatar Vladimir Lavor avatar Milan Lenฤo avatar  avatar  avatar  avatar

osseus's Issues

Load All projects

Loading all projects saved by the user is currently only partially implemented. Progress is blocked by the broker's listKeys() method not listing the keys of prefix '/config/generator/v1/projects/' in etcd. ListValues() works, given the correct key.

Relevant code can be found in
https://github.com/sctsui/osseus/blob/feature/loadAll/plugins/restapi/handlers.go
method getAlProjects() (line 308)
Frontend recieving allProjects json will be done in line 77-79 of client/server.js

Etcd api-gateway watch error

Currently, watching using Node Webhooks has an issue when using Etcd's api-gateway, specifically with watching. Using the watch api request used to work but stopped randomly after working with the node error:

MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 connection listeners added. Use emitter.setMaxListeners() to increase limit

Change the following code in /client/server.js to that of how Etcd's doc uses watching to see that nothing is returned:
This: webHooks.add('etcd', http://${etcd}/v3beta/kv/range)
To this: webHooks.add('etcd', http://${etcd}/v3beta/watch)

Deploy a K-V store

Pick a KV store of your choice (why did you choose it)

  • Make yourself familiar w/ the data structures saved and how to access it.
  • Define some arbitrary data to use
  • Where is it going to be running (vm, container, host) and why

Create CN-Infra skeleton

a. Add default plugins
b. Plugins to be used

Which plugins are going to be used by default and why?
What plugins are going to be needed in the future and why?

Conf files generation

UI
-Create a way of capturing user input on plugin click. A possible method would be to use a swal2 popup and have input fields for each configurable option of the given plugin.
-This user input needs to be reflected in the model ('client/src/components/Model.js') sent to the backend.

Backend
-Generate a template for each configuration file with the values sent from the frontend model.
-Add the generated template files for each conf.go into the tar file and template structure object under cmd/agent.

Link to compiled configurations:
https://docs.google.com/document/d/1enneGXDEmkgr_Gm0UvLTEeiHsl2YK6VxFSyXWWYGqhE/edit?usp=sharing

Change async loading of template and tar file

Loading of template is done in 'client/src/components/GeneratorApp/GeneratorApp.js' lines 59-69.
Loading of tar is done in 'client/src/components/GeneratorApp/Header/GeneratorAppHeader.js' lines 77-82.

Current implementation, due to time constraints, was to use a setTimeout() before attempting to retrieve data from the node.js server in server.js. This is an issue due to not being fault tolerant. ie. a slow connection might create a situation where the data isn't ready to be retrieved after the setTimeout() has expired or a large template generation might take longer, hence not be ready to be retrieved in time.

Methods to try:
Redux connect()
Redux subscribe()
Using Promises to wait until state changes

Update README with project info

I think an accurate README would be very useful to describe our system. If we could update it to show objectives, requirements, installation, & how to run it locally, it would help a lot for those seeing our project for the first time. Please look at CN-Infra or Gatsby as a reference on how to do it correctly.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.