acm-uiuc / clearwaters-container Goto Github PK
View Code? Open in Web Editor NEWContainer Management for the Clearwaters Cluster
Container Management for the Clearwaters Cluster
Making a REST endpoint to create/delete new docker containers. Similar to gpu-cluster-backend.
As stated in #2 , there will exist worker daemons for managing each worker locally, however a head-node manager must exist as well. The head node manage will do the following tasks:
A communication protocol will need to be decided between the workers and the head-node .
Before we can implement the scheduler in code, we should decide on how the jobs will be scheduled.
We have a few options, based on current schedulers, such as round robin, CFS, or even FIFO. However, since we will most likely have a job queue and also are running on multiple systems, we will need to ensure that all tasks are run fairly and also that workers are not overloaded with tasks while some do no work at all.
A worker daemon will be created for monitoring the status of worker and executing tasks on the worker. The daemon should be able to complete the following tasks at a minimum.
Ideally the worker daemon has a config file located somewhere in /etc/
and runs as a service, so that if a worker crashes, it can immediately be restarted.
What we want to do is create containers on all of our worker nodes with a single command from the head node.
The container image we are using is:
registry.gitlab.com/acm-uiuc/sigops/clearwaters-docker/ubuntu-mpich-arm64
Which can be grabbed using:
docker pull registry.gitlab.com/acm-uiuc/sigops/clearwaters-docker/ubuntu-mpich-arm64
I imagine a very simple workflow being the head-node sending a message to all client daemons to create the containers, and the client daemons then responding with the IDs of the created containers.
Its lacking at the moment
Make sure the round robin algorithms labels each container formed with an id and name and adds to a log and also documents the size of f for simulation purposes before it actually goes on the cluster
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.