Git Product home page Git Product logo

scalable_mandelbrot's Introduction

scalable_mandelbrot

A mandelbrot generator written in go to demonstrate architecture scalability, horizontal and vertical.

What does it consist of ?

  • a single thread mandelbrot generator
  • a multi thread mandelbrot generator
  • a multi machines mandelbrot generator using http servers locally
  • a multi machines mandelbrot generator using docker
  • a multi machines mandelbrot generator using kubernetes

Single Thread Mandelbrot

This go script simply compute the mandelbrot on a single thread

Multi Thread Mandelbrot (vertical scalability)

This go script makes use of goroutines to compute the mandelbrot. The mandelbrot generating works in pixels such as to generate an iamge of a certain width and height. This means that the width equals the number of columns. That being said, the mandelbrot is generated by having a goroutine for each column.

Multi Machines Mandelbrot (horizontal scalability)

This part had two main steps. First we developed a system with a master and multiple slaves. This part is hardcoded, the master knows how many slaves are running and will divide the mandelbrot generation in the number of slaves that are available. The computation is thus done by the slaves which all run as http servers. When they are done calculating, they return the encoded bytes of de mandelbrot specific part that they have generated and the master script combines these arrays of bytes to generate the final image of the mandelbrot.

The second step was to migrate these script in docker images to simulate a horizontal scalability. This would keep the same structure of master and slaves except that we would have an additional player, a kind proxy. The slaves (servers) register to the proxy. Then when the client or master wants to generate a mandelbrot, it will first make a request for the servers that are up. Depending on the result, it will divide the work in the number of connected servers and start the generation of the mandelbrot.

This was then followed by a migration of the docker application to kubernetes which asked some refactoring as teh goal here was to have only a single connection between the kubernetes cluster and client. This connection was thus handled by the proxy. More information can found in the README in horizontal_k8s.

scalable_mandelbrot's People

Contributors

g3ryon avatar haroldsnyers avatar

Watchers

 avatar

scalable_mandelbrot's Issues

Update proxy such as it acts like a real proxy

Problem ?
Proxy acts now only as register server and client makes direct request to the slave servers.

What should it be
Client should only interact with proxies and have no ideas of what is happening behind the scene. The idea would thus be to make the get request for the different servers directly from the proxy. Client would then only make the request to the proxy which would handle the payload by itself and return the sum of pixels back.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.