Git Product home page Git Product logo

ronanlee / ec2-spotter Goto Github PK

View Code? Open in Web Editor NEW

This project forked from refacktor/ec2-spotter

0.0 1.0 0.0 40.89 MB

Scripts for setting up an Active/Passive High Availability two-node cluster where the primary node is a Spot Instance and Elastic IP routing without the extra cost of a Load Balancer. Works at the OS level as to not require rearchitecting the application. Use for webservers and appservers. Do not attempt for database servers.

License: GNU General Public License v3.0

Shell 26.89% Python 0.32% Ruby 36.68% Batchfile 36.11%

ec2-spotter's Introduction

ec2-spotter 24x7

This folder contains the "24x7" High Availability version of ec2-spotter, which differs in both goals and implementation from the obsolete "classic" version.

This setup provides mostly uninterrupted availability by automatically failing over between a (cheap) reserved instance and a (powerful) spot instance, automatically managing Elastic IP Address reassignment as well as re-cloning the primary node every time a new spot instance is launched (to cut down on maintenance chores on applications that are not fully CI/CD'd).

This configuration is good for web or other servers which need to be up 24x7 but can tolerate a performance hit during a spot instance roll-over and do not keep critical dynamic information on the local filesystem. It's not good for databases.

This code was in production use for a while at CTC, until we migrated to Elastic Beanstalk. If EB is not a viable choice for your specific application, you may find ec2-spotter to be useful.

Basic Architecture

It's very simple. You have an application server running as a single-node, On-Demand or Reserved EC2 instance. Once you configure ec2-spotter, the base instance will clone itself onto a freshly launched Spot Instance, and will give its Elastic IP Address to the clone. The base instance will then watch the clone. When the clone dies, the base instance will reclaim the Elastic IP Address, then launch a new Spot Instance clone, rinse, repeat.

What's the point of all this?

The idea is that you size your base instance a t3.nano Reserved Instance (costs around ~$3/month) and size the Spot Instance to something like a t3.xlarge (normally ~$120/month comes down to $20-$30/month with spot savings). The t3.xlarge becomes your primary server and the t3.nano will step in when the spot price spikes, which is an extremely rare occurrence.

Really? Is that how you run your cloud workloads?

Actually no, I recommend AWS Lambda, ECS, and Elastic Beanstalk for newly developed applications. ec2-spotter is more of a niche solution for applications that can't be ported to those modern platforms.

Running ec2-spotter 24x7

  1. Copy example.conf to $(hostname).conf and modify to your environment.

  2. Install prereqs: AWS CLI and jq

  3. Setup two crontab entries on the base machine:

    * * * * * [ -f /etc/ec2spotter ] || spot /root/ec2-spotter/example.conf

    * * * * * [ -f /etc/ec2spotter ] && self-spot /root/ec2-spotter/example.conf

The reason you need two crontab entries is that they will run on both the base (reserved) and the cloned (spot) machines, but need to do something different in each (spot the other instance, or spot itself).

How does it work?

A crontab task is setup to run once every minute on the base instance and runs /root/ec2-spotter/spot. This "spot" script watches to ensure the spot instance is always running.

If a spot instance is not running, 'spot' starts by cloning the base instance (on-demand or reserved) into a Spot Instance. The spot instance takes over the Elastic IP address on successful launch.

When the Spot Instance is terminated, the Elastic IP address is reclaimed by the original base instance (again, "spot" script kicks in).

ec2-spotter's People

Contributors

refacktor avatar atramos avatar pndiku avatar emrul avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.