groeney / streaming-serverless Goto Github PK

Inspired by this blog and TodoMVC, this tool will leave the reader with an intuitive sense for event-driven systems, how they work and what their benefits are.

As opposed to Apache Kafka where all of the streaming components are packaged up into a platform this tool aims to give you more control of your event-driven system. Using Terraform and your preferred cloud provider, you can add and remove components to suit your needs. The architecture laid out here is production ready and works nicely in most cases.

This tool uses Change Data Capture as the event source and Serverless Architecture as the fundamental design pattern for data processing.

Prerequisites:

• darwin* OS
• Docker for Mac

Run ./scripts/setup.sh from the root to install project dependencies

• Create dotenv file: dotenv=./.env && cp "$dotenv.example" "$dotenv" && vi $dotenv -- update keys where appropriate (Sendgrid & Twilio)
• Checkout a version + runtime branch: git checkout v2-py
• Start stack: grunt start

This tutorial takes inspiration from TodoMVC and extends it by using serverless microservice architecture to send notifications to users when there is a change in there task list.

• Two versions of the stack.
  • v1: send email notifications on new task
  • v2: send email and sms notifications on new, edit and delete task.
• For each version there is a python3.6 lambda runtime and node8.10 lambda runtime.
• Checkout the following branches to see the respective version and runtime:
  • v1-js
  • v2-js
  • v1-py
  • v2-py
• Note that we set LAMBDA_EXECUTOR=docker-reuse on docker-compose.yaml:32 for these branches. This is for performance gains as LAMBDA_EXECUTOR=docker starts from "cold start" for every invocation.
• Copy .env.example into .env and replace keys where appropriate
• Running grunt start will build three services from docker-compose.yml:
  • localstack: mock AWS Cloud services on local. See here
  • client: this is a basic vuejs application inspired by TodoMVC
  • server: simple node + express application for Todo list

• Running grunt watch:lambda will deploy changes to infrastructure when it detects changes to any lambda file
• This is nice when for watching the lambda containers: watch -n 1 'docker ps -a | grep lambda'

Messages must be matched by the following regex:

/^(revert: )?(feat|fix|style|refactor|chore)(\(.+\))?: .{1,50}/;