Git Product home page Git Product logo

smokex's Introduction

Deploy to Heroku Run test suite and linters

Smokex.Umbrella

Deployment

Heroku

This project is deployed to Heroku using Docker containers.

  • Login into Heroku registry heroku container:login
  • Build and push an image heroku container:push web

Logs

heroku drains:add 'https://http-intake.logs.datadoghq.com/v1/input/<DD_APY_KEY>?ddsource=heroku&service=smokex&host=gigalixir' -a smokex

Run a release locally

There are multiple environment variables that are expected to be set in order to run a release:

SECRET_KEY_BASE=kGXrNEYUVAm2zOpB8UQMRfK+JkDnqFcH4WOcM8nYApN/fMWVJoQPMGqrUTwv15w5 DATABASE_HOSTNAME=postgres-free-tier-1.gigalixir.com DATABASE_USERNAME=test DATABASE_PASSWORD=test DATABASE_NAME=test PORT=4000 POOL_SIZE=1 DATABASE_URL="" STRIPE_API_KEY="" STRIPE_SIGNING_SECRET="" _build/prod/rel/smokex/bin/smokex start_iex

Need to know

  • The Stripe button to buy a subscription requires environment variables to be configured or it will throw the unexpected item shape error if not.
  • The Slack integration also requires a runtime configuration therefore it will fail locally. Also, the redirection done by Slack is to production host.

Architecture

architecture.png

Why Smokex?

  • Not everybody has CI pipelines
  • You don't want to mix CI executions with smoke testing
  • You want a clear way to identify downtimes, errors...
  • Monitor any legacy project with little effort
  • But also easily run smoke tests on a new project to validate deployments
  • Can give access to third parties without to have access to source code o sensitive tools
  • Won't require technical knowledge
  • Won't be coupled to any testing library and making changes transparent to the repo

Experience situations where Smokex tests were needed

  • Empty deployment with health check workng ok but actual endpoints were all failing
  • App requires some environment variables set before start, they were available on deployment so all went good. After some days the app got restarted but the variables were not available anymore (were managed by a third party service), entering in a crash loop and no one noticing.

Uses cases (smoke tests & HTTP automation)

  • Schedule a HTTP request to keep Heroku instances up.
  • Send a ping request to ensure a system is reachable.
  • Retrieve a information from currency prices and send it to your server to get update value #nocode

Distributed application

The current application is a single monolith, although it uses umbrella applications running in a single node.

The goal is release several application instances with role worker that will be executing the actions triggered by an orchestrator, which can be considered as producer.

Considerations for distributed architecture:

  • The project is using quantum to schedule jobs, it can use mnesia to store the schedule jobs but keep in mind the network partition problems. This application uses Cachex already so consider implementing a backend for it (https://hexdocs.pm/quantum/configuration.html#persistent-storage). To solve this problem we can have a producer node that will be handling the schedule jobs and when the node taking the role starts just puts all the scheduled jobs in a cache, so everytime the producer node dies the new one will regenerate the state.
  • Oban might need extra configurations for distributed systems, need to check

smokex's People

Contributors

agtraveso avatar dependabot[bot] avatar qgadrian avatar

Watchers

 avatar  avatar  avatar

smokex's Issues

Show currently running steps

Show the current step being executed in the execution detail view as "ongoing" (right now is only success or fail, so the ongoing request is not shown yet)

Improve how schedule jobs are handled

Improve the scheduled jobs management, by either persisting them or finding a better solution than using a start_phase. Tight now they are being loaded in memory when the instance starts.

This is low prio until there is more workload until the project becomes a distributed application and this has real repercussions.

Better cron/scheduling experience

Right now is a bit tricky & pain in the ass having to remember which cron sentences you want.

Add a select option with predefine cron sentences to make it easier to use (including a custom option that will show the input to be edited)

Simplify worker/scheduler architecture

Simplify worker/scheduler architecture, executions don't need to be created on schedule.

See the architecture diagram, it is pretty obvious what can be improved.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.