Git Product home page Git Product logo

gofer's Introduction

Gofer: Run short-lived jobs easily.

gofer

godoc for clintjedwards/gofer docs site project status

Gofer is an opinionated, streamlined automation engine designed for the cloud-native era. It's basically remote code execution as a platform.

It specializes in executing your custom scripts in a containerized environment, making it versatile for both developers and operations teams. Deploy Gofer effortlessly as a single static binary, and manage it using expressive, declarative configurations written in real programming languages. Once set up, Gofer takes care of scheduling and running your automation tasks—be it on Nomad, Kubernetes, or even Local Docker.

Its primary function is to execute short-term jobs like code linting, build automation, testing, port scanning, ETL operations, or any task you can containerize and trigger based on events.

Features:

  • Simple Deployment: Install Gofer effortlessly with a single static binary and manage it through its intuitive command-line interface.
  • Language Flexibility: Craft your pipelines in programming languages you're already comfortable with, such as Go or Rust—no more wrestling with unfamiliar YAML.
  • Local Testing: Validate and run your pipelines locally, eliminating the guesswork of "commit and see" testing.
  • Extensible Architecture: Easily extend Gofer's capabilities by writing your own plugins, backends, and more, in any language via gRPC.
  • Built-In Storage: Comes with an integrated Object and Secret store for your convenience.
  • DAG Support: Harness the power of Directed Acyclic Graphs (DAGs) for complex workflow automation.
  • Robust Reliability: Automatic versioning, Blue/Green deployments, and canary releases ensure the stability and dependability of your pipelines.

Demo:

Documentation & Getting Started

If you want to fully dive into Gofer, check out the documentation site!

Install

Extended installation information is available through the documentation site.

Download a specific release:

You can view and download releases by version here.

Download the latest release:

  • Linux: wget https://github.com/clintjedwards/gofer/releases/latest/download/gofer

Build from source:

You'll need to install protoc and its associated golang/grpc modules first

  1. git clone https://github.com/clintjedwards/gofer && cd gofer
  2. make build OUTPUT=/tmp/gofer

The Gofer binary comes with a CLI to manage the server as well as act as a client.

Dev Setup

Gofer is setup such that the base run mode is the development mode. So simply running the binary without any additional flags allows easy authless development.

You'll need to install the following first:

To run Gofer dev mode:

To build protocol buffers:

Run from the Makefile

Gofer uses flags, env vars, and files to manage configuration (in order of most important). The Makefile already includes all the commands and flags you need to run in dev mode by simply running make run.

In case you want to run without the make file simply run:

export GOFER_LOG_LEVEL=debug
go build -o /tmp/$gofer
/tmp/gofer service start --dev-mode

Editing Protobufs

Gofer uses grpc and protobufs to communicate with both plugins and provide an external API. These protobuf files are located in /proto. To compile new protobufs once the original .proto files have changed you can use the make build-protos command.

Editing Documentation

Documentation is done with mdbook.

To install:

cargo install mdbook
cargo install mdbook-linkcheck

Once you have mdbook you can simply run make run-docs to give you an auto-reloading dev version of the documentation in a browser.

Regenerating Demo Gif

The Gif on the README page uses vhs; a very handy tool that allows you to write a configuration file which will pop out a gif on the other side.

In order to do this VHS has to run the commands so we must start the server first before we regenerate the gif.

rm -rf /tmp/gofer* # Start with a fresh database
make run # Start the server in dev mode
cd documentation/src/assets
vhs < demo.tape # this will start running commands against the server and output the gif as demo.gif.

Authors

This software is provided as-is. It's a hobby project, done in my free time, and I don't get paid for doing it.

gofer's People

Contributors

clintjedwards avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

isgasho

gofer's Issues

Cron trigger shutdown results in nil channel close

Jan 07 10:02:03 gofer gofer[135002]: panic: close of nil channel
Jan 07 10:02:03 gofer gofer[135002]: goroutine 31 [running]:
Jan 07 10:02:03 gofer gofer[135002]: main.(*trigger).Shutdown(0xc00026bb60, 0x40ac25)
Jan 07 10:02:03 gofer gofer[135002]: 	/files/triggers/cron/main.go:98 +0x1c

Triggers not correctly inheriting main process log level

triggers are getting an empty string when main process is set to info mode via the configuration file.

Jan 06 02:44:06 gofer gofer[66423]: {"level":"error","trigger":"interval","time":1641437046,"caller":"/files/sdk/sdk.go:170","message":"loglevel not recognized; defaulting to debug"}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.