Git Product home page Git Product logo

logs-collector's Introduction

logs-collector



the main idea

Server runs "jobs" in a background. Each job is attached to a specific container and reads the logs until the stream is valid or job is stopped by the user. Each job has its status: following, paused, completed or error.

The user can manage the jobs using CLI.

The read logs are saved into MongoDB but it can be replaced by another custom storage provider. The jobs are preserved in a storage provider too.

On the unintended reboot of the server - the jobs which were in status "following" are recovered.

When recovering a job - it reads the logs starting from the place (timestamp) where it stopped last time.


server

prerequisites

  • docker installation with exposed TCP connection or Unix-socket

configuration

  • review and adjust environmental variables in .env

running

cd ./server
docker-compose up --build

API description

  • get the list of the available containers from the remote Docker Engine:
GET /available-containers
  • start gathering the logs for multiple containers (by ids/names):
POST /logs/start
  • pause collection the logs for multiple containers (by ids/names):
POST /logs/pause
  • remove the collected logs for multiple containers (by ids/names):
DELETE /logs
  • get all running jobs for multiple containers:
GET /logs
  • tail a container for logs (by id/name):
POST /logs/tail

cli

synopsis

logs-collector --list-available
logs-collector --list
logs-collector start  [--id] [--name]
logs-collector stop   [--id] [--name]
logs-collector delete [--id] [--name]
logs-collector tail   [--id] [--name]

room for the IMPROVEMENTS

  1. add ability to add containers by labels/tags
  2. provide the ability to tail multiple containers at once by client
  3. tail with --follow from client to server
  4. add Authorization
  5. save error message to DB for job - if it occurs
  6. unit tests
  7. write the logs of the app itself to the filesystem/db
  8. CLI in interactive mode

logs-collector's People

Watchers

Vasily avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.