Git Product home page Git Product logo

qelog's Introduction

Qelog

Build&Test Go Report Card Go Reference codecov

Qelog is a small, low cost and light operation and maintenance of the log service. The purpose of its birth is to solve the problem of small and medium-sized team multi-service group log and alarm.

It has been running stably for 2+ years and has been rolling over 100+TB of data.

中文文档

admin manager example address username:admin passwd:111111

Log System Features:

Client

  • qezap is the Go version Client of the project. Wrap Uber-zap ,Extension for local and remote implementations of WriteSyncer.
  • Local extension: log cutting, compression, retention period, dynamic switching level, etc.
  • Remote expansion: The buffer is packed and compressed for transmission log data, and the error backup is retried to ensure that the data is not lost. The GRPC(default) HTTP protocol is supported, and the I/O control ensures the bandwidth usage is controllable and the memory usage is low.

Log receiver server

  • The Receiver process can be expanded horizontally to ensure high availability and high performance.
  • Configure multiple storage instances on the Receiver to improve the storage capacity and write performance of the cluster.
  • Alarm module detects alarm rules for each log. The hit can be delivered according to the rules and different alarm methods. Currently supported DingTalk | Telegram
  • Implement data fragmentation storage rules, support automatic capacity management, monitoring and early warning. Store separate instances of extensions without bottlenecks due to middleware.
  • Log statistics, level distribution, and trend report.

Log manager server

  • Based on the vue-element-admin modification,support rich query dimensions, such as level, keyword, TraceId, ClientIP, multi-level conditions, etc. Due to cost and performance issues ** Full-text indexing ** is currently not supported
  • Friendly operation interaction, simple configuration, efficient content display, almost can be done out of the box quick used.
  • Cluster capacity statistics query, manual intervention, and other functions.

Performance test tool

  • /tools Includes benchmark and memory footprint tests.
  • Different operating environments and configurations have different performance. Therefore, you can conduct a comprehensive analysis to preliminarily check whether the service meets the requirements of scenarios.

Design drawing

Design drawing

Usage

Qezap Client import your project

go get -u github.com/bbdshow/qelog/qezap

Client use example

Quick Deploy

Docker Deploy
git clone https://github.com/bbdshow/qelog.git
cd qelog
# custom you config
vim config.docker.toml
# build docker image
make image
# docker-compose start container
docker-compose up -d
Machine Deploy
git clone https://github.com/bbdshow/qelog.git
cd qelog

# go build, output ./bin
make
cd ./bin
vim configs/config.toml
cd ./admin
# admin suggest single server, because have background task
nohup ./qelog_admin -f ../configs/config.toml >> nohup.out 2>&1  &
nohup ./qelog_receiver -f ../configs/config.toml >> nohup.out 2>&1  &

Thank you for your support. If it is useful to you, I hope the Star can support you. If you have any questions, please Issues, keep updating and solve problems.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.