Git Product home page Git Product logo

marciopocebon / ade Goto Github PK

View Code? Open in Web Editor NEW

This project forked from openmainframeproject/ade

0.0 0.0 0.0 56.8 MB

ADE detects anomalous time slices and messages in Linux logs (either RFC3164 or RFC5424 format) using statistical learning.

Home Page: https://www.openmainframeproject.org/projects/anomaly-detection-engine-for-linux-logs-ade

License: GNU General Public License v3.0

XSLT 1.09% CSS 0.06% Shell 2.44% Java 96.33% HTML 0.05% TeX 0.04%

ade's Introduction

Anomaly Detection Engine for Linux Logs (ADE)

ADE can process a large numbers of logs from a large number of Linux systems to create a compact summary of those logs. The summary identifies and consolidates similar text strings into a single message example and assigns it a key (message id).  The summary determines if  the message id are being issued when expected, are being issued at the expected rate during a time slice, and how often during the day are the message or a similar message (same message id) issued.

You can use those results to examine

  • A set of logs to find anomalies which may be helpful when attempting to find the root cause of a problem or incident
  • The currently generated logs to find anomalies which may be helpful when attempting to find the cause of an on-going problem or incident

Please see http://openmainframeproject.github.io/ade/ for documentation on ADE.

Releases

Saw Kill 1.0.4

  • Fix problems with timezone in Junit test
  • Fix problems with year of test data in function test

Fall Kill 1.0.3

CII Best Practices

  • Support for Core Infrastructure Initiative
    • add travis-ci build for every pull request
    • analyze every pull request with Sonarqube
    • store results of Sonarqube analysis at Sonarqube.com
  • Add sample to mask sensitive data within Linux logs to allow sharing of logs
  • Fix problem with train_test.sh
  • Fix additional problems identified by Sonarqube

Poesten Kill 1.0.2

  • Support for changing analytics
    • command to check syntax of model (flowlayout.xml file)
    • command to print out statistical information contained within model file (.bin file) to text file
    • command to print out version of code and data base
  • Multiple SonarQube(TM) issues fixed
  • Fix to problem with regression test
  • Wiki article "Example of reading ADE data into R objects"

Esopus Creek 1.0.1

  • Support for MariaDB(TM)
  • Verify script - determine if sufficient messages are available to create a valid model
  • Multiple SonarQube(TM) issues fixed
  • Wiki article "Hints on how to update XSLT - tailor the output shown in a browser to problem"

Initial release 1.0.0

  • Parsing of Linux Logs in RFC5424 and RFC3164 format
  • Splitting logs into time slices
  • Handling wrapper messages
  • Statistical analysis of logs
  • Creates output

ade's People

Contributors

aymandf avatar ayush-1506 avatar caffreyj avatar caffreyjm avatar cbrooker27 avatar davidohana avatar jmertic avatar nealef avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.