View Code? Open in Web Editor
NEW
This project forked from saltycrane/live-log-analyzer
Real-time (Nginx, PHP, syslog, "mysqladmin extended") log file analysis and plotting using Python, mongoDB, Orbited, and flot
Home Page: http://www.saltycrane.com/lla/
JavaScript 64.59%
Python 35.41%
live-log-analyzer's Introduction
- Stores log file data from multiple remote hosts to MongoDB
- Creates processes that ssh to a remote host and run "tail --follow=name" on the log file
- Parses the log file using Regular Expressions
- Stores the data in MongoDB
- Queries MongoDB for interesting statistics e.g.:
- Requests/mintute
- Cache hit rate
- Count of HTTP 500 statuses
- Average upstream response time
- MySQL questions/second
- Creates real-time plots in a web browser
- MongoDB
- PyMongo
- Orbited
- Twisted
- stompservice
- flot
- a few more in pip-requirements.txt
- Copy settings_template.py to settings.py and edit it
- Start mongod
- Start orbited
- In first terminal, run "sourceexecutive.py"
- In second terminal, run "analyzerexecutive.py"
- Leaves "mysqladmin extended" processes running on remote host
- The bulk of this is set up for my custom Nginx log file (but it is easy to
write your own custom sources, parsers, and analyzers.
live-log-analyzer's People
Contributors
Stargazers
Watchers
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent