Git Product home page Git Product logo

cerberus's Introduction

cerberus CircleCI

Automatic alert system for telemetry histograms.

Cerberus detects changes in the distribution of histograms (histogram regressions), and posts alerts for these to the medusa alert service.

Cerberus is also responsible for sending out email reminders for histograms that are expiring in upcoming versions.

Development and deployment

To start hacking on your local machine:

vagrant up
vagrant ssh

To deploy cerberus on AWS:

ansible-playbook ansible/deploy.yml -i ansible/inventory [ --private-key /path/to/mykey.pem ]

Note that the deployment requires medusa to be deployed.

Code Overview

run.sh is a shell script that will set up all the dependencies, then do a full run of the Cerberus detectors. These are the components used in the script:

  • exporter/export.js downloads the histogram evolutions from the v4 aggregates API.
  • The code for detecting regressions lives in alert/alert.py. This file is intended to be run as a script.
    • This is a script that reads histogram definitions from Histograms.json (which is downloaded automatically by run.sh).
    • Detected regressions are written out to dashboard/regressions.json.
  • alert/post.py reads in new regressions from dashboard/regressions.json, and posts alerts to Medusa with this data.
    • Posting new alerts to Medusa is done using alert/poster.py.
    • By default, the Medusa server URL is set to localhost:8080 - it expects to be on the same machine as the Medusa server. This can be changed by editing alert/post.py.
  • alert/expiring.py is the histogram expiry detector - it notifies people via email when histograms are expiring soon.
    • Some configurable number of days before the versions where histograms are set to expire, it sends out emails using Amazon SES to watchers, and the dev-telemetry-alerts mailing list.
  • dashboard/ contains a debugging/development dashboard for viewing detected regressions. It is intended to be hosted via GitHub Pages or a similar static hosting solution.

cerberus's People

Contributors

chutten avatar dexterp37 avatar fbertsch avatar jonasfj avatar mozilla-github-standards avatar mreid-moz avatar uberi avatar vitillo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cerberus's Issues

CODE_OF_CONDUCT.md file missing

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

  1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
  2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please see Mozilla-GitHub-Standards or email [email protected].

(Message COC001)

Add direct link to evolution view to alerts

I usually end up looking at the evolution view dashboard to check out any regressions.
Can we get a direct link to it in the mail?
Currently getting there requires a lot of steps: click link in mail -> find and click link to get to distribution view -> click evolution view -> wait for loading...

Support regressions for histogram relationships

Consider number of errors / number of migrations. If this stays constant, everything is okay. However, number of errors can go up (with no regression), as long as number of migrations go up as well. Similarly, number of migrations might regress down.

We can't necessarily look at them separately, because that could lead to false positives and false negatives (if they moved in separate directions).

We could solve this in a few place:

  1. A new ping at the browser
  2. In the aggregates dataset
  3. Here

Build is failing

Details here: https://circleci.com/gh/mozilla/cerberus/41

Error:

Traceback (most recent call last):
  File "alert/expiring.py", line 279, in <module>
    main()
  File "alert/expiring.py", line 244, in main
    if sys.argv[1] == "test": run_tests()
  File "alert/expiring.py", line 177, in run_tests
    assert len(get_release_dates()) > 4 # this function should return several versions if the table is formatted correctly
  File "alert/expiring.py", line 98, in get_release_dates
    result.update(get_version_table_dates(table))
  File "alert/expiring.py", line 50, in get_version_table_dates
    if "Firefox" not in field.string:
TypeError: argument of type 'NoneType' is not iterable
Exited with code 1

Release Dates Scraping is Failing

This can be verified by running python alert/expiring.py test. Result:

Traceback (most recent call last):
  File "alert/expiring.py", line 279, in <module>
    main()
  File "alert/expiring.py", line 244, in main
    if sys.argv[1] == "test": run_tests()
  File "alert/expiring.py", line 177, in run_tests
    assert len(get_release_dates()) > 4 # this function should return several versions if the table is formatted correctly
  File "alert/expiring.py", line 93, in get_release_dates
    table = soup.find(id="Future_branch_dates").find_parent("h2").find_next_sibling("table")
AttributeError: 'NoneType' object has no attribute 'find_parent'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.