Git Product home page Git Product logo

munin-plugins-gitlab's Introduction

GitLab Plugins for Munin

This is a collection of plugins for monitoring your GitLab instance with Munin.

These plugins are developed for and tested with GitLab versions >= 8.0.0. Maybe some plugins may work with older GitLab instances, but there's no support for occuring bugs or errors.

Setup

  1. Install dependencies depending on your distribution:
  • Ubuntu
    sudo apt-get install python-psycopg2
    
  1. Clone this repository to your GitLab server. Create a new file called gitlab in /etc/munin/plugin-conf.d and copy+paste the following lines:
[gitlab_*]
user git
#env.gitlab_dir /var/opt/gitlab    # optional, defaults to GitLab omnibus package setup directory
 
## using a PostgreSQL database
#env.db_engine postgresql          # optional, defaults to postgres, valid values: postgresql. mysql
#env.db_dsn host=/var/opt/gitlab/postgresql user=gitlab dbname=gitlabhq_production  # optional, defaults to GitLab omnibus database
#env.db_pg_search_path gitlab     # optional, set search_path before executing any query. Useful if not using GitLab omnibus package
  
## using a MySQL database
#env.db_engine mysql
#env.db_dsn host=localhost user=gitlab db=gitlabhq_production
  
[gitlab_redis_*]
user gitlab-redis
#env.redis_socket /var/opt/gitlab/redis/redis.socket  # optional, defaults to GitLab omnibus redis instance
  
[gitlab_total_registry_size]
user registry
  1. Change your directory to /etc/munin/plugins. Create symlinks for each plugin (ln -s) which you want to activate. Please take a look at the plugin specific documentation.

Plugins

Plugin configuration

All plugins will use the default omnibus gitlab setup configuration. You can customize the behaviour with the following parameters:

Further Monitoring

bundled nginx

If you want to monitor the bundled nginx instance you have to enable the status module. Supposed you have no other nginx instance running, create a file /etc/nginx/conf.d/status.conf and add the following lines:

server  {
    listen *:80;
    listen [::]:80;
    server_name localhost;
    location /nginx_status {
        stub_status on;
        access_log off;
        allow 127.0.0.1;
        allow ::1;
        deny all;
    }
}

In /etc/gitlab/gitlab.rb update your configuration:

nginx['custom_nginx_config'] = "include /etc/nginx/conf.d/*.conf;"

After gitlab-ctl reconfigure and gitlab-ctl restart you should be able to use the default nginx plugins for munin to monitor your gitlab nginx instance. You can activate them via:

ln -s /usr/share/munin/plugins/nginx_request /etc/munin/plugins
ln -s /usr/share/munin/plugins/nginx_status /etc/munin/plugins
/etc/init.d/munin-node restart

munin-plugins-gitlab's People

Contributors

ap-wtioit avatar boudekerk avatar matthiaslohr avatar saily avatar tnir avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

munin-plugins-gitlab's Issues

gitlab_total_issues_*: schema "gitlab" does not exist

I guess it's only me being stupid, but running the gitlab_total_issues_* plugins, I get the following error:

~# munin-run gitlab_total_issues_open
Traceback (most recent call last):
  File "/etc/munin/plugins/gitlab_total_issues_open", line 19, in <module>
    cursor.execute("SELECT COUNT(*) FROM gitlab.issues WHERE state = 'opened'")
psycopg2.ProgrammingError: schema "gitlab" does not exist
LINE 1: SELECT COUNT(*) FROM gitlab.issues WHERE state = 'opened'

But when trying to do the same thing, your plugin does, manually, it's working just fine:

~# su git
~# embedded/bin/psql --username=gitlab -h /var/opt/gitlab/postgresql gitlabhq_production
psql (9.2.10)
Type "help" for help.

gitlabhq_production=> SELECT COUNT(*) FROM gitlab.issues WHERE state = 'opened'
gitlabhq_production->

There's no output, 'cos there're no issues, yet - but at least I'm not presented an error, when doing it manually. Database and table do exist and user gitlab has access.

Running gitlab v8.1.2-ce.0 on Debian Jessie. I used their install-script and repo and didn't change any of their sql configuration. (Did add Mattermost, SSL and some other stuff, though..)

Hope, you can help - thanks in advance!

Add tracking of uploaded artifact storage

Since gitlab-ci-multi-runner v0.7.x you can upload artifacts to gitlab-workhorse which is stored on the filesystem. Administrators may want to track usage/load of this folder to put in more storage or a different file system to handle more load.

what do you think about adding a plugin (or an additional graph to repo-size) which tracks artifacts size as well?

SQL syntax error in gitlab_users_active when using postgres

Error output from gitlab_users_active:
2018/11/15-13:00:25 [2769]      Traceback (most recent call last):
2018/11/15-13:00:25 [2769]        File "/etc/munin/plugins/gitlab_users_active", line 32, in <module>
2018/11/15-13:00:25 [2769]          (day,)
2018/11/15-13:00:25 [2769]      psycopg2.ProgrammingError: syntax error at or near "1"
2018/11/15-13:00:25 [2769]      LINE 1: ... WHERE last_sign_in_at > CURRENT_TIMESTAMP - INTERVAL 1 day 

Changing

"WHERE last_sign_in_at > CURRENT_TIMESTAMP - INTERVAL %s day ",

to
"WHERE last_sign_in_at > CURRENT_TIMESTAMP - INTERVAL '%s day' ",
fixes it for postgres.
Unfortunately it does not work with mysql anymore. So some condition is needed.

gitlab_total_repo_size not working after fs check failed

gitlab_total_repo_size does not work after fs check failed.

The script expects only /var/opt/gitlab/git-data/repositories/namespace/repo, that is, there is no file in /var/opt/gitlab/git-data/repositories/.

How to reproduce

  1. Create a file in the git repo root with touch /var/opt/gitlab/git-data/repositories/fs_shards_checkYYYYMMDD-1234-abc123+deleted
  2. Run gitlab_total_repo_size and get an error as follows:
munin-run gitlab_total_repo_size
[Errno 20] Not a directory: '/var/opt/gitlab/git-data/repositories/fs_shards_checkYYYYMMDD-1234-abc123+deleted'

The root reason of this would be the same as #18 (gitlab_total_repo_count).

N.B. The directory structure in this issue is in case of omnibus-gitlab.

More precise Wiki repo count

While Wiki repo count is now based on the existing *.wiki.git directory, it's better to check if there are effective objects.

The main repo count is out of scope.

ToDos by users

Measuring pending ToDos by users is important for task management. Let us add ToDos by users.

Support DAU/WAU/MAU etc.

Daily/weekly/monthly active users will help GitLab Administrators to understand their retention rates.

On GitLab, it looks no raw data for the retention rate though users.last_sign_in_at may be helpful for this kind of metrics.

gitlab_total_repo_count not working after fs check failed

gitlab_total_repo_count does not work after fs check failed.

The script expects only /var/opt/gitlab/git-data/repositories/namespace/repo, that is, there is no file in /var/opt/gitlab/git-data/repositories/.

How to reproduce

  1. Create a file in the git repo root with touch /var/opt/gitlab/git-data/repositories/fs_shards_checkYYYYMMDD-1234-abc123+deleted
  2. Run gitlab_total_repo_count and get an error as follows:
munin-run gitlab_total_repo_count
[Errno 20] Not a directory: '/var/opt/gitlab/git-data/repositories/fs_shards_checkYYYYMMDD-1234-abc123+deleted'

N.B. The directory structure in this issue is in case of omnibus-gitlab.

Are external runners supported?

I installed this plugin on my gitlab server which also has a runner installed and the graphs for the runner work fine. Now I installed the plugin (only linking gitlab_ci_builds and gitlab_ci_builds_current to /etc/munin/plugins) on an external runner instance that is connected to my gitlab instance but I can not get the plugin to work.

So: Are external runners supported by this plugin?

Cannot find local modules

Running the scripts out of the box, I was having problems finding the local modules with Python 3.8:
ModuleNotFoundError: No module named 'dirsize'

Adding . in front of modules in init.py solvd it:

from .dirsize import dirsize
from .gitlab import get_gitlab_instance
from .munin import munin_safe_id

author = 'Matthias Lohr [email protected]'

Package dependencies

We should add a list of package dependencies for the main distributions.

E.g. for ubuntu, python-psycopg2 is needed.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.