Git Product home page Git Product logo

opsdis / icinga2-exporter Goto Github PK

View Code? Open in Web Editor NEW
16.0 3.0 14.0 86 KB

The Icinga2-exporter utilize Icinga2 API to fetch service based performance data and publish it in a way that lets prometheus scrape the performance data as metrics.

Home Page: https://www.opsdis.com

License: GNU General Public License v3.0

Python 97.91% Dockerfile 2.09%
prometheus icinga2 monitor-promdiscovery grafana

icinga2-exporter's Introduction

PyPI version

icinga2-exporter

Overview

The icinga2-exporter utilizes the Icinga 2 REST API to fetch service based performance data and publish it in a way that lets Prometheus scrape the performance data as metrics.

The service is based on Quart. Quart's is compatible with Flask but based on Asyncio.

Benefits:

  • Enable advanced queries and aggregation on timeseries
  • Prometheus based alerting rules
  • Grafana graphing
  • Utilize investments with Icinga 2 of collecting metrics

Metrics naming

Metric names

Metrics that is scraped with the icinga2-exporter will have the following name structure:

icinga2_<check_command>_<perfname>_<unit>

The icinga2 prefix can be changed by the configuration Unit is only added if it exists on performance data

Example from check command check_ping will result in two metrics:

icinga2_ping_rta_seconds
icinga2_ping_pl_ratio

Metric labels

The icinga2-exporter adds a number of labels to each metrics:

  • hostname - is the host_name in icinga2
  • service - is the display_name in icinga2

Optional icinga2-exporter can be configured to add specific custom variables configured on the host.

Note:

Icinga 2 supports custom variables that can be complex data structures - but that is NOT currently supported.

Labels created from custom variables are all transformed to lowercase.

Performance metrics name to labels

As described above the default naming of the Prometheus name is:

icinga2_<check_command>_<perfname>_<unit>

For some checks this does not work well like for the disk check command where the perfname are the unique mount paths. For checks like that the where the perfname is defined depending on environment you can change so the perfname instead becomes a label. This is defined in the configuration like:

  perfnametolabel:
      # The command name
      disk:
        # the label name to be used
        label_name: mount

So if the check command is disk the Prometheus metrics will have a format like, depending on other custom variables :

icinga2_disk_bytes{hostname="icinga2", service="disk", os="Docker", mount="/var/lib/icinga2"} 48356130816.0

If we did not make this translation we would got the following:

icinga2_disk_slashvarslashibslashicinga2_bytes{hostname="icinga2", service="disk", os="Docker"} 48356130816.0

This would not be good from a cardinality point of view.

Scrape duration

The scrape duration is a metrics that is reported for all targets.

icinga2_scrape_duration_seconds{hostname="<target>", server="<icinga2_server_url>"} 0.160983

Scrape response

When requests are made to the exporter the following responses are possible:

  • A target that exists - return all metrics and http status 200
  • A target does not exists - return no metrics, empty response, and http status 200
  • The export fail to scrape metrics from icinga2 - return empty response and http status 500

In the last scenario the exporter will log the reason for the failed scrape. A failed scrape can have multiple reasons, for example:

  • The icinga2 server is not responding
  • Not having valid credentials
  • Request to icinga2 timeout

Configuration

icinga2-exporter

The icinga2-exporter is configured by a yaml based configuration file.

Example:

# Port can be overridden by using -p if running development quart
#port: 9638

icinga2:
  # The url to the icinga2 server
  url: https://127.0.0.1:5665
  user: root
  passwd: cf593406ffcfd2ef
  # All prometheus metrics will be prefixed with this string
  metric_prefix: icinga2
  # Example of custom vars that should be added as labels and how to be translated
  host_custom_vars:
    # Specify which custom_vars to extract from hosts in icinga2
    - env:
        # Name of the label in Prometheus
        label_name: environment
    - site:
        label_name: dc

  # This section enable that for specific check commands the perfdata metrics name will not be part of the
  # prometheus metrics name, instead moved to a label
  # E.g for the disk command the perfdata name will be set to the label disk like:
  # icinga2_disk_bytes{hostname="icinga2", service="disk", os="Docker", disk="/var/log/icinga2"}
  perfnametolabel:
      # The command name
      disk:
        # the label name to be used
        label: mount

logger:
  # Path and name for the log file. If not set send to stdout
  logfile: /var/tmp/icinga2-exporter.log
  # Log level
  level: INFO

When running with gunicorn the port is selected by gunicorn

Logging

The log stream is configure in the above config. If logfile is not set the logs will go to stdout.

Logs are formatted as json so its easy to store logs in log servers like Loki and Elasticsearch.

Prometheus configuration

Prometheus can be used with static configuration or with dynamic file discovery using the project monitor-promdiscovery.

Please add the job to the scrape_configs in prometheus.yml.

The target is the host_name configured in icinga2.

Static config

scrape_configs:
  - job_name: 'icinga2'
    metrics_path: /metrics
    static_configs:
      - targets:
        - icinga2
        - google.se
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: localhost:9638

File discovery config for usage with monitor-promdiscovery

scrape_configs:
  - job_name: 'icinga2'
    scrape_interval: 1m
    metrics_path: /metrics
    file_sd_configs:
    - files:
      - 'sd/icinga2_sd.yml'
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: localhost:9638

Installing

  1. Check out the git repo.

  2. Install dependency

    pip install -r requirements.txt

  3. Build a distribution

    python setup.py sdist

  4. Install locally

    pip install dist/icinga2-exporter-X.Y.Z.tar.gz

Running

Development

Run the icinga2-exporter with the built-in Quart webserver:

python -m  icinga2_exporter -f config.yml

To see all options:

python -m  icinga2_exporter -h

Production with hypercorn as ASGI continer

Hypercorn is the recommended ASGI container for Quart. Install hypercorn with:

pip install hypercorn

Running with default config.yml. The default location is current directory

hypercorn "icinga2_exporter.main:create_app()

Set the path to the configuration file.

hypercorn "icinga2_exporter.main:create_app('/etc/icinga2-exporter/config.yml')"

Port 8000 is the default port for hypercorn. For more configuration for hypercorn please visit https://pgjones.gitlab.io/hypercorn/index.html

Test the connection

Check if exporter is working.

curl -s http://localhost:9638/health

Get metrics for a host where target is a host, host_name that exists in icinga2

curl -s http://localhost:9638/metrics?target=google.se

System requirements

Python 3.

For required packages please review requirements.txt.

icinga2-exporter's People

Contributors

antiarchitect avatar delvers avatar dependabot[bot] avatar guidoffm avatar thenodon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

icinga2-exporter's Issues

Add project to Helm chart repository

Good day.
Can this project be included in helm chart repository? Since some dependencies are now missing in Debian repository, we are not allowed to install it. It would be great if you could prepare this feature.
Thanks

SSLV3_ALERT_HANDSHAKE_FAILURE

Here is my Dockerfile:

FROM python:3.10.6-bullseye
WORKDIR /app/icinga2-exporter
RUN apt-get update \
    && apt-get install -y --no-install-recommends ca-certificates curl openssl \
    && curl -sqL "https://github.com/opsdis/icinga2-exporter/archive/refs/tags/v1.3.0.tar.gz" | tar --strip-components=1 -xzf - -C . \
    && pip install -r requirements.txt \
    && python setup.py sdist \
    && pip install dist/icinga2-exporter-0.0.1.tar.gz \
    && rm -rf dist \
    && pip install gunicorn uvicorn pyOpenSSL

CMD gunicorn --access-logfile /dev/stdout -w 4 -k uvicorn.workers.UvicornWorker "wsgi:create_app('/app/icinga2-exporter/config.yml')"

Here is the config.yml:

    icinga2:
      url: https://127.0.0.1:5665
      user: icingaweb2
      passwd: xxx
      metric_prefix: icinga2
      verify: false

This is what I'm getting after curl 127.0.0.1:8000/metrics:

{"timestamp": "2022-08-12T11:19:41.659402Z", "level": "WARNING", "name": "icinga2-exporter", "message": "Connection error", "target": null, "url": "http://127.0.0.1:8000/metrics", "remote_url": "https://127.0.0.1:5665", "err": "Cannot connect to host 127.0.0.1:5665 ssl:False [[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:997)]"}

Python 3.8: ValueError: Invalid format '(timestamp) (level) (name) (message)' for '%' style

icinga2-exporter does not work with Python 3.8.

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/src/app/icinga2_exporter/__main__.py", line 26, in <module>
    main.start()
  File "/usr/src/app/icinga2_exporter/main.py", line 58, in start
    log.configure_logger(configuration)
  File "/usr/src/app/icinga2_exporter/log.py", line 38, in configure_logger
    formatter = CustomJsonFormatter('(timestamp) (level) (name) (message)')
  File "/usr/local/lib/python3.8/site-packages/pythonjsonlogger/jsonlogger.py", line 112, in __init__
    logging.Formatter.__init__(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/logging/__init__.py", line 576, in __init__
    self._style.validate()
  File "/usr/local/lib/python3.8/logging/__init__.py", line 429, in validate
    raise ValueError("Invalid format '%s' for '%s' style" % (self._fmt, self.default_format[0]))
ValueError: Invalid format '(timestamp) (level) (name) (message)' for '%' style

Caused by changes in 3.8:
https://docs.python.org/3/library/logging.html#logging.Formatter

Changed in version 3.8: The validate parameter was added. Incorrect or mismatched style and fmt will raise a ValueError. For example: logging.Formatter('%(asctime)s - %(message)s', style='{').

Sugested resolution:
Update https://github.com/opsdis/icinga2-exporter/blob/master/icinga2_exporter/log.py#L38

Exception if host does not have custom properties

Thanks for the plugin! I tried adding my first test host and with a single ping service, no custom properties on either the Host or the Service, and I see this error:

icinga2-exporter    | [2020-07-10 09:59:10,491] ERROR in app: Exception on request GET /metrics                                                                                                              
icinga2-exporter    | Traceback (most recent call last):                                                                                                                                                     
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1451, in handle_request                                                                                             
icinga2-exporter    |     return await self.full_dispatch_request(request_context)                                                                                                                           
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1473, in full_dispatch_request                                                                                      
icinga2-exporter    |     result = await self.handle_user_exception(error)                                                                                                                                   
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 892, in handle_user_exception                                                                                       
icinga2-exporter    |     raise error                                                                                                                                                                        
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1471, in full_dispatch_request                                                                                      
icinga2-exporter    |     result = await self.dispatch_request(request_context)                                                                                                                              
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1519, in dispatch_request                                                                                           
icinga2-exporter    |     return await handler(**request_.view_args)                                                                                                                                         
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/icinga2_exporter/proxy.py", line 49, in get_ametrics                                                                                    
icinga2-exporter    |     await asyncio.get_event_loop().create_task(monitor_data.get_perfdata())                                                                                                            
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/icinga2_exporter/perfdata.py", line 63, in get_perfdata                                                                                 
icinga2-exporter    |     labels.update(Perfdata.get_host_custom_vars(serivce_attrs))                                                                                                                        
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/icinga2_exporter/perfdata.py", line 141, in get_host_custom_vars                                                                        
icinga2-exporter    |     for custom_vars_key, custom_vars_value in serivce_attrs['joins']['host']['vars'].items():                                                                                          
icinga2-exporter    | AttributeError: 'NoneType' object has no attribute 'items'   

And the webserver returns a 500 error.

nssl.SSLError

When running as in the example config, I get SSL-errors:

{"timestamp": "2021-07-23T12:19:50.527116Z", "level": "INFO", "name": "icinga2-exporter", "message": "http://localhost:9638/metrics?target=icinga2"}
{"timestamp": "2021-07-23T12:19:50.527286Z", "level": "INFO", "name": "icinga2-exporter", "message": "Collect metrics", "target": "icinga2"}
[2021-07-23 12:19:50,546] 127.0.0.1:56630 GET /metrics 1.1 200 0 19830
{"timestamp": "2021-07-23T12:19:50.547602Z", "level": "ERROR", "name": "asyncio", "message": "SSL error in data received\nprotocol: <asyncio.sslproto.SSLProtocol object at 0x7fd6592306a0>\ntransport: <_SelectorSocketTransport fd=9 read=polling write=<idle, bufsize=0>>", "exc_info": "Traceback (most recent call last):\n  File \"/usr/lib/python3.7/asyncio/sslproto.py\", line 526, in data_received\n    ssldata, appdata = self._sslpipe.feed_ssldata(data)\n  File \"/usr/lib/python3.7/asyncio/sslproto.py\", line 207, in feed_ssldata\n    self._sslobj.unwrap()\n  File \"/usr/lib/python3.7/ssl.py\", line 767, in unwrap\n    return self._sslobj.shutdown()\nssl.SSLError: [SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2609)"}

Seem like this bug, but not sure how to fix this in production systems.

Host meta data is missing tags

The output from the different host meta data is missing all host tags that should be based on the vars in the same way as for service metrics

Exception if service do not include performance data

Need to verify that the structure include performance_data
serivce_attrs['attrs']['last_check_result']['performance_data']
Exception is:
[2019-09-02 09:22:06,104] ERROR in app: Exception on /metrics [GET]
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
response = self.full_dispatch_request()
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
rv = self.dispatch_request()
File "/home/ubuntu/.local/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
return self.view_functionsrule.endpoint
File "/home/ubuntu/icinga2-exporter/icinga2_exporter/proxy.py", line 48, in get_metrics
monitor_data.get_perfdata()
File "/home/ubuntu/icinga2-exporter/icinga2_exporter/perfdata.py", line 63, in get_perfdata
for perf_string in serivce_attrs['attrs']['last_check_result']['performance_data']:
TypeError: 'NoneType' object is not iterable

Performance data from host check is missing

Looks like only performance data from services are fetched.

self.url_query_service_perfdata = self.host + '/v1/objects/services'

Need to export performance data for the host check as well. Available from API endpoint /v1/objects/hosts?host=host.example.com. Attrs should be the same as for the services apart from host_name which is name instead.

curl -s -k -H 'X-HTTP-Method-Override: GET' -H 'content-type: application/json' -H 'accept: application/json' -u user:password https://localhost:5665/v1/objects/hosts?host=host.example.com -d '{"attrs": ["__name", "display_name", "check_command", "last_check_result", "vars", "name", "downtime_depth", "acknowledgement", "max_check_attempts", "last_reachable", "state", "state_type"] }' | jq
{
  "results": [
    {
      "attrs": {
        "__name": "host.example.com",
        "acknowledgement": 0,
        "check_command": "hostalive",
        "display_name": "host.example.com",
        "downtime_depth": 0,
        "last_check_result": {
          "active": true,
          "check_source": "host.example.com",
          "command": [
            "/usr/lib/nagios/plugins/check_ping",
            "-H",
            "10.0.30.11",
            "-c",
            "5000,100%",
            "-w",
            "3000,80%"
          ],
          "execution_end": 1630593397.788697,
          "execution_start": 1630593393.700949,
          "exit_status": 0,
          "output": "PING OK - Packet loss = 0%, RTA = 0.47 ms",
          "performance_data": [
            "rta=0.469000ms;3000.000000;5000.000000;0.000000",
            "pl=0%;80;100;0"
          ],
          "schedule_end": 1630593397.788903,
          "schedule_start": 1630593393.7,
          "scheduling_source": "host.example.com",
          "state": 0,
          "ttl": 0,
          "type": "CheckResult",
          "vars_after": {
            "attempt": 1,
            "reachable": true,
            "state": 0,
            "state_type": 1
          },
          "vars_before": {
            "attempt": 1,
            "reachable": true,
            "state": 0,
            "state_type": 1
          }
        },
        "last_reachable": true,
        "max_check_attempts": 3,
        "name": "host.example.com",
        "state": 0,
        "state_type": 1,
        "vars": null
      },
      "joins": {},
      "meta": {},
      "name": "host.example.com",
      "type": "Host"
    }
  ]
}

Scrape all Performancedata

Hey,

is it possible to scrape all perfdata from all services at once ? or at least an easy way to scrape everything? I have many hosts and dont want to configure them in the prometheus configuration.

Another Question: Is it possible to exclude service vars ?

Thanks and kr
fl0wx

Internal Server Error for external Icinga2 targets

Hello,

I have Icinga2 performing checks on several hostnames with .com domains which is working fine. However, I cannot query the metrics from these hosts with the exporter as I am receiving 500 Internal Server Error in Prometheus or upon calling the metrics URL manually.
What I noticed is that the hosts have the format subdomain.domain.com and if I remove the subdomain or simply replace it with a non-existing one, I am able to get a result (icinga2_scrape_duration_seconds). Querying internal hosts with host.domain.local addresses is also working well.

Error in icinga2-exporter.log:

{"timestamp": "2022-03-29T17:55:08.543706Z", "level": "ERROR", "name": "quart.app", "message": "Exception on request GET /metrics", "exc_info": "Traceback (most recent call last):\n File "/usr/local/lib/python3.7/dist-packages/quart/app.py", line 1467, in handle_request\n return await self.full_dispatch_request(request_context)\n File "/usr/local/lib/python3.7/dist-packages/quart/app.py", line 1492, in full_dispatch_request\n result = await self.handle_user_exception(error)\n File "/usr/local/lib/python3.7/dist-packages/quart/app.py", line 968, in handle_user_exception\n raise error\n File "/usr/local/lib/python3.7/dist-packages/quart/app.py", line 1490, in full_dispatch_request\n result = await self.dispatch_request(request_context)\n File "/usr/local/lib/python3.7/dist-packages/quart/app.py", line 1536, in dispatch_request\n return await self.ensure_async(handler)(**request_.view_args)\n File "/usr/local/lib/python3.7/dist-packages/icinga2_exporter/proxy.py", line 55, in get_metrics\n await fetch_metadata_task\n File "/usr/local/lib/python3.7/dist-packages/icinga2_exporter/perfdata.py", line 133, in get_host_metrics\n labels.update(Perfdata.get_host_meta_custom_vars(host_attrs))\n File "/usr/local/lib/python3.7/dist-packages/icinga2_exporter/perfdata.py", line 384, in get_host_meta_custom_vars\n labels[custom_vars_key.lower()] = Perfdata.valid_prometheus_label_values(custom_vars_value)\n File "/usr/local/lib/python3.7/dist-packages/icinga2_exporter/perfdata.py", line 391, in valid_prometheus_label_values\n if '\\' in value:\nTypeError: argument of type 'int' is not iterable"}

Show permission error instead of json.decoder.JSONDecodeError

Hi, I get a JSONDecodeError. What am I doing wrong? I'm using the latest version from this repo and the example configuration.

[2021-07-23 11:36:08,761] ERROR in app: Exception on request GET /metrics
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/quart/app.py", line 1451, in handle_request
    return await self.full_dispatch_request(request_context)
  File "/usr/local/lib/python3.7/dist-packages/quart/app.py", line 1473, in full_dispatch_request
    result = await self.handle_user_exception(error)
  File "/usr/local/lib/python3.7/dist-packages/quart/app.py", line 892, in handle_user_exception
    raise error
  File "/usr/local/lib/python3.7/dist-packages/quart/app.py", line 1471, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/usr/local/lib/python3.7/dist-packages/quart/app.py", line 1519, in dispatch_request
    return await handler(**request_.view_args)
  File "/usr/local/lib/python3.7/dist-packages/icinga2_exporter/proxy.py", line 56, in get_ametrics
    await fetch_perfdata_task
  File "/usr/local/lib/python3.7/dist-packages/icinga2_exporter/perfdata.py", line 54, in get_perfdata
    data_json = await self.monitor.async_get_perfdata(self.query_hostname)
  File "/usr/local/lib/python3.7/dist-packages/icinga2_exporter/monitorconnection.py", line 162, in async_get_perfdata
    data_json = await self.async_post(self.url_query_service_perfdata, body)
  File "/usr/local/lib/python3.7/dist-packages/icinga2_exporter/monitorconnection.py", line 180, in async_post
    return json.loads(re)
  File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

host_custom_vars config is not optional

It's mentioned on your readme that host custom vars to labels config is an optional parameter:

Optional icinga2-exporter can be configured to add specific custom variables configured on the host.

However, when I omit this block from my config.yml (I'm trying to workaround issue #4) I get a separate exception:

icinga2-exporter    | {"timestamp": "2020-07-10T10:06:40.469733Z", "level": "INFO", "name": "icinga2-exporter", "message": "Collect metrics", "target": "172.16.11.62"}
icinga2-exporter    | [2020-07-10 10:06:40,469] ERROR in app: Exception on request GET /metrics
icinga2-exporter    | Traceback (most recent call last):
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1451, in handle_request
icinga2-exporter    |     return await self.full_dispatch_request(request_context)
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1473, in full_dispatch_request
icinga2-exporter    |     result = await self.handle_user_exception(error)
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 892, in handle_user_exception
icinga2-exporter    |     raise error
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1471, in full_dispatch_request
icinga2-exporter    |     result = await self.dispatch_request(request_context)
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/quart/app.py", line 1519, in dispatch_request
icinga2-exporter    |     return await handler(**request_.view_args)
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/icinga2_exporter/proxy.py", line 46, in get_ametrics
icinga2-exporter    |     monitor_data = Perfdata(monitorconnection.MonitorConfig(), target)
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/icinga2_exporter/perfdata.py", line 41, in __init__
icinga2-exporter    |     self.configured_labels = monitor.get_labels()
icinga2-exporter    |   File "/usr/local/lib/python3.7/site-packages/icinga2_exporter/monitorconnection.py", line 102, in get_labels
icinga2-exporter    |     for label in self.labels:
icinga2-exporter    | TypeError: 'NoneType' object is not iterable
icinga2-exporter    | [2020-07-10 10:06:40,470] 169.234.220.180:54954 GET /metrics 1.1 500 126 1600
icinga2             | 172.31.19.203 - - [10/Jul/2020:10:06:44 +0000] "GET /v2/health-check HTTP/1.1"

Illegal char in metric name

Illegal char not sanitized which leads to no scraping of the target.

curl -s localhost:9638/metrics?target=xxx | /usr/local/bin/promtool check metrics
error while linting: text format parsing error in line 2: expected float as value, got "'reserve_processor_usage'_ratio{hostname="xxx","

Fix:
perfdata.py
prometheus_key = prometheus_key.replace(''', '_')

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.