Git Product home page Git Product logo

son-monitor's Introduction

Build StatusJoin the chat at https://gitter.im/sonata-nfv/Lobby

Monitoring Framework

5GTango/Sonata monitoring system gathers and analyzes performance information from network service (NS) and Virtual Network Functions (VNF) and provides alarm notifications, based on rule definitions which are defined by the users. The architecture of the system is based on data exporters and a monitoring server. Data exporters send monitoring data from NS and respective VNF to monitoring server which collects, analyses and stores data and generates the appropriate notifications in case of rule violations. In general, monitoring server consists of a RESTful API interface, an alerting mechanism, a timeseries database and a set of convenient notification services, including a pub/sub based message bus, SMS, email, etc.

Installing / Getting started

Each micro service of the framework is executed in its own Docker container. Building steps are defined in a Dockerfile of each service

Build containers

docker build -f pushgatwway/Dockerfile -t registry.sonata-nfv.eu:5000/son-monitor-pushgateway .
docker build -f prometheus/Dockerfile -t registry.sonata-nfv.eu:5000/son-monitor-prometheus .
docker build -f alertmanager/Dockerfile -t registry.sonata-nfv.eu:5000/son-monitor-alertmanager .
docker build -f manager/Dockerfile -t registry.sonata-nfv.eu:5000/son-monitor-manager .
docker build -f snmpmng/Dockerfile -t registry.sonata-nfv.eu:5000/son-monitor-snmpmng .

Installation

docker run -d --name son-monitor-influxdb -p 8086:8086 registry.sonata-nfv.eu:5000/son-monitor-influxdb
docker run -d --name son-monitor-postgres -e POSTGRES_DB=dbname -e POSTGRES_USER=user -e POSTGRES_PASSWORD=pass -p 5433:5432 ntboes/postgres-uuid
docker run -d --name son-monitor-pushgateway -p 9091:9091 registry.sonata-nfv.eu:5000/son-monitor-pushgateway
docker run -d --name son-monitor-alertmanager -p 9093:9093 registry.sonata-nfv.eu:5000/son-monitor-alertmanager
docker run -d --name son-monitor-prometheus -p 9090:9090 -p 9089:9089 -e RABBIT_URL=<son-broker-ip>:5671 --add-host pushgateway:127.0.0.1 --add-host influx:127.0.0.1 registry.sonata-nfv.eu:5000/son-monitor-prometheus
docker run -d --name son-monitor-manager --add-host postgsql:127.0.0.1 --add-host prometheus:127.0.0.1 -p 8000:8000 registry.sonata-nfv.eu:5000/son-monitor-manager
docker run -d --name son-monitor-snmpmng -e POSTGS_PORT=<postgres_port> -e POSTGS_HOST=<postgres_ip> -e DB_USER_NAME=<user_name> -e DB_USER_PASS=<password> -e PROM_SRV=[\"<pushgateway_url>\"]  registry.sonata-nfv.eu:5000/son-monitor-snmpmng

Developing

SONATA's monitoring system is based on following services:

  1. Monitoring manager: is a Django/rest-framework server including a relational database (mysql,postgres ect) that relates each monitoring metric in Prometheus DB (timeseries) with Sonata's monitored entities like NS/VNFs, VMs and VIMs.

  2. Prometheus server: Prometheus is an open-source monitoring and alerting toolkit, which is bundled in a standalone server, independent of any strict requirements, like network storage or dependency on remote services.

  3. Prometheus pushgateway: Despite the fact that the default approach proposed by Prometheus is to retrieve the metrics data by performing http get requests to exporters (containers, vms etc), the utilization of Pushgateway might provide several advantages in Sonata implementation, like no need for exporters to implement a web socket in order to be reached from Prometheus, no need to reconfigure Prometheus each time a VNF is created or changes IP address etc. In that sense, Pushgateway functoinality is supported by the current implementation of Sonata monitoring framework.

  4. SNMP manager: Monitoring framework provides a mechanism (based on SNMP protocol) to collect metrics from VNFs which supports this fuctionality. The needed SNMP information (i.e. port, OIDs, metric names etc.) are defined within the VNF descriptor.

Built With

  • Django
  • django-filter
  • django-rest-multiple-models
  • django-rest-swagger
  • Prometheus.io

Submiting changes

To contribute to the development of the 5GTango/SONATA monitoring framwork you have to fork the repository, commit new code and create pull requests.

Versioning

The most up-to-date version is v5.0.

Tests

Unit tests are automatically executed during the building stage.

Api Reference

Documentation of the RESTful API of Monitoring Manager is provided by a Swagger UI from each instance of the Monitoring Manager and also in 5GTango documentation page.

Licensing

Monitoring framework is published under Apache 2.0 license. Please see the LICENSE file for more details.

Lead Developers

The following lead developers are responsible for this repository and have admin rights. They can, for example, merge pull requests.

  • Panos Karkazis (pkarkazis)
  • Panos Trakadas (trakadasp)

Feedback-Chanel

  • You may use the mailing list sonata-dev-list
  • You may use the GitHub issues to report bugs

son-monitor's People

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

son-monitor's Issues

[VnV standalone monitoring API ]

monitoring api points to a pre-int-vnv-bcn.5gtango.eu instead of our internal address

message:172.18.0.15 - - [21/Nov/2019:15:37:41 +0000] "GET /api/v3/monitoring/graphs/d-solo/sp_dash/sp?orgId=1&panelId=2&var-id=4fd837f6-5f49-11dc-a9ed-000ea629289b&var-entity=vm&var-env=pre-int-vnv-bcn.5gtango.eu%3A4fd837f6-5f49-11dc-a9ed-000ea629289b&theme=light&refresh=10s HTTP/1.0" 200 - 0.3487 container_name:tng-api-gtw host:VV_5GTango source_host:192.168.1.181 level:3 created:November 14th 2019, 18:07:24.852 type:logs version:1.1 command:bundle exec rackup -p 5000 --host 0.0.0.0 image_name:sonatanfv/tng-api-gtw:v5.0 @timestamp:November 21st 2019, 16:37:41.446 @Version:1

Support of string values for SNMP variables

In our VNFs we are returning some numeric SNMP values as STRING since it the format provided by default by the EXTEND-MIB variables.

ubuntu@wac-vnf:~$ snmpwalk -v 3 -l authPriv -u "quobis" -a sha -A "5gt4ng00" -x aes -X "5gt4ng00" "localhost" .1.3.6.1.4.1.8072.1.3.2.3.1.2.22.115.105.112.112.111.83.101.114.118.101.114.67.111.110.102.101.114.101.110.99.101.115
iso.3.6.1.4.1.8072.1.3.2.3.1.2.22.115.105.112.112.111.83.101.114.118.101.114.67.111.110.102.101.114.101.110.99.101.115 = STRING: "25"
ubuntu@wac-vnf:~$  snmpwalk -v 3 -l authPriv -u "quobis" -a sha -A "5gt4ng00" -x aes -X "5gt4ng00" "localhost" NET-SNMP-EXTEND-MIB::nsExtendOutputFull.\"sippoServerConferences\"
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."sippoServerConferences" = STRING: 0

Will this mean a problem for the monitoring tool?

In this example: https://github.com/sonata-nfv/tng-y1-demo/blob/master/tango/squid/Definitions/squid_vnf.yaml we see that the variables are defined this way:

oids:
        - oid: "1.3.6.1.4.1.3495.1.1.3.0"
          metric_name: "cacheUptime"
          metric_type: "gauge"
          unit: "secs"
          mib_name: "SQUID-MIB"

We are planning to deifne it as follows, but we have some doubts:

oids:
        - oid: "1.3.6.1.4.1.8072.1.3.2.3.1.2.22.115.105.112.112.111.83.101.114.118.101.114.67.111.110.102.101.114.101.110.99.101.115"
          metric_name: "sippoServerConferences"
          metric_type: "gauge" ???
          unit: "number" ???
          mib_name: "NET-SNMP-EXTEND-MIB" ??

Do you know what values must we set in the field with '???' we don't know if those values must be understood by the platform, we think so. Are the possible values defined anywhere?

Thanks in advance

SLM still receiving monitoring data for deleted services

I have noticed that the Monitoring Manager is still sending monitoring information of services that have been deleted by executing a DELETE on the url <base_monitoring_link>/<service_instance_uuid>, which responded with a 204 message.

Integration test 8 is failing

Hello the integration test 8 have the following error:

03:37:57 docker: Error response from daemon: invalid header field value "oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"rootfs_linux.go:53: mounting \\\"/home/jenkins/workspace/int-8-sdk-son-monitor/misc/prometheus_jenkins.yml\\\" to rootfs \\\"/var/lib/docker/aufs/mnt/97d60da71fdbeebea7675cb5449835a564031e2f7450ce14725d3f7b6d0dcfbd\\\" at \\\"/var/lib/docker/aufs/mnt/97d60da71fdbeebea7675cb5449835a564031e2f7450ce14725d3f7b6d0dcfbd /etc/prometheus/prometheus.yml\\\" caused \\\"not a directory\\\"\""\n".

More information:
https://jenkins.sonata-nfv.eu/view/INTEGRATION/job/int-8-sdk-son-monitor/327/console

Monitoring API(s)

@trakadasp, @pkarkazis: we need to start the discussion on the Monitoring API(s), I know there's a URL with the current version documentation (link, please?). For the GK, we need to 'subscribe' a certain set of monitoring parameters. Moreover, we need to 'mask' some of that data. After a successful subscription, Monitoring would provide the subscribed data to the SDK, upon request.

Shall we start discussing this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.