Git Product home page Git Product logo

nginx-vts-zbx-n9e's Introduction

需求

  • 抓取nginx-module-vts的数据
  • 计算每个servername、upstream的每秒请求数
  • 发送到监控平台zabbix和夜莺

文档

设计

  • 方案一(不合理): 利用zabbix的LLD每次执行时发送请求抓数据,出现断点,机器负载高
  • 方案二: 定期每秒抓取数据计算后写文件,zabbix的LLD每次执行读文件数据

方案一

  • 源码: main.go

    # 编译
    go build -o nginx-vts-zbx
    mv nginx-vts-zbx ./bin
    
    # for zabbix LLD
    # 查看所有servernames
    bin/nginx-vts-zbx -s
    # 查看servername的req/s
    bin/nginx-vts-zbx -s -o "test.abc.com"
    
    # 查看所有upstreams
    bin/nginx-vts-zbx -u
    # 查看单个upstream的req/s
    bin/nginx-vts-zbx -u -o "test_upstream-10.11.100.79:9000"
    
    # for nightinagle
    # 一分钟上报一次, 默认endpoint为10.10.10.10
    # -t 开启nightingale,关闭zabbix LLD
    # -a 指定transfer
    # -p 上报endpoint
    bin/nginx-vts-zbx -t -a "http://10.51.1.31:5810/api/transfer/push" -p 10.10.10.111
    # 部署
    # for zabbix
    # zabbix client
    # cat /usr/local/zabbix/conf/zabbix_agentd/userparameter_ngx_vts.conf
    UserParameter=serverzones.discovery,/bin/nginx-vts-zbx -c "http://127.0.0.1/status/format/json" -s
    UserParameter=serverzone.reqs[*],/bin/nginx-vts-zbx -s -o $1
    UserParameter=upstreamzones.discovery,/bin/nginx-vts-zbx -u
    UserParameter=upstreamzone.reqs[*],/bin/nginx-vts-zbx -u -o $1
    
    # zabbix server 验证
    zabbix_get -s 10.201.0.11 -k serverzones.discovery
    zabbix_get -s 10.201.0.11 -k serverzone.reqs[s.abc.com]
    zabbix_get -s 10.201.0.11 -k upstreamzones.discovery
    zabbix_get -s 10.201.0.11 -k upstreamzone.reqs["weixin-10.21.4.157:8087"]

方案二

  • fetch-nginx-vts: 每分钟获取nginx-vts的数据计算后写文件

  • ngx-vts-zbx: 读文件提供给zabbix和夜莺

    ## fetch-nginx-vts
    go build -o fetch-nginx-vts
    # 指定nginx-vts源
    ./fetch-nginx-vts -u "http://screen.abc.com/ngx_status/format/json"
    # 指定写的文件
    ./fetch-nginx-vts -f "nginx-request.json"
    
    ## ngx-vts-zbx
    go build -o ngx-vts-zbx
    # zabbix的LLD  s: serverzone  u: upstream
    go run  main.go -f "nginx-request.json" -s
    go run  main.go -f "nginx-request.json" -u
    go run  main.go -f "nginx-request.json" -s -o screen.abc.com
    go run  main.go -f "nginx-request.json" -u -o test-10.11.100.79:9093
    # 夜莺
    go run  main.go -f "nginx-request.json" -n
    # 指定endpoint上报地址
    go run  main.go -f "nginx-request.json" -c "http://10.51.1.31:5810/api/transfer/push" -p 10.10.10.11 -n
    ## 部署
    ## for nightinagle : 用cron每分钟写入
    * * * * * /usr/local/monitor/nginx_vts/fetch-ngx-vts >/dev/null 2>&1
    * * * * * /usr/local/monitor/nginx_vts/ngx-vts-zbx -n -p 10.11.100.230 >/dev/null 2>&1
    
    ## for zabbix LLD
    /etc/zabbix/zabbix_agentd.d/scripts/ngx-vts-zbx
    # 不改zabbix用户自定义参数
    ln -s  /etc/zabbix/zabbix_agentd.d/scripts/ngx-vts-zbx /bin/nginx-vts-zbx

知识

nginx-moudle-vts

  • install && config

    # check nginx install module
    nginx/sbin/nginx -t
    configure arguments: ... --add-module=../nginx-module-vts
    
    # install
    wget http://nginx.org/download/nginx-1.15.10.tar.gz
    tar xvf nginx-1.15.10.tar.gz -C /opt
    cd /opt/nginx-1.15.10
    git clone git://github.com/vozlt/nginx-module-vts.git
    ./configure --prefix=/usr/local/nginx --add-module=nginx-module-vts
    
    # config
    http {
        vhost_traffic_status_zone;
        ...
        server {
            ...
            location /status {
                vhost_traffic_status_display;
                vhost_traffic_status_display_format html;
            }
        }
    }
    
    # access
    http://127.0.0.1/status/format/json

zabbix LLD

  • 理解:通过自动发现配置宏 ; 通过用户参数获取自动发现的宏

  • 用zabbix-agent的实现逻辑

    1. 用户自定参数UserParameter配置两个脚本
      xxx.discovery,script1
      yyy[*],script2 $1
      
      脚本一: 返回JSON数据,里面的key为zabbix使用宏
      脚本二: 利用脚本一中宏的value作为参数,获取的详细信息
    2. 界面配置, 如ngx-vts-zbx.pdf所示

zabbix-client

  • config

    # zabbix agent config
    #  -c可选 默认为 http://127.0.0.1/ngx_status/format/json
    
    # cat /usr/local/zabbix/conf/zabbix_agentd/userparameter_ngx_vts.conf
    UserParameter=serverzones.discovery,/bin/nginx-vts-zbx -c "http://xxx/ngx_status/format/json" -s
    UserParameter=serverzone.reqs[*],/bin/nginx-vts-zbx -s -o $1
    
    UserParameter=upstreamzones.discovery,/bin/nginx-vts-zbx -u
    UserParameter=upstreamzone.reqs[*],/bin/nginx-vts-zbx -u -o $1
    
    # restart 
    ps aux |grep zabbix |grep -v grep | awk '{print $2}' | xargs kill
    /usr/local/zabbix/sbin/zabbix_agentd -c /usr/local/zabbix/conf/zabbix_agentd.conf
  • confirm

    # /usr/local/zabbix/sbin/zabbix_agentd -c /usr/local/zabbix/conf/zabbix_agentd.conf -t serverzones.discovery
    {
        "data": [
                {
                    "{#SERVERZONE}": "test.abc.com"
                },
                {
                    "{#SERVERZONE}": "all"
                }
           ]
    }
    # /usr/local/zabbix/sbin/zabbix_agentd -c /usr/local/zabbix/conf/zabbix_agentd.conf -t serverzone.reqs["test.abc.com"]
    10
    # /usr/local/zabbix/sbin/zabbix_agentd -c /usr/local/zabbix/conf/zabbix_agentd.conf -t upstreamzones.discovery
    {
         "data": [
             {
                 "{#UPSTREAMNAME}": "test_upstream-10.11.100.17:9000"
             },
             {
                 "{#UPSTREAMNAME}": "test_upstream-10.11.100.79:9000"
             }
         ]
     }
    # /usr/local/zabbix/sbin/zabbix_agentd -c /usr/local/zabbix/conf/zabbix_agentd.conf -t upstreamzone.reqs["127.0.0.1:9000"]
    9

trobule

  • Value should be a JSON object.
    # zabbix server执行, 返回为空数据 { "data": [] }, 重启agent正常 :)
    zabbix_get -s 10.11.9.23 -k "serverzones.discovery"
  • Cannot create item: item with the same key “upstreamzone.reqs[{#UPSTRAMNAME}]” already exists
    返回的key{#xxx}必须为大写{#XXX}
    {#UPSTREAMNAME} 拼写错误 {#UPSTRAMNAME} 
  • ZBX_NOTSUPPORTED: Special characters "\, ', ", `, *, ?, [, ], {, }, ~, $, !, &, ;, (, ), <, >, |, #, @, 0x0a" are not allowed
    # 替换*为all或者其他
    strings.Replace(o, "*", "all", -1)

nginx-vts-zbx-n9e's People

Stargazers

 avatar

Watchers

 avatar  avatar

nginx-vts-zbx-n9e's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.