Git Product home page Git Product logo

zabbix-agent-extension-elasticsearch's Introduction

zabbix-agent-extension-elasticsearch

zabbix-agent-extension-elasticsearch - this extension for monitoring Elasticsearch cluster and node health/status.

Supported features

This extension obtains stats of two types:

Node stat

https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html

  • roles
  • attributes
  • indices (partly)
  • os
  • processes
  • jvm
  • thread_pool
  • fs
  • transport
  • http
  • breakers
  • script
  • discovery
  • ingest

Cluster health

https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html

  • cluster_name
  • status
  • timed_out
  • number_of_nodes
  • number_of_data_nodes
  • total indices docs count
  • total indices deleted docs count
  • primary indices docs count
  • primary indices deleted docs count
  • total indices store size
  • primary indices store size
  • active_primary_shards
  • active_shards
  • relocating_shards
  • initializing_shards
  • unassigned_shards
  • delayed_unassigned_shards
  • number_of_pending_tasks
  • number_of_in_flight_fetch
  • task_max_waiting_in_queue_millis
  • active_shards_percent_as_number

Installation

Notice

Before manual installation you should check Include option in your zabbix-agent configuration, it should be uncomment and check that include path are the same with this installation rule - https://github.com/zarplata/zabbix-agent-extension-elasticsearch/blob/master/Makefile#L54 otherwise you should change it to your include path.

After installation you should restart your zabbix-agent manually for inclusion new UserParameter from extension configuration.

Manual build

# Building
git clone https://github.com/zarplata/zabbix-agent-extension-elasticsearch.git
cd zabbix-agent-extension-elasticsearch
make

#Installing
make install

# By default, binary installs into /usr/bin/ and zabbix config in /etc/zabbix/zabbix_agentd.conf.d/ but,
# you may manually copy binary to your executable path and zabbix config to specific include directory

Arch Linux package

# Building
git clone https://github.com/zarplata/zabbix-agent-extension-elasticsearch.git
git checkout pkgbuild

./build.sh

#Installing
pacman -U *.tar.xz

Dependencies

zabbix-agent-extension-elasticsearch requires zabbix-agent v2.4+ to run.

Zabbix configuration

In order to start getting metrics, it is enough to import template and attach it to monitored node.

WARNING: You must define macro with name - {$ZABBIX_SERVER_IP} in global or local (template) scope with IP address of zabbix server.

On one node of cluster set MACRO {$GROUPNAME} = REAL_ZABBIX_GROUP. This group must include all nodes of the cluster. Only this one node will be triggered cluster status (low level discovery added aggregate checks of cluster health).

Customize key prefix

It may you need if key in template already used.

If you need change key elasticsearch.* -> YOUR_PREFIX_PART.elasticsearch.*, run script custom_key_template.sh whit YOUR_PREFIX_PART and import updated zabbix template template_elasticsearch_service.xml.

./custom_key_template.sh YOUR_PREFIX_PART

Elasticsearch API authentication (X-Pack security)

This extension support basic authentication which provided by X-Pack. For authentication in Elasticsearch you must set valid values in template macros - ${ES_USER} and ${ES_PASSWORD}

Customize Elasticsearch address.

You can customize you Elasticsearch listen address. Just change{$ES_ADDRESS} macros in template. Possible values are - (http|https)://host:port

Be note if you choose https and have self-signed certificate you also should add path to you CA in marcos {$CA_PATH}

zabbix-agent-extension-elasticsearch's People

Contributors

adriankirchner avatar adrienne-cln avatar akitsul avatar rafael-romero-carmona-claranet avatar tears-of-noobs avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zabbix-agent-extension-elasticsearch's Issues

How to configure in zabbix-agent-extension-elasticsearch.conf file and make it work?

Hi,
I am facing some issue for extension configuration. Can anyone help to figure out how to make it work?

Zabbix-agent version: 3.4.11
My configuration like below:
es-01:/usr/local/zabbix/etc/zabbix_agentd.conf.d # cat zabbix-agent-extension-elasticsearch.conf
UserParameter=elasticsearch.discovery[], /usr/bin/zabbix-agent-extension-elasticsearch --discovery --agg-group $1
UserParameter=elasticsearch.stats[
], /usr/bin/zabbix-agent-extension-elasticsearch --zabbix-host $1 --zabbix-prefix $2
UserParameter=elasticsearch.indices.discovery[], /usr/bin/zabbix-agent-extension-elasticsearch indices --discovery --agg-group $1
UserParameter=elasticsearch.indices.stats[
], /usr/bin/zabbix-agent-extension-elasticsearch indices --zabbix-host $1 --zabbix-prefix $2 --agg-group $3
elasticsearch.stats[{$ZABBIX_SERVER_IP},{$ES_ZBX_PREFIX},ESIP:ESPORT]

This is my zabbix-agentd configuration:
LogFile=/tmp/zabbix_agentd.log
Server=$zabbix_server_ip
ServerActive=$zabbix_server_ip:10051
Hostname=es-01
Include=/usr/local/zabbix/etc/zabbix_agentd.conf.d/

The Macros for one node in cluster:
host_macro

The Macros for Elastic Template:
inkedtemplate_macro

Hope someone can elaborate on the specific configuration steps and the set of key parameters. If my statement is not clear enough, please point out. thanks in advance!

Debugging tip

If you're having problems with the parameters sent, a debugging trick is to add echo here:
UserParameter=elasticsearch.stats[*], echo /usr/bin/zabbix-agent-extension-elasticsearch....

The zabbix item returned is then what you would try to pass to the command.

I didn't know where to comment other than make an issue.

No Latest Data

Hi,

Recently I installed the extension and the zabbix-server just have received the datas from zabbix-agent. The another datas from zabbix-trapper didn't came.

screenshot from 2018-08-30 13-04-11

Executing manually the commands below, return ok or the indexes correctly.

/usr/bin/zabbix-agent-extension-elasticsearch --discovery --agg-group my-group --user None --password None
/usr/bin/zabbix-agent-extension-elasticsearch --zabbix my-zabbix-server --prefix None_pfx --user None --password None
/usr/bin/zabbix-agent-extension-elasticsearch --type indices --discovery --agg-group my-group --user None --password None
/usr/bin/zabbix-agent-extension-elasticsearch --type indices --zabbix my-zabbix-server --prefix None_pfx --agg-group my-group --user None --password None

I did another tests, sending data manually using zabbix-sender, for example:

zabbix_sender -z my-zabbix-server -s "my-elastic-server" -k elasticsearch.cluster_health.active_primary_shards -o 1

but the extension didn't send the datas.

What can doing wrong?

Sorry for my bad english.

deps error

When I try to make , I have an error. Could you please help me for this? Im newbie.

cd /root/zabbix-agent-extension-elasticsearch/.gopath/src/zabbix-agent-extension-elasticsearch;
GOPATH=/root/zabbix-agent-extension-elasticsearch/.gopath
dep ensure -v
/bin/sh: 2: dep: not found
Makefile:40: recipe for target 'deps' failed
make: *** [deps] Error 127

Problemas to make work.

Hey Guys,
im sorry for ask but i really need help with this matter. After installing the packages and install go and move the .conf file from your .zip to the directory /zabbix/zabbix_agentd.d/. Im stuck with this messages on latest data from zabbix:

"Usage:
zabbix-agent-extension-elastic [options]"

and i dont understand properly this instructions here.

"Customize key prefix
It may you need if key in template already used.

If you need change key elasticsearch.* -> YOUR_PREFIX_PART.elasticsearch.*, run script custom_key_template.sh whit YOUR_PREFIX_PART and import updated zabbix template template_elasticsearch_service.xml.

./custom_key_template.sh YOUR_PREFIX_PART"

PLEASE! can u help-me understand?

No values for key "elasticsearch.cluster_health.status_int" in group(s)

I have my 3 master nodes in a zabbix group called ELSPRD1
On master node 1 set the macro {$GROUPNAME} = ELSPRD1
On this node I get the following item errors :

No values for key "elasticsearch.cluster_health.unassigned_shards" in group(s) "ELSPRD1".
No values for key "elasticsearch.cluster_health.active_shards_percent" in group(s) "ELSPRD1".
No values for key "elasticsearch.cluster_health.status_int" in group(s) "ELSPRD1".
No values for key "elasticsearch.cluster_health.delayed_unassigned_shards" in group(s) "ELSPRD1".
No values for key "elasticsearch.cluster_health.relocating_shards" in group(s) "ELSPRD1".

Zabbix can't recieve data when using agent encryption

I'm using zabbix-agent-extention-elasticsearch 20180831.32_7cbc5d1 and I noticed that Zabbix will not receive most of the items when only encrypted communication is allowed from host.

As a workaround allow No encryption from host together with PSK or Certificate.

Not sure if this is a problem in the newer versions too, feel free to close this issue if it is already fixed. Just wanted to save somebody time debugging the issue, because it's not really mentioned anywhere.

myslq problems with graphical characters(Non ASCII characters)

I get the following error on zabbix

201:20191025:143752.888 [Z3005] query failed: [1366] Incorrect string value: '\xE2\x94\x94\xE2\x94\x80...' for column 'value' at row 1 [insert into history_str (itemid,clock,ns,value) values (391012,1572028672,559058877,'can`t get indices stats from Elasticsearch https://127.0.0.1:9201
this appears to be caused by the leading characters on:

└─ Get https://127.0.0.1:9201/_cluster/health: dial tcp 127.0.0.1:9201: connect: connection refused

A simple fix is to add "| cat -v" to the zabbix conf to filter out the non-ascii characters.

Error in Elasticsearch discovery rule

The error is

Cannot create item: item with the same key "elasticsearch.node_stats.jvm.buffer_polls.count.[{#JVMBUFFERSPOOLS}]" already exists

How to correct it?

Most data missing data from missing. Possible missing zabbix-agent-extension-elasticsearch.conf entries?

Hello,

I have Zabbix 4.0.2, Latest zabbix-agent-extensoin-elasticsearch from master [today], Elasticsearch 6.5.2, and when checking the graphs, I don't see any metrics except for port list and process num when viewing the graphs.

my zabbix-agent-extension-elasticsearch.conf says the following
UserParameter=elasticsearch.discovery[], /usr/bin/zabbix-agent-extension-elasticsearch --discovery --elasticsearch $1 --agg-group $2 --user $3 --password $4
UserParameter=elasticsearch.stats[
], /usr/bin/zabbix-agent-extension-elasticsearch --zabbix $1 --elasticsearch $2 --prefix $3 --user $4 --password $5

UserParameter=elasticsearch.indices.discovery[], /usr/bin/zabbix-agent-extension-elasticsearch --type indices --discovery --elasticsearch $1 --agg-group $2 --user $3 --password $4
UserParameter=elasticsearch.indices.stats[
], /usr/bin/zabbix-agent-extension-elasticsearch --type indices --zabbix $1 --elasticsearch $2 --prefix $3 --agg-group $4 --user $5 --password $6

But I noticed that the key items in the Elasticsearch xml template [that are empty on the graph] are

elasticsearch.cluster_health.*
elasticsearch.node_stats.*

which their are no entries in the zabbix-agent-extension-elasticsearch.conf file which would explain why it's empty in the graphs but I can't options anywhere for it. Any idea?

Some documentations precisions

  1. Is is necessary to have zabbix sender installed ? The zabbix documentation mentions that the use of trapper requires zabbix sender.

  2. A macro need to be defined with {$ZABBIX_SERVER_IP} in global or local (template) scope with IP address of zabbix server. What if a zabbix proxy sits in the middle ?

extention doesn't work with proxy zabbix servers

Hello!

We have a huge infrastructure to monitor with our Zabbix server so we use several proxy servers for load balancing purposes. And main Zabbix server is closed for monitored hosts.
Is it possible to configure this extension to send data via proxy server?

Thank you.

Error when manual build!

HI,

I get an error when manual build,
Clean dependencies
rm -rf vendor/*
Clean builded binaries
rm -rf .out/
rm -rf .gopath/
Done
Creating GOPATH
mkdir -p .gopath/src
ln -s /root/zabbix-agent-extension-elasticsearch /root/zabbix-agent-extension-elasticsearch/.gopath/src/zabbix-agent-extension-elasticsearch
Fetch dependencies
cd /root/zabbix-agent-extension-elasticsearch/.gopath/src/zabbix-agent-extension-elasticsearch;
GOPATH=/root/zabbix-agent-extension-elasticsearch/.gopath
dep ensure -v
/bin/sh: 2: dep: not found
Makefile:40: recipe for target 'deps' failed
make: *** [deps] Error 127
How can resolve this error? Plz help me.
Tks so much

I can't Customize Elasticsearch ip/port, zabbix-server port ?

Elasticsearch
IP 127.0.0.1
Port 2900
I want to config elasticsearch port: 2900, Help me please ?
elasticsearch.stats[{$ZABBIX_SERVER_IP},{$ES_ZBX_PREFIX},ESIP:ESPORT]

image

image

file : /etc/zabbix/zabbix_agentd.d/zabbix-agent-extension-elasticsearch.conf
UserParameter=elasticsearch.stats[*], /usr/bin/zabbix-agent-extension-elasticsearch --zabbix $1 --prefix $2 --user $3 --password $4

Unsupported item key -discover rules

my es version: 6.6.5
my zabbix version: 4.0.4

deployed follow the instruction,but the discovery rule doesn't work.
I'm sure that there's no x-pack active, no user&password needed
How can i deal with this?
微信截图_20190317133959
微信截图_20190317134117

Basic user auth

Does anyone know a way to authenticate user created with X-Pack ?

I have 3-node elasticsearch cluster. I installed elastizabbix and try to connect to ES cluster. But the problem is that zabbix-agent-extension-elasticsearch does not have permission to get cluster stats or health.
( Elasticsearch cluster returned 401 HTTP code, expected 200 HTTP code)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.