infonova / prometheusbeat Goto Github PK
View Code? Open in Web Editor NEWsend prometheus metrics to elastic
License: Other
send prometheus metrics to elastic
License: Other
There is a HTTP header with the remote write version sent, so the user shouldn't need to configure that.
Hi,
I'am runing prometheusbeat in a K8S environment. I have deployed it through Helm. I have this FATAL ERROR at runtime :
2018-10-23T09:59:44.857Z INFO instance/beat.go:544 Home path: [/] Config path: [/] Data path: [//data] Logs path: [//logs]
2018-10-23T09:59:44.862Z INFO instance/beat.go:551 Beat UUID: 883dfa0d-82b0-4f17-97a2-fcb708eaa4b1
2018-10-23T09:59:44.862Z INFO [seccomp] seccomp/seccomp.go:116 Syscall filter successfully installed
2018-10-23T09:59:44.862Z INFO [beat] instance/beat.go:768 Beat info {"system_info": {"beat": {"path": {"config": "/", "data": "/data", "home": "/", "logs": "/logs"}, "type": "prometheusbeat", "uuid": "883dfa0d-82b0-4f17-97a2-fcb708eaa4b1"}}}
2018-10-23T09:59:44.862Z INFO [beat] instance/beat.go:777 Build info {"system_info": {"build": {"commit": "unknown", "libbeat": "6.4.1", "time": "1754-08-30T22:43:41.128Z", "version": "6.4.1"}}}
2018-10-23T09:59:44.862Z INFO [beat] instance/beat.go:780 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.11"}}}
2018-10-23T09:59:44.863Z INFO [beat] instance/beat.go:784 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2018-10-10T13:56:05Z","containerized":true,"name":"prombeat-778894dcfd-pqx27","ip":["127.0.0.1/8","::1/128","100.117.221.82/32","fe80::fc8e:37ff:fe77:b550/64"],"kernel_version":"4.4.121-k8s","mac":["fe:8e:37:77:b5:50"],"os":{"family":"debian","platform":"debian","name":"Debian GNU/Linux","version":"9 (stretch)","major":9,"minor":0,"patch":0,"codename":"stretch"},"timezone":"UTC","timezone_offset_sec":0,"id":"3e608929fbd39b959f388bf468c9f0b1"}}}
2018-10-23T09:59:44.864Z INFO [beat] instance/beat.go:813 Process info {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":null,"effective":null,"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/", "exe": "/prometheusbeat", "name": "prometheusbeat", "pid": 1, "ppid": 0, "seccomp": {"mode":"filter"}, "start_time": "2018-10-23T09:59:44.540Z"}}}
2018-10-23T09:59:44.864Z INFO instance/beat.go:273 Setup Beat: prometheusbeat; Version: 6.4.1
2018-10-23T09:59:44.864Z INFO elasticsearch/client.go:163 Elasticsearch url: https://esclient.mydomain.com:80
2018-10-23T09:59:44.865Z INFO pipeline/module.go:98 Beat name: prombeat-778894dcfd-pqx27
2018-10-23T09:59:44.865Z INFO instance/beat.go:367 prometheusbeat start running.
2018-10-23T09:59:44.865Z INFO beater/prometheusbeat.go:38 prometheusbeat is running! Hit CTRL-C to stop it.
2018-10-23T09:59:44.865Z INFO [monitoring] log/log.go:114 Starting metrics logging every 30s
2018-10-23T10:00:03.727Z INFO [monitoring] log/log.go:149 Total non-zero metrics {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":0,"time":{"ms":8}},"total":{"ticks":20,"time":{"ms":32},"value":20},"user":{"ticks":20,"time":{"ms":24}}},"info":{"ephemeral_id":"91d792f9-af5d-4e50-9a22-d470adb2ed14","uptime":{"ms":18875}},"memstats":{"gc_next":4194304,"memory_alloc":1895960,"memory_total":3658672,"rss":24375296}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"elasticsearch"},"pipeline":{"clients":0,"events":{"active":0}}},"system":{"cpu":{"cores":2},"load":{"1":0.27,"15":0.24,"5":0.27,"norm":{"1":0.135,"15":0.12,"5":0.135}}}}}}
2018-10-23T10:00:03.727Z INFO [monitoring] log/log.go:150 Uptime: 18.877659338s
2018-10-23T10:00:03.727Z INFO [monitoring] log/log.go:127 Stopping metrics logging.
2018-10-23T10:00:03.727Z INFO runtime/asm_amd64.s:523 prometheusbeat stopped.
2018-10-23T10:00:03.727Z FATAL [prometheusbeat] instance/beat.go:161 Failed due to panic. {"panic": "interface conversion: interface {} is nil, not int64", "stack": "github.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat/cmd/instance.Run.func1.1\n\t/go/src/github.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat/cmd/instance/beat.go:162\nruntime.call32\n\t/usr/local/go/src/runtime/asm_amd64.s:522\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:513\nruntime.panicdottypeE\n\t/usr/local/go/src/runtime/iface.go:248\ngithub.com/infonova/prometheusbeat/beater.(*Prometheusbeat).Run\n\t/go/src/github.com/infonova/prometheusbeat/beater/prometheusbeat.go:60\ngithub.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat/cmd/instance.(*Beat).launch\n\t/go/src/github.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat/cmd/instance/beat.go:373\ngithub.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat/cmd/instance.Run.func1\n\t/go/src/github.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat/cmd/instance/beat.go:188\ngithub.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat/cmd/instance.Run\n\t/go/src/github.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat/cmd/instance/beat.go:189\ngithub.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat/cmd.genRunCmd.func1\n\t/go/src/github.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat/cmd/run.go:36\ngithub.com/infonova/prometheusbeat/vendor/github.com/spf13/cobra.(*Command).execute\n\t/go/src/github.com/infonova/prometheusbeat/vendor/github.com/spf13/cobra/command.go:766\ngithub.com/infonova/prometheusbeat/vendor/github.com/spf13/cobra.(*Command).ExecuteC\n\t/go/src/github.com/infonova/prometheusbeat/vendor/github.com/spf13/cobra/command.go:852\ngithub.com/infonova/prometheusbeat/vendor/github.com/spf13/cobra.(*Command).Execute\n\t/go/src/github.com/infonova/prometheusbeat/vendor/github.com/spf13/cobra/command.go:800\nmain.main\n\t/go/src/github.com/infonova/prometheusbeat/main.go:12\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:201"}
I have checked that my ES cluster is reachable from inside the pod. I have no other idea about the reason of this pb
I'm using the Docker version of Prometheusbeat with ES 6.2. Prometheus is using cadvisor to gather Docker metrics. The name of the metrics does not seem to be arriving to Elasticsearch.
This is a sample as taken from Prometheus:
Mon, 04 Mar 2019 13:23:18 GMTP
container_spec_cpu_shares: 1024
container_label_com_docker_compose_service: prometheus,
container_label_com_docker_compose_slug: 41c1993279644d72a7854d00f3121e226fa39ed87583f23e0dc1ffae7c4237e,
container_label_com_docker_compose_oneoff: False,
container_label_maintainer: The Prometheus Authors <[email protected]>,
container_label_com_docker_compose_container_number: 1,
container_label_com_docker_compose_version: 1.23.1,
container_label_com_docker_compose_config_hash: 167f95bd6d7f050d957025016fb2c77620d525554cd24ef9a3dc8c13380f35bf,
container_label_com_docker_compose_project: 01,
job: cadvisor
instance: cadvisor:8080,
id: /docker/5675a33223e73fba11cbbee3a234451814d2bced7bc3afb8da4796505d4a1c7f,
image: prom/prometheus:v2.7.1,
name: prometheus
However, what arrives to Elasticsearch misses the name of the metric. See the source field in the ES documents:
{
"@timestamp": "2019-03-04T14:08:19.903Z",
"value": 1024,
"labels": {
"id": "/docker/5675a33223e73fba11cbbee3a234451814d2bced7bc3afb8da4796505d4a1c7f",
"image": "prom/prometheus:v2.7.1",
"container_label_com_docker_compose_service": "prometheus",
"container_label_com_docker_compose_slug": "41c1993279644d72a7854d00f3121e226fa39ed87583f23e0dc1ffae7c4237e",
"instance": "cadvisor:8080",
"container_label_com_docker_compose_oneoff": "False",
"container_label_maintainer": "The Prometheus Authors <[email protected]>",
"state": "iowaiting",
"container_label_com_docker_compose_container_number": "1",
"container_label_com_docker_compose_version": "1.23.1",
"container_label_com_docker_compose_config_hash": "167f95bd6d7f050d957025016fb2c77620d525554cd24ef9a3dc8c13380f35bf",
"container_label_com_docker_compose_project": "01",
"job": "cadvisor"
},
"name": "prometheus"
}
container_spec_cpu_shares
is missing. What am I doing wrong?
Issue with empty @metadata
fields, see https://discuss.elastic.co/t/empty-metadata-fields-when-upgrading-to-7-1-x/187081
# cat /etc/*release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
# go version
go version go1.7.4 linux/amd64
Got the following issues:
# make setup
mkdir -p vendor/github.com/elastic/
cp -R /home/projects/src/src/github.com/elastic/beats vendor/github.com/elastic/
rm -rf vendor/github.com/elastic/beats/.git
make update
make[1]: Entering directory '/home/projects/src/src/github.com/infonova/prometheusbeat'
make[2]: Entering directory '/home/projects/src/src/github.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat'
echo /home/projects/src/src/github.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat
/home/projects/src/src/github.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat
Generated fields.yml for libbeat
make[2]: Leaving directory '/home/projects/src/src/github.com/infonova/prometheusbeat/vendor/github.com/elastic/beats/libbeat'
Updating generated files for prometheusbeat
mkdir -p include
go run ./vendor/github.com/elastic/beats/dev-tools/cmd/asset/asset.go -pkg include fields.yml prometheusbeat > include/fields.go
Invalid file path: %s fields.yml
exit status 1
vendor/github.com/elastic/beats/libbeat/scripts/Makefile:319: recipe for target 'update' failed
make[1]: *** [update] Error 1
make[1]: Leaving directory '/home/projects/src/src/github.com/infonova/prometheusbeat'
Makefile:18: recipe for target 'setup' failed
make: *** [setup] Error 2
# make
fatal: Not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
go build -i -ldflags "-X github.com/elastic/beats/libbeat/version.buildTime=2018-06-08T09:03:40Z -X github.com/elastic/beats/libbeat/version.commit="
import cycle not allowed
package github.com/infonova/prometheusbeat
imports github.com/infonova/prometheusbeat/beater
imports github.com/elastic/beats/libbeat/publisher
imports github.com/elastic/beats/libbeat/outputs
imports github.com/elastic/beats/libbeat/publisher
vendor/github.com/elastic/beats/libbeat/scripts/Makefile:112: recipe for target 'prometheusbeat' failed
make: *** [prometheusbeat] Error 1
Can you please add support for Elasticsearch 7.x?
Exporting the template with:
prometheusbeat -c "$prometheusbeatHome\prometheusbeat.yml" export template
And putting it in ES:
PUT _template/prometheusbeat
{
"index_patterns": [
"prometheusbeat-*"
],
"mappings": {
"doc": {
"_meta": {
"version": "6.4.1"
},
"date_detection": false,
"dynamic_templates": [
{
"strings_as_keyword": {
"mapping": {
"ignore_above": 1024,
"type": "keyword"
},
"match_mapping_type": "string"
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"name": {
"ignore_above": 1024,
"type": "keyword"
},
"value": {
"type": "double"
}
}
}
},
"order": 1,
"settings": {
"index": {
"mapping": {
"total_fields": {
"limit": 10000
}
},
"number_of_routing_shards": 30,
"refresh_interval": "5s"
}
}
}
Gives the following error:
{
"error": {
"root_cause": [{
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [doc : {_meta={version=6.4.1}, dynamic_templates=[{strings_as_keyword={mapping={ignore_above=1024, type=keyword}, match_mapping_type=string}}], date_detection=false, properties={@timestamp={type=date}, name={ignore_above=1024, type=keyword}, value={type=double}}}]"
}],
"type": "mapper_parsing_exception",
"reason": "Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters: [doc : {_meta={version=6.4.1}, dynamic_templates=[{strings_as_keyword={mapping={ignore_above=1024, type=keyword}, match_mapping_type=string}}], date_detection=false, properties={@timestamp={type=date}, name={ignore_above=1024, type=keyword}, value={type=double}}}]",
"caused_by": {
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [doc : {_meta={version=6.4.1}, dynamic_templates=[{strings_as_keyword={mapping={ignore_above=1024, type=keyword}, match_mapping_type=string}}], date_detection=false, properties={@timestamp={type=date}, name={ignore_above=1024, type=keyword}, value={type=double}}}]"
}
},
"status": 400
}
{
"_index": "prometheusbeat-7.3.1-2019.09.10",
"_type": "doc",
"_id": "UcjkGG0Bq4bH-F3rF0q1",
"_version": 1,
"_score": null,
"_source": {
"@timestamp": "2019-09-10T01:39:28.491Z",
"ecs": {
"version": "1.0.1"
},
"host": {
"os": {
"platform": "debian",
"version": "9 (stretch)",
"family": "debian",
"name": "Debian GNU/Linux",
"kernel": "5.0.7-1.el7.elrepo.x86_64",
"codename": "stretch"
},
"id": "689cfabc30776e6bfe2e7477e81eaa6d",
"containerized": false,
"hostname": "prometheusbeat-b4d994dd-d89xb",
"architecture": "x86_64",
"name": "prometheusbeat-b4d994dd-d89xb"
},
"agent": {
"version": "7.3.1",
"type": "prometheusbeat",
"ephemeral_id": "063b8cbb-45ef-49f7-bb9b-7af21ebdf94d",
"hostname": "prometheusbeat-b4d994dd-d89xb",
"id": "3b0510d4-100d-433d-a77b-15cad1ccc940"
},
"name": "node_cpu_seconds_total",
"value": 0,
"labels": {
"cpu": "6",
"endpoint": "https",
"instance": "*****:9100",
"namespace": "monitoring",
"service": "node-exporter-outk8s",
"job": "node-exporter-outk8s",
"mode": "steal",
"prometheus": "monitoring/k8s",
"prometheus_replica": "prometheus-k8s-0"
}
},
"fields": {
"@timestamp": [
"2019-09-10T01:39:28.491Z"
]
},
"sort": [
1568079568491
]
}
the data prometheusbeat remote write into es ,didn't hava the time,is the doc normal?
Error observed in prometheus beat logs:
2019-02-07T13:40:56.848Z INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":140,"time":141},"total":{"ticks":530,"time":539,"value":530},"user":{"ticks":390,"time":398}},"info":{"ephemeral_id":"16da06d7-11fa-4512-93f0-5fe4eb48ca2b","uptime":{"ms":180010}},"memstats":{"gc_next":22617376,"memory_alloc":18169120,"memory_total":38017664,"rss":1413120}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":10168,"published":1688,"total":1688}}},"system":{"load":{"1":2.12,"15":2.42,"5":2.29,"norm":{"1":0.0883,"15":0.1008,"5":0.0954}}}}}}
2019-02-07T13:39:29.731Z ERROR elasticsearch/client.go:299 Failed to perform any bulk index operations: Post http://192.168.194.143:9200/_bulk: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
2019-02-07T13:39:30.732Z ERROR pipeline/output.go:92 Failed to publish events: Post http://192.168.194.143:9200/_bulk: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
Error observed in prometheus:
Feb 08 05:29:50 miq-sap-prod-mgt-01.gvs.ggn prometheus[16515]: level=warn ts=2019-02-08T05:29:50.593010061Z caller=queue_manager.go:531 component=remote queue=0:http://192.168.162.71:10080/prometheus msg="Error sending samples to remote storage" count=20 err="context deadline exceeded"
Feb 08 05:29:53 miq-sap-prod-mgt-01.gvs.ggn prometheus[16515]: level=warn ts=2019-02-08T05:29:53.342480882Z caller=queue_manager.go:230 component=remote queue=0:http://192.168.162.71:10080/prometheus msg="Remote storage queue full, discarding sample. Multiple subsequent messages of this kind may be suppressed."
Logs prometheus beat in debug mode:
2019-02-08T05:37:47.903Z DEBUG [event] common/event.go:55 Dropped nil value from event where key=name
2019-02-08T05:37:47.903Z DEBUG [event] common/event.go:55 Dropped nil value from event where key=tags
2019-02-08T05:37:47.904Z DEBUG [publish] pipeline/processor.go:275 Publish event: {
"@timestamp": "2019-02-08T05:37:47.780Z",
"@metadata": {
"beat": "prometheusbeat",
"type": "doc",
"version": "6.2.4"
},
"value": 2.147483648e+09,
"labels": {
"name": "jvm_memory_bytes_max",
"environment": "****",
"instance": "",
"job": "kubernetes-pods"
},
"beat": {
"version": "6.2.4",
"name": "",
"hostname": "*****"
}
}
ES version is also 6.2.4
Can you help with resolving the issue?
==========prometheus logs:
level=warn ts=2018-11-26T07:44:39.128148501Z caller=queue_manager.go:530 component=remote queue=0:http://172.32.149.137:8089/prometheus msg="Error sending samples to remote storage" count=100 err="Post http://172.32.149.137:8089/prometheus: dial tcp 172.32.149.137:8089: connect: connection refused"
==========prometheus beat logs:
2018-11-24T16:23:44.944Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://172.32.150.100:9200)): Get http://172.32.150.100:9200: dial tcp 172.32.150.100:9200: connect: connection refused
2018-11-24T16:24:13.992Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://172.32.150.100:9200)) with 86 reconnect attempt(s)
2018-11-24T16:24:35.465Z ERROR pipeline/output.go:121 Failed to publish events: temporary bulk send failure
2018-11-24T16:25:06.044Z INFO pipeline/output.go:95 Connecting to backoff(elasticsearch(http://172.32.150.98:9200))
2018-11-24T16:25:16.773Z ERROR pipeline/output.go:121 Failed to publish events: temporary bulk send failure
2018-11-24T16:28:20.393Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://172.32.150.100:9200)): Get http://172.32.150.100:9200: dial tcp 172.32.150.100:9200: connect: connection refused
2018-11-24T16:26:23.369Z INFO pipeline/output.go:95 Connecting to backoff(elasticsearch(http://172.32.150.99:9200))
2018-11-24T16:29:03.007Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://172.32.150.100:9200)) with 87 reconnect attempt(s)
2018-11-24T16:29:12.393Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://172.32.150.98:9200)): Get http://172.32.150.98:9200: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
2018-11-24T16:32:43.126Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://172.32.150.98:9200)) with 1 reconnect attempt(s)
2018-11-24T16:34:19.741Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://172.32.150.99:9200)): Get http://172.32.150.99:9200: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
2018-11-24T16:35:43.159Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://172.32.150.100:9200)): Get http://172.32.150.100:9200: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2018-11-24T16:35:26.607Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://172.32.150.99:9200)) with 1 reconnect attempt(s)
2018-11-24T16:36:03.726Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://172.32.150.100:9200)) with 88 reconnect attempt(s)
2018-11-24T16:36:07.941Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://172.32.150.98:9200)): Get http://172.32.150.98:9200: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2018-11-24T16:37:00.245Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://172.32.150.98:9200)) with 2 reconnect attempt(s)
2018-11-24T16:38:49.730Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://172.32.150.99:9200)): Get http://172.32.150.99:9200: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2018-11-24T16:39:00.350Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://172.32.150.99:9200)) with 2 reconnect attempt(s)
2018-11-24T16:39:53.643Z INFO elasticsearch/client.go:712 Connected to Elasticsearch version 5.2.2
2018-11-24T16:39:55.999Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://172.32.150.100:9200)): Get http://172.32.150.100:9200: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2018-11-24T16:40:53.891Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://172.32.150.100:9200)) with 89 reconnect attempt(s)
Hi, I'm running prometheus 2.5 inside K8s and prometheusbeat on a different host behind nginx,
It seems prometheusbeat can't write metrics to elasticsearch, It keeps repeating the following error:
ERROR elasticsearch/client.go:374 Failed to encode event: unsupported float value: -Inf
in the source code file above, the kubernetes resource "replicasets" is defined as an "extensions" instead of "app". this causes error when running metric-beat in AWS EKS 1.15 cluster.
I added the "replicasets" resource to the apiGroups "apps" in order to fix the error that I got.
For libraries used within prometheus/prometheus.go
Add Dockerfile for prometheusbeat
prometheusbeat logs is normal,but ES has no data
can you help me
Not sure if this is the correct place for this issue. I cloned the repo to go/src/github.com instead of go/github.com, since the src directory is where all the other packages were installed. Any help is appreciated. Thank you,
ubuntu % make setup
mkdir -p vendor/github.com/elastic/
cp -R /home/bwillcox/go/src/github.com/elastic/beats vendor/github.com/elastic/
rm -rf vendor/github.com/elastic/beats/.git
make update
make[1]: Entering directory '/home/bwillcox/go/src/github.com/infonova/prometheusbeat/prometheusbeat'
Running virtualenv with interpreter /usr/bin/python2
New python executable in /home/bwillcox/go/src/github.com/infonova/prometheusbeat/prometheusbeat/build/python-env/bin/python2
Also creating executable in /home/bwillcox/go/src/github.com/infonova/prometheusbeat/prometheusbeat/build/python-env/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
Generated fields.yml for prometheusbeat to /home/bwillcox/go/src/github.com/infonova/prometheusbeat/prometheusbeat/fields.yml
Updating generated files for prometheusbeat
mkdir -p include
go run ./vendor/github.com/elastic/beats/dev-tools/cmd/asset/asset.go -license ASL2 -pkg include -in fields.yml -out include/fields.go prometheusbeat
if [ -d /home/bwillcox/go/src/github.com/infonova/prometheusbeat/prometheusbeat/_meta/kibana ]; then
cp -pr /home/bwillcox/go/src/github.com/infonova/prometheusbeat/prometheusbeat/_meta/kibana/* /home/bwillcox/go/src/github.com/infonova/prometheusbeat/pr
ometheusbeat/_meta/kibana.generated ;
fi
ERROR: field <@timestamp> is duplicated, remove it or set 'overwrite: true'
exit status 1
vendor/github.com/elastic/beats/libbeat/scripts/Makefile:315: recipe for target 'update' failed
make[1]: *** [update] Error 1
make[1]: Leaving directory '/home/bwillcox/go/src/github.com/infonova/prometheusbeat/prometheusbeat'
Makefile:18: recipe for target 'setup' failed
make: *** [setup] Error 2
ubuntu % go version
go version go1.10.4 linux/amd64
ubuntu % uname -a
Linux ubuntu 4.15.0-42-generic #45-Ubuntu SMP Thu Nov 15 19:32:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
few seconds after running : ~/work/src/github.com/infonova/prometheusbeat# ./prometheusbeat -c prometheusbeat.yml
the output says and don't data to Elastic
2019/03/28 13:57:27 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 80ms
2019/03/28 13:57:27 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 160ms
2019/03/28 13:57:27 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 320ms
2019/03/28 13:57:28 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 5ms
2019/03/28 13:57:28 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 10ms
2019/03/28 13:57:28 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 20ms
2019/03/28 13:57:28 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 40ms
2019/03/28 13:57:28 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 80ms
2019/03/28 13:57:28 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 5ms
2019/03/28 13:57:28 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 10ms
2019/03/28 13:57:28 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 5ms
2019/03/28 13:57:28 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 10ms
2019/03/28 13:57:28 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 20ms
2019/03/28 13:57:28 http: Accept error: accept tcp [::]:9080: accept4: too many open files; retrying in 40ms
Hi, I´m trying to install the prometheusbeat but when I run make setup after downloading repository with git clone, I get this output in the terminal:
~/Desktop/prometheusbeat$ make setup
mkdir -p vendor/github.com/elastic
cp -R /src/github.com/elastic/beats vendor/github.com/elastic/
cp: cannot stat '/src/github.com/elastic/beats': No such file or directory
make: *** [Makefile:28: copy-vendor] Error 1
Am I missing something? Thanks in advance
Can you please provide an Windows amd64 binary?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.