Comments (8)
kakkoyun
commented on June 8, 2024
Hey @darwin67, thanks a lot for reaching out. I'm happy that you're using it.
From your log output, I suspect you have multiple entries for rule groups in your Prometheus configuration.
level=error ts=2019-09-24T17:44:32.749Z caller=manager.go:833 component="rule manager" msg="loading groups failed" err="groupname: \"thanos-querier.rules\" is repeated in the same file"
level=error ts=2019-09-24T17:44:32.749Z caller=manager.go:833 component="rule manager" msg="loading groups failed" err="groupname: \"thanos-receive.rules\" is repeated in the same file"
level=error ts=2019-09-24T17:44:32.749Z caller=manager.go:833 component="rule manager" msg="loading groups failed" err="groupname: \"thanos-store.rules\" is repeated in the same file"
Could you share more of your jsonnet files, especially the ones that you use mixin.libsonnet?
from kube-thanos.
darwin67
commented on June 8, 2024
@kakkoyun this is the full list of imports I have in my jsonnet.
local kp = (import 'kube-prometheus/kube-prometheus.libsonnet') +
(import 'kube-prometheus/kube-prometheus-kubeadm.libsonnet') +
(import 'kube-prometheus/kube-prometheus-static-etcd.libsonnet') +
(import 'kube-prometheus/kube-prometheus-anti-affinity.libsonnet') +
(import 'kube-prometheus/kube-prometheus-thanos-sidecar.libsonnet') +
(import 'kube-thanos/kube-thanos-querier.libsonnet') +
(import 'kube-thanos/kube-thanos-store.libsonnet') +
// (import 'kube-thanos/kube-thanos-pvc.libsonnet') + // Uncomment this line to enable PVCs
(import 'kube-thanos/kube-thanos-compactor.libsonnet') +
(import 'kube-thanos/kube-thanos-receive.libsonnet') +
(import 'kube-thanos/kube-thanos-sidecar.libsonnet') +
(import 'kube-thanos/kube-thanos-servicemonitors.libsonnet') +
// (import 'thanos-mixin/mixin.libsonnet') +
(import 'lib/node-tolerations.libsonnet') +
I think the only other mixin besides thanos are the etcd stuff. I haven't configured any rules of my own so there's nothing custom that could have caused the collusion.
So the only thing that might have collided would be the things generated from the kube-prometheus. Ring any bells on your end?
This is the source for jsonnetfile.json btw.
{
"dependencies": [
{
"name": "kube-prometheus",
"source": {
"git": {
"remote": "https://github.com/coreos/kube-prometheus",
"subdir": "jsonnet/kube-prometheus"
}
},
"version": "v0.2.0"
},
{
"name": "kube-thanos",
"source": {
"git": {
"remote": "https://github.com/metalmatze/kube-thanos",
"subdir": "jsonnet/kube-thanos"
}
},
"version": "master"
},
{
"name": "thanos-mixin",
"source": {
"git": {
"remote": "https://github.com/metalmatze/kube-thanos",
"subdir": "jsonnet/thanos-mixin"
}
},
"version": "master"
}
]
}
from kube-thanos.
darwin67
commented on June 8, 2024
Oh wait, could it be because I'm importing it twice?
And it somehow duplicated the rules?
from kube-thanos.
darwin67
commented on June 8, 2024
So I tried changing the jsonnetfile.json to this.
{
"dependencies": [
{
"name": "kube-prometheus",
"source": {
"git": {
"remote": "https://github.com/coreos/kube-prometheus",
"subdir": "jsonnet/kube-prometheus"
}
},
"version": "v0.2.0"
},
{
"name": "kube-thanos",
"source": {
"git": {
"remote": "https://github.com/metalmatze/kube-thanos",
"subdir": "jsonnet"
}
},
"version": "master"
}
]
}
Then updated the subpath in the jsonnet like the following.
local kp = (import 'kube-prometheus/kube-prometheus.libsonnet') +
(import 'kube-prometheus/kube-prometheus-kubeadm.libsonnet') +
(import 'kube-prometheus/kube-prometheus-static-etcd.libsonnet') +
(import 'kube-prometheus/kube-prometheus-anti-affinity.libsonnet') +
(import 'kube-prometheus/kube-prometheus-thanos-sidecar.libsonnet') +
(import 'kube-thanos/kube-thanos/kube-thanos-querier.libsonnet') +
(import 'kube-thanos/kube-thanos/kube-thanos-store.libsonnet') +
// (import 'kube-thanos/kube-thanos/kube-thanos-pvc.libsonnet') + // Uncomment this line to enable PVCs
(import 'kube-thanos/kube-thanos/kube-thanos-compactor.libsonnet') +
(import 'kube-thanos/kube-thanos/kube-thanos-receive.libsonnet') +
(import 'kube-thanos/kube-thanos/kube-thanos-sidecar.libsonnet') +
(import 'kube-thanos/kube-thanos/kube-thanos-servicemonitors.libsonnet') +
(import 'kube-thanos/thanos-mixin/mixin.libsonnet') +
(import 'lib/node-tolerations.libsonnet') +
And problem still persist. I'm now suspecting one of the two,
- there are duplicate rules in
kube-prometheusandkube-thanos - there are duplicate rules in
kube-thanos
from kube-thanos.
metalmatze
commented on June 8, 2024
You don't want to import the thanos-mixin with the rest of the kube-thanos ksonnet/Kubernetes files.
Here's an example on how we use it ourselves:
https://github.com/observatorium/configuration/blob/master/environments/sre/prometheusrules.jsonnet#L1
https://github.com/observatorium/configuration/blob/master/environments/sre/prometheusrules.jsonnet#L52
from kube-thanos.
darwin67
commented on June 8, 2024
You don't want to import the thanos-mixin with the rest of the kube-thanos ksonnet/Kubernetes files.
Ugh, is this by design? It does seems to be a leaky abstration to me if that's what I need to do...
from kube-thanos.
metalmatze
commented on June 8, 2024
Yes, this is by design. In the future, maybe the thanos-mixin would be moved to Thanos directly, as this is strictly coupled to what metrics Thanos exposes in each version.
Think of kube-thanos as a project for Kuberentes and thanos-mixin is completely separate defining rules, alerts and dashboards.
from kube-thanos.
darwin67
commented on June 8, 2024
Okay, got it. Closing then.
from kube-thanos.
Related Issues (20)
- Switch CI to github actions HOT 1
- thanos-store without volumeClaimTemplate not working HOT 4
- Support new endpoint flag in querier HOT 2
- questions: add secret for receiver object bucket HOT 1
- Configure compactor CPU threads HOT 1
- Support specify the annotations for SAs HOT 4
- what can i replace here "dnssrv+_client._tcp.<MEMCACHED_SERVICE>.thanos.svc.cluster.local" HOT 5
- Where is Thanos compactor? and ruler? HOT 1
- Cannot connect to prometheus-operated HOT 3
- Can't save db with objectstorage( Aliyun OSS)
- Add release process to Thanos Docs / automation
- The `manifests` directory is outdated HOT 1
- Can't get notifications from GitHub App HOT 1
- DATA LOSS: thanos-compact deduplication is experimental and should not be enabled by default
- Support creating PrometheusRule
- Can not quick start when following up the tutorials
- PodDistruptionBudget should be added for store and query
- Possible misconfiguration with rules and serviceMonitor
- Thanos Support Policy
- Add NetworkPolicy to all components
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kube-thanos.