Thank you about cluster-monitoring HOT 19 CLOSED

Another thanks! A lot of the times I struggle with ARM image support, so it was relieving to see kube-prometheus ported, with all of the same ease-of-access. Now I can monitor my two pi's and two bare metal x86 on my homelab

Great work, and thanks a bunch.

from cluster-monitoring.

@carlosedp muito obrigado!, I will take a closer look and see which one I can use as example. Also check out our Edge solution at Open Horizon for your Nvidia Jetson projects :)

from cluster-monitoring.

Glad to find this thread. Gratitude from me too. I've got a full-fat k8s cluster on Pi 4Bs running Ubuntu 18.04 and had got prom/grafana working but without pi temps and with very few dashboards.

Thanks for the work @carlosedp - particularly the recent updates supporting named PVs.

In case it's of interest, I didn't have jsonnet set up to build the files so I set up a Dockerfile to create an image to do it. You can find it and the build script at https://gist.github.com/Kolossi/c9ee0900a23f73d41ce16d68231abd7d

The dockerfile includes the edits I want to the vars.jsonnet file:

• enable arm exporter
• named pv persistence
• domain suffix
• no TLSingress

It also copies all the "manifests/setup" up into the manifests directory - it's far easier to have them all together, I prefix the setup filenames with "00..." so they get run first.

Lastly, the Dockerfile post-processes the produced files:

• add "www." _pre_fix to domains
• set security context runAsUser and fsGroup

@carlosedp - the last two might provide ideas for further customisation in vars.jsonnet? (though of course you have no obligation whatsoever to meet my use case :-) )

Finally the build.sh copies the files out of the builder conatiner to my local.

from cluster-monitoring.

Neat, thanks @Kolossi !
Gonna take a look to integrate the Docker build in the Makefile.

from cluster-monitoring.

@Kolossi what about this: 956a160

from cluster-monitoring.

@carlosedp : I'm unable to try it just now, but looks good. How do I approve the PR in your repo? 😄 (it's ok, just kidding, I do get it - it's a commit already).

For some reason I didn't think about just doing a "docker run" with golang rather than docker build-ing it 🤦

I'd still have to do the pre- and post- processing to get the files I want, but it's certainly easier to shell that than have to docker build.

Not sure if it will be quicker this way than my docker build, but certainly 20 mins seems a long time to get some text files from templates - is it downloading a load of stuff it doesnt actually need?

Anyway, no need to answer if you are busy, thanks again for your work on this.

from cluster-monitoring.

I don't have a Rpi4 to test this but on my computer (a 2018 Macbook Pro) it took around 1 minute and 10 secs (not considering the Docker container pull).

On the Odroid-N2 (ARM SBC), it took around 2 minutes.

from cluster-monitoring.

thnx! carlosedp
this project helped me a lot!

from cluster-monitoring.

@carlosedp : I was building on MINGW64 prompt hooking up to my windows docker daemon. Although the configs were destined for the pi, I wasn't building it there. So I don't know why it takes so long. The docker command says it's using overlay2 so it doesnt seem to be a slowdown caused by using docker vfs.

I've tried the "make docker" but I can't get it to fly just now.

1. I needed to prefix the docker run with winpty on windows. Probably fixed with WSL2 docker integration but I'm not ready to jump on that just yet (latest windows update required for it has stability issues).
2. I couldn't persuade the makefile to point at the work dir I'd created. It's horribly confusing in MINGW64 under windows due to both "\" & "/" escaping - possibly made worse by having to pass through winpty command - plus whether to use "C:..." or "/c/" or "/mnt/c/" in paths. Even when it parsed the path I hardcoded in the Makefile, it still wanted to look for "C:\Program Files\Git\work" ?!?!? I could probably fix it in the end but I don't have time just now to work on it :-(
3. although not needed for the docker target, the Makefile barfs on the first 3 lines as they still rely on finding gopath. It seems to carry on but the errors are "off-putting"

If I make any progress, I'll let you know.

from cluster-monitoring.

Mingw is really something I've never tested with (and probably won't). WSL2 fills this gap now.

from cluster-monitoring.

Thanks for replying @carlosedp I only used it because it came with git when I installed it way back and since I have docker and git working correctly, I've been too lazy to transfer it across to wsl. Due to the issues with the latest windows 10 version required for wsl2 I'm not prepared to jump to that until they fix it.

My point 3. shouldn't have anything to do with mingw though - even if running the docker target (which doesn't need gopath) the make will fail if it can't find gopath.

from cluster-monitoring.

@carlosedp excellent work, thank for making so simple to install. Quick question, do you have some guidelines to add additional exporters to prometheus monitoring? I need to add a json_exporter to my project. Thank you!

from cluster-monitoring.

@jiportilla Take a look in the modules directory. I've implemented a couple of external exporters in jsonnet.
They could be used as a guide.

from cluster-monitoring.

Hi @carlosedp

Thank you very much for this project.

I would like to ask a quick question. I already have prometheus and grafana running and it's working just fine. I even imported your dashboard and looks great, but of course the only issue I have now it's the temperature of the CPU, which is not being displayed.

Is there any way to just install the arm_exporter pods into the monitoring deployment to get the CPU temp and the add the target too prometheus?

Thank you in advance and best regards,

CD

from cluster-monitoring.

Hey @nzkller, thanks!
Take a look into the arm-exporter daemonset manifest and the ServiceMonitor that is used to dynamically configure Prometheus thru the prometheus-operator.

Otherwise it would be just a prometheus configuration like node_exporter, just pointing to the correct target port.

from cluster-monitoring.

Carlos thank very much for this project. Very very helpful. Brasil!

from cluster-monitoring.

@carlosedp thanks a lot for the great project!! it helped me a lot 💪🙏

from cluster-monitoring.

@carlosedp thank you very much for the project it's working on another set of raspberry pi i have, do you know if some reconfiguration are needed in order to support Rancher k3s HA deployment with the cluster monitoring application?
i am experiencing issues with nodes not responding after loading the make deploy.
Any help will be awesome - Thank you Carlos
I am attaching some pics for reference let me know if you need anything else

from cluster-monitoring.

Sorry @robmit68, I've never deployed K3s in HA mode. Can't help much there.
Maybe post in the conversations section and someone who did it can help.

from cluster-monitoring.