Grafana returns 401 on direct image render link from the image renderer about grafana-image-renderer HOT 9 CLOSED

We also ran into this, there's a Grafana deployment with 2 pods in k8s for us, and Grafana renderer as separate deployment. And GF_RENDERING_CALLBACK_URL is using a k8s Service, so might hit one of the 2 Grafana pods, sometimes failing sometimes succeeding. Would be nice to have support for this setup too.

from grafana-image-renderer.

I may be missing some details above. It looks like this problem exists if I run more than 1 grafana container. When grafana calls the renderer, it passes the renderKey and it looks like the renderer needs to callback to the original grafana within the cluster. Am I correct?

from grafana-image-renderer.

@weedmaps-andychan that's correct.

from grafana-image-renderer.

The documentation is not too clear about that. May be adding a few lines in README.

from grafana-image-renderer.

When deploying multiple grafana webheads in a cluster you'll want to use a shared database. If each one has a separate database you will have all sorts of problems.

from grafana-image-renderer.

The main issue right now is that Grafana stores the render key in memory and not in shared database and that is the reason renderer must callback to original Grafana.

from grafana-image-renderer.

Aha, that is a problem then. Would this be an application for the remote cache?

from grafana-image-renderer.

Yes, I definitely think so and should be quite straightforward to implement. Just need to create issue and prioritize it.

from grafana-image-renderer.

Closed by grafana/grafana#22031 and will be included in Grafana v6.7.0. You'll need to configure rendering callback url in Grafana to be able for renderer to callback into public/load balancer url.

from grafana-image-renderer.