Loki components stuck on `CrashLoopBackOff` when deploying example manifests on GKE about observatorium HOT 6 OPEN

Just noticed that the same thing happens to Dex deployment as well

from observatorium.

@ArthurSens This sounds strange because the /data directory is a persistent volume with ReadWriteOnce access mode. Changing the security context to access the volume sounds like brute force. Can you elaborate on your GKE configuration? Especially take a look on the default StorageClass used for Loki's PVCs.

from observatorium.

Any idea? I'm still seeing that when deploying on EKS

from observatorium.

I can confirm that adding a securityContext to the container fixed the problem for me here, even with runAsNonRoot set as true

      securityContext:
        fsGroup: 2000
        runAsNonRoot: true
        runAsUser: 1000

from observatorium.

We test all of the manifests on every PR using KinD. Without Dex and Loki working, the tests wouldn't pass. Something seems fishy here. It's worth checking if this is because of a K8s change or if our tests are silently failing, etc

from observatorium.

Well there is some difference between kind and EKS then

But note that all the other deployments have an explicit securityContext defined

from observatorium.