outage in europe-west3 (Frankfurt) about registry.k8s.io HOT 9 CLOSED

thanks, I am in Germany and our whole company was grinding to a halt as our tooling was awaiting the image :/ was able to switch to k8s.gcr.io in the meantime after some investigation

from registry.k8s.io.

For me I can confirm everything is working as expect now.

from registry.k8s.io.

Can you please share more details?
We're not aware of an outage.

from registry.k8s.io.

@ameukam may have found something affecting europe-west3 (GCP) that should resolved now xref: kubernetes/k8s.io#4337

from registry.k8s.io.

k8s.gcr.io will be redirected to registry.k8s.io in the near future, so we should make sure the issue is resolved 😅

a small rollout will be coming next week.

https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md#urgent-upgrade-notes-3

from registry.k8s.io.

I'm from Thailand and having a problem with registry.k8s.io too.

$ docker pull registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20220916-gd32f8c343@sha256:39c5b2e3310dc4264d638ad28d9d1d96c4cbb2b2dcfb52368fe4e3c63f61e10f
Error response from daemon: unauthorized: not authenticated: No valid credential was supplied.

My GKE in SG region

Failed to pull image "registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.3.0@sha256:549e71a6ca248c5abd51cdb73dbc3083df62cf92ed5e6147c780e30f7e007a47": rpc error: code = Unknown desc = failed to pull and unpack image "registry.k8s.io/ingress-nginx/kube-webhook-certgen@sha256:549e71a6ca248c5abd51cdb73dbc3083df62cf92ed5e6147c780e30f7e007a47": failed to resolve reference "registry.k8s.io/ingress-nginx/kube-webhook-certgen@sha256:549e71a6ca248c5abd51cdb73dbc3083df62cf92ed5e6147c780e30f7e007a47": failed to do request: Head "https://asia-southeast1-docker.pkg.dev/v2/k8s-artifacts-prod/images/ingress-nginx/kube-webhook-certgen/manifests/sha256:549e71a6ca248c5abd51cdb73dbc3083df62cf92ed5e6147c780e30f7e007a47": failed to authorize redirect: failed to fetch anonymous token: unexpected status: 403 Forbidden

from registry.k8s.io.

@winggundamth that's not clearly related to the root cause of the original post here. Can you please file another issue with more details about your environment

EDIT: cannot replicate locally, will need more details when you file your issue:

$ docker pull registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20220916-gd32f8c343@sha256:39c5b2e3310dc4264d638ad28d9d1d96c4cbb2b2dcfb52368fe4e3c63f61e10f
registry.k8s.io/ingress-nginx/kube-webhook-certgen@sha256:39c5b2e3310dc4264d638ad28d9d1d96c4cbb2b2dcfb52368fe4e3c63f61e10f: Pulling from ingress-nginx/kube-webhook-certgen
1cd0595314a5: Pull complete 
12fc225329e5: Pull complete 
Digest: sha256:39c5b2e3310dc4264d638ad28d9d1d96c4cbb2b2dcfb52368fe4e3c63f61e10f
Status: Downloaded newer image for registry.k8s.io/ingress-nginx/kube-webhook-certgen@sha256:39c5b2e3310dc4264d638ad28d9d1d96c4cbb2b2dcfb52368fe4e3c63f61e10f
registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20220916-gd32f8c343@sha256:39c5b2e3310dc4264d638ad28d9d1d96c4cbb2b2dcfb52368fe4e3c63f61e10f

EDIT: can reproduce that on GKE in Singapore, tracking in #116
EDIT: Singapore region issue should be resolved now. Sorry about these folks.

from registry.k8s.io.

@xeruf Any issue again ? Can we call this done ?

from registry.k8s.io.

Will reopen if we see anything further, or you can comment /reopen and the bot will reopen.
This should be resolved, with no further reports.

from registry.k8s.io.