Comments (5)
drigz
commented on July 3, 2024
1
Thanks! FYI, as a workaround, we deleted the letsencrypt-keypair secret. This makes kubernetes-letsencrypt create a new user with an empty quota.
kubectl --namespace kube-system delete secret letsencrypt-keypair
from kubernetes-letsencrypt.
drigz
commented on July 3, 2024
1
Note: LE just enabled pending authorization recycling, which might (help) avoid this issue:
https://community.letsencrypt.org/t/automatic-recycling-of-pending-authorizations/41321
from kubernetes-letsencrypt.
drigz
commented on July 3, 2024
I've looked in the logs for the kubernetes-letsencrypt and noticed two things.
One: the CloudDnsResponder threw an exception early on:
Exception in thread "Thread-2" java.lang.UnsupportedOperationException: Empty collection can't be reduced.
at in.tazj.k8s.letsencrypt.acme.CloudDnsResponder.findMatchingZone(CloudDnsResponder.kt:123)
at in.tazj.k8s.letsencrypt.acme.CloudDnsResponder.updateCloudDnsRecord(CloudDnsResponder.kt:55)
at in.tazj.k8s.letsencrypt.acme.CloudDnsResponder.addChallengeRecord(CloudDnsResponder.kt:26)
at in.tazj.k8s.letsencrypt.acme.CertificateRequestHandler.prepareDnsChallenge(CertificateRequestHandler.kt:176)
at in.tazj.k8s.letsencrypt.acme.CertificateRequestHandler.authorizeDomain(CertificateRequestHandler.kt:77)
at in.tazj.k8s.letsencrypt.acme.CertificateRequestHandler.access$authorizeDomain(CertificateRequestHandler.kt:27)
at in.tazj.k8s.letsencrypt.acme.CertificateRequestHandler$requestCertificate$1.accept(CertificateRequestHandler.kt:41)
at in.tazj.k8s.letsencrypt.acme.CertificateRequestHandler$requestCertificate$1.accept(CertificateRequestHandler.kt:27)
[SNIP: java.util.stream.*]
at in.tazj.k8s.letsencrypt.acme.CertificateRequestHandler.requestCertificate(CertificateRequestHandler.kt:41)
at in.tazj.k8s.letsencrypt.kubernetes.ServiceManager.handleCertificateRequest(ServiceManager.kt:64)
at in.tazj.k8s.letsencrypt.kubernetes.ServiceManager.access$handleCertificateRequest(ServiceManager.kt:20)
at in.tazj.k8s.letsencrypt.kubernetes.ServiceManager$reconcileService$1.run(ServiceManager.kt:45)
at java.lang.Thread.run(Thread.java:745)
This appears to be because our Cloud DNS configuration had the wrong zone, so the responder didn't work.
Two: this error occurs 300 times before the rate limit error takes its place. This takes about an hour because the operation is retried very frequently. The retries continue, leading to rate limit errors every 45 seconds or so.
Two things that could help this:
- The
authzshould be deleted if the CloudDnsResponder crashes, to avoid hitting the "pending authorizations" limit. - Exponential backoff should be used in case of failures.
from kubernetes-letsencrypt.
tazjin
commented on July 3, 2024
Thanks for reporting this, I'll look into handling this more gracefully!
from kubernetes-letsencrypt.
tazjin
commented on July 3, 2024
Interesting! I started working on the issues you reported yesterday - but time is currently a scarce resource :-)
from kubernetes-letsencrypt.
Related Issues (20)
- Allow single certificate for multiple services HOT 1
- Route 53 Split-horizon DNS HOT 8
- prepareDnsChallenge cleanup exception HOT 2
- Always determine authoritative NS from root
- Support ACME V2 API and wildcard certificates HOT 5
- Influence the cert filenames HOT 3
- Add support for ingress controller secret format HOT 2
- NullPointerException in DnsRecordObserver.findAuthoritativeNameservers HOT 4
- Transient error: "Must agree to subscriber agreement" HOT 1
- DnsException: Login Required HOT 6
- Exception in thread "Thread-23" HOT 2
- 403 Forbidden HOT 1
- create a chained cert for nginx as well HOT 7
- Exception: empty collection can't be reduced HOT 8
- Is this project still active? HOT 22
- No pom.xml? HOT 1
- hanging in fail-loop HOT 2
- Failed due to invalid challenge HOT 5
- LetsencryptException: No matching zone found. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kubernetes-letsencrypt.