Single broken entity breaks the whole batch about external-dns HOT 11 CLOSED

Hi @linki

Some folks I work with (or myself) may have interest in solving this problem as it's something we run into.

I'm curious on a few points, if you'd know off hand.

• Does this issue exist for all providers? Or just AWS?
• Is our preferred approach to attempt to use batch still? But fall back to single records if batch fails? Or perhaps reattempt batches with the 'tainted' record removed?

from external-dns.

Ho @joshrosso,

It depends whether the provider implementation in ExternalDNS uses batch logic (e.g. AWS and Google do and DigitalOcean doesn't, afaik) and then how individual failure is handled (e.g. ignore the failure, stop any further processissing, rollback everything).

If we could identify the broken record from the error message we should - like you say - ignore it and optionally taint it so it doesn't get retried in the next iteration.

If we cannot identify the record we could use a more generic strategy - like you say as well - stop using batch and apply records individually and ignore the failing ones or - a bit more efficient - split the batches in half recursively and apply them until only the smallest failing ones are left.

from external-dns.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

from external-dns.

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

from external-dns.

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

from external-dns.

@fejta-bot: Closing this issue.

from external-dns.

/reopen

from external-dns.

@linki: Reopened this issue.

from external-dns.

Collecting the related issues:

• bug report with first batch size feature: #374
• another bug report: #421
• another bug report and fix: #515 #596
• another bug report: #731

from external-dns.

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

from external-dns.

@fejta-bot: Closing this issue.

from external-dns.