External DNS not managing multiple zones about external-dns HOT 7 OPEN

You could try adding the following extraArgs in case it's a timeout issue, or a rate limit issue:

--interval=3m
--request-timeout=60s

from external-dns.

Thanks for the suggestion. Unfortunately none of those seem to work. I have also tried one of a mix of the following:

aws-batch-change-size
aws-batch-change-size-values
aws-batch-change-interval

Each zone will have around 250 records, so in total it should be able to add about 1250 in the 5 zones. I have tried with 10s intervals and 200 records at a time but no luck so far...

from external-dns.

I've been looking at the code (keep in mind I don't speak Go...) and I can see a few things that I'm not sure if I understand:

aws.go, line 602

// submitChanges takes a zone and a collection of Changes and sends them as a single transaction.
func (p *AWSProvider) submitChanges(ctx context.Context, changes Route53Changes, zones map[string]*profiledZone) error {
	// return early if there is nothing to change
	if len(changes) == 0 {
		log.Info("All records are already up to date")
		return nil
	}

After removing all the records in Z0005 and re-installing the chart, I can see from the logs in the pod that len(changes) == 0, which is not right because there are plenty of changes to be applied to that zone. I can see the 'All records are already up to date' message in the logs.

When I look back to line 585 (func (p *AWSProvider) ApplyChanges(ctx context.Context, changes *plan.Changes) error {), I can see that the list with the combined changes is created and sent to the function submiChanges above. However, it seems that the list of changes is empty, which shouldn't be.

At this point, I'm not sure where this function is called from (maybe from aws_sd.go?) and/or what's in the context. I have the suspicion that the code is not managing well having multiple zones ids all with the same name, but I can't figure out if that's the case by looking at the code.

I have tried all possible combinations of values in the chart, but the only way I can get this to work is by doing the zones one by one manually, which is far from ideal. I'm looking at maybe trying to separate the deployments so to have one per zone (pretty sure that would solve my issue) but I don't think I can do that without modifying the chart myself.

Any help would be appreciated.

from external-dns.

@ophintor Can you test if with external-dns version 0.13.6 this issue also happens?

from external-dns.

We actually moved from 0.13.6 to 0.14.1, and then to 0.14.2 because of the issue, in the hope that a newer version would fix it.

At the moment I have found a workaround that involves creating one deployment per zone and that works for us, but as it is I cannot make it work.

Thanks!

from external-dns.