Struggling to catch retryable errors despite KeepRetryableFetchErrors() option about franz-go HOT 7 CLOSED

This is failing before a fetch is even issued. Currently, KeepRetryableFetchErrors only applies to errors that are returned from an actual FetchResponse (not MetadataResponse). I don't see a way in the client to hook into that right now. I'm not sure where this would necessarily fit... I wouldn't want to trigger a fetch error whenver there's a metadata load error, but there's also no real way to hook into metadata errors while consuming at the moment (unless I'm forgetting something)...

from franz-go.

Thanks for your response.

Would something like a OnMetadataResponsError callback option be a workable pattern?

from franz-go.

I think that'd work, and I also wonder if franz-go should bubble errors from metadata updates as well (i.e. inject a fake fetch)

from franz-go.

I'm facing the same issue for dialer errors too. I am not sure if this is the same root cause; if it is not I'm happy to open a different issue.

If I set my seedbrokers to an address that results in a connection unreachable, connection refused, resolution failure, etc, it warns about the error but continues to try indefinitely to connect. I understand all these errors could be temporary and retryable, but it retries even if I give it an invalid address that will never work - e.g. 10.0.0.257. The dialer error in this case is no such host.

I understand NewClient has lazy initialization so it returns a nil error before the network error has occurred, but I'm not sure of a workaround at this stage. Maybe client.Ping() ?

from franz-go.

client.Ping will help that use case, yes

from franz-go.

Metadata topic or partition failures now inject fake fetches; retryable topic or partition errors now inject fake fetches if KeepRetryableFetchErrors is used. This is in a2340eb

from franz-go.

I did not go the route of adding a new callback -- I think injecting a fake fetch is better here.

from franz-go.