Interface for kgo.Client about franz-go HOT 3 CLOSED

In this case, no. Returning an interface type itself is not really standard Go (barring some small things occasionally, such as an io.Reader or hash.Hash).

If I define and return an interface in kgo, then technically any function I add to the client needs to be added to the interface, and technically this is a breaking change. I just wrote about mocks and their purpose on a different thread recently, so I'll copy and rephrase some of that here.

This blog post walks through some aspects of what follows, but imo the most important aspect could be described more, and I do so below. Dave Cheney's blog post (linked within that linked blog post) also has a section, the "Interface Segregation Principle", which describes this further. There's also other small writeups about this, e.g. "Use many small interfaces to model dependencies".

The purpose of a mock is to make it such that dependents of the api (in this case, your tests) can inject behavior and test it. But, these dependents are not interested in the entire kgo.Client API. They are interested in some type that provides a subset of the client's API -- probably Produce or PollFetches or Request. The tests do not care if the type has other functions (e.g., Broker). The standard way to define this contract in Go is by having the dependency declare "I want something that provides the two functions I am interested in", i.e., to have your tests define the interface they accept, where the interface is small and local to it.

Over time, the client API may grow to 30 or 40 methods. Your test code is not interested in those other methods. It is still only interested in a type that provides the two functions it needs. So, rather than accepting some kgo.ClientInterface type, your test code could locally define something like:

type KafkaClient interface {
  Produce(context.Context, *kgo.Record, promise...)
}

and use your local KafkaClient interface everywhere. This will allow you to inject what you want.

By having your tests be dependent on an external mock/interface (kgo.ClientInterface), it creates unnecessary dependencies. Your mocks would forever be dependent on the kgo.ClientInterface type, meaning if I introduce functions, I break your mocks. Instead, if you define interfaces locally, your mocks need not know anything of the client, and my client can add methods independently with no worry about breaking external code.

Mocks have their place, but imo not usually, and often not in tests for a package's own code. They may be useful when we, when writing package A, use a package B that takes an interface we do not want to implement, and the mock provides hooks for us to check invariants to ensure we are using the middle package B properly. For example, if I use a SQL package B, I could pass it a SQL mock in tests and then later check the mock to ensure my usage of B was correct. In contrast, per the above writeup, package B itself does not need to define that it takes the SQL mock. It just needs to take a few functions. Package B's tests do not need to use a mock, they can just use a few custom functions with a custom test interface type. The end user of package B is doing the orchestrating.

Hopefully this paraphrasing / rephrasing somewhat answered the question?

I do hope to eventually add a kmem or something package that is a tiny recreation of Kafka in memory, but that's different from this issue. It would be useful in tests, though, so you don't need to talk to a Kafka cluster and can just use a kgo.Client blindly against the memory recreation.

from franz-go.

Let me know what you think -- for now I'm going to close this issue, but depending on your thoughts it may be worth reopening?

from franz-go.

Makes sense, thanks!

from franz-go.