Kafka protocol support about sdk-python HOT 4 CLOSED

Hey @davidwmartines, thx for opening this issue.

Yes, as you mentioned, the repo is very http-centric, unfortunately :-(

I'm looking forward to having some free time to bring Protobuf support, hopefully, this should bring some clarity regarding adding more formatting options.
Also, we now have CloudEvent abstract base, so technically you can subclass it and add any required properties.

I personally don't have Kafka experience, but curious how this complies with the Cloudevents spec. Need to check.

And yeah, PRs are very welcome

from sdk-python.

OK, that makes sense. I have updated my PR to remove the kafa.CloudEvent class, and the kafka conversion functions should work with AnyCloudEvent (both http and pydantic work, at least in my simple testing). I still need to add some unit tests. I think I like this approach.

from sdk-python.

@xSAVIKx I started exploring this and came up with a draft of something that seems to work.

I'm not sure if it's the right approach, though. For one thing, the kafka.CloudEvent class I created is basically a copy of the http.CloudEvent. It seems that if we are creating protocol-specific CloudEvent classes, then we might need a more functional base class inheriting the abstract.CloudEvent, with the common behavior implemented for the protocol-specific implementations to build upon (if needed).

Alternatively, I could simply not create a kafka.CloudEvent, with the expectation that consumers would use the http.CloudEvent. Then the kafka module would then only need to provide the protocol-specific conversion functions. This actually seems more in the spirit of cloudevents - a common, protocol-agnostic event structure, with extensions provided for converting to/from the native protocol formats.

Your thoughts?

(BTW, I researched more and discovered the cloudevents and knative connection, so now the http focus of all this makes more sense.)

from sdk-python.

Yeah, I guess kafka is just a conversion format and there's no need for extra CloudEvent class. Pydantic approach is a bit different and the abstract base was created for framework-specific implementations.

from sdk-python.