tylerholien / milena Goto Github PK

It would be great if milena supported user authentication, introduced in kafka 0.9.

Hi, is Kafka 0.9.0 support planned?
Looks like it introduced some features that would make life easier for writing consumers, such as the ability to subscribe to a topic and built-in balancing consumer functionality.

Would you accept a pull request with simple non-derived instances for show and read that render topics as any other text?

Could we get a point release?

I'm working on this right now. I'll hopefully have something by the end of today.

I had been using code in the form of:

xs <- parseFile
produceMessages xs

In one case length xs == 70000 and the process took significant processing power. Changing the code to instead use mapM_ produceMessages (chunksOf 1024 xs) is a huge improvement. Perhaps the exactly implications of produceMessages (xs ++ ys) vs produceMessages xs >> produceMessages ys should be documented?

Not cereal. Why you ask?

Because of stuff like this: http://hackage.haskell.org/package/cereal-0.4.1.0/docs/Data-Serialize.html#v:decode

Ick.

Possible alternative (still in development): https://github.com/NICTA/tickle

This isn't an immediate priority, we can roll with the homespun stuff for now just to focus on the more Kafka specific stuff. Hopefully around the time we're ready to tackle serialization, Tickle is ready.

Well. I pinged the two Zookeeper client maintainers. They both said their respective clients worked.

https://hackage.haskell.org/package/hzk-1.0.0 - they aren't properly Cabalized

https://github.com/motus/haskell-zookeeper-client

I really have no idea for sure if they work. This a bookmark for further investigation and seeing what we want to do about integration.

I'm still not sold on clients controlling their offsets, but there might be other reasons to need/want to talk to the ZK cluster.

We're gonna start deploying a couple of milena applications to our staging servers this week. It'd be best for us to use a release rather than master.

Most or at least many haskell packages went the way of using MonadThrow instead of typed Exceptions for IO code.

See Michael Snoyberg's essay for a rationale.

No email or twitter on your GH account, so just popping in to say that I'm glad Milena is still bumping along :) 👍 🐻 🍯 😄

Any thoughts on the delta between Kafka 0.8 and 0.9? I'm looking to use Kafka 0.9 + melena so if there isn't already progress we might send in a PR - any experience you already have on this path would be appreciated.

If I try to use an invalid topic name, I get a KafkaDeserializationError.

Error code 17 is documented as "InvalidTopicName" here: https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-ErrorCodes

Took me a while to figure out what I am doing wrong. So it would be great if this error could be handled in order to provide a better error message.

Example:

λ> :set -XOverloadedStrings
λ> let k = mkKafkaState "foo" ("192.168.99.100", 9092)
λ> runKafka k $ produceMessages [TopicAndMessage "\"foo" (makeMessage "msg")]
Left (KafkaDeserializationError "Failed reading: invalid error code: 17\nEmpty call stack\n")