Comments (22)
I'm now working on replace poseidon with ruby-kafka to support v0.9.
from fluent-plugin-kafka.
kafka_buffered
output have already supported kafka v0.9 and v0.10.
Does anyone need to support kafka v0.9 or v0.10 with input plugins?
from fluent-plugin-kafka.
This plugin should support Kafka 0.9 also, and it can work with 0.9.
Regarding to Kerberos, this plugin doesn't support yet. let me think about it. thanks,
from fluent-plugin-kafka.
is there a branch for that upgrade to ruby-kafka?
from fluent-plugin-kafka.
any date for 0.9 release? trying to use it with mapr streams (0.9 api ) but having problems fetching metadata from broker (for sure is the api version )
thanks mate!
from fluent-plugin-kafka.
Hi @repeatedly, any update on the migration to ruby-kafka, is there a branch for it ?
Would be happy to help if I can
from fluent-plugin-kafka.
0.10 is being released
https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+0.10.0
from fluent-plugin-kafka.
nice, so that version will be 0.9 kafka api compliance right? its not clear for me in the content included.
thanks!
from fluent-plugin-kafka.
Sorry for the delay. I opened a PR here: #54
from fluent-plugin-kafka.
thank you @repeatedly !!
from fluent-plugin-kafka.
I'd love to see Kafka 0.9 input supported, along with SSL capabilities. I'm guessing the Zookeeper config option would be dropped given that 0.9 consumer doesn't need to talk to ZK.
The ability to read a topic from the beginning would be even better, but that's for another day :)
from fluent-plugin-kafka.
Actually I need v0.10 support with input plugins.
from fluent-plugin-kafka.
@repeatedly I need the kafka v0.9 support with input plugins.
from fluent-plugin-kafka.
I wrote a patch for input plugins.
Please test: #64
from fluent-plugin-kafka.
@repeatedly I tested this plugin with Kafka 0.10.0 as Kafka consumer. Input plugin (@type 'kafka' and @type 'kafka_group') are working well in my environment.
from fluent-plugin-kafka.
@kentarosasaki Thanks for testing!
Does anyone also test the patch?
If there is no problem, I will merge the patch and release fluent-plugi-kafka v0.3.0-dev.1
from fluent-plugin-kafka.
I released fluent-plugin-kafka v0.3.0.rc1 for testing.
You can install this version with --pre
option.
Please check it on your environment before release v0.3.0.
from fluent-plugin-kafka.
@repeatedly I've tested v0.3.0.rc1 which was installed by using --pre
option. It's also working fine on my environment.
from fluent-plugin-kafka.
Thanks for the test again!
I need more tester :)
from fluent-plugin-kafka.
@repeatedly fwiw, I've been testing v0.3.0.rc1 today and everything looks good. I'll keep it in pre-production testing for the next few weeks.
from fluent-plugin-kafka.
@hmmbug thanks for the testing!
from fluent-plugin-kafka.
I will release fluent-plugin-kafka v0.3.0 in the end of this week.
from fluent-plugin-kafka.
Related Issues (20)
- Newly introduced share_producer setting in 0.18.0 for out_kafka2 plugin results in BufferOverflowError HOT 3
- rdkafka 0.12.0 is incompatible with fluent-plugin-kafka v0.18.0 or former HOT 7
- Fluent-bit failed to connecting to the Kafka brokers using Self Signed certs HOT 3
- Unexpected behavior by fluentd-collector from one Elastic search index to kafka HOT 4
- Output plugin kafka2 error: Kafka::BufferOverflow error="Cannot produce to XXX, max buffer size (1000 messages) reached HOT 3
- Support AWS MSK Authentication HOT 2
- Every buffer configuration returns Kafka::MessageSizeTooLarge or chunk bytes limit exceeds for an emitted event stream HOT 2
- Scram mechanism is not supported HOT 1
- SASL_PLAIN configuration does not work HOT 1
- encrypt / suppress sensitive data in the log HOT 2
- logging level HOT 2
- Implement `use_default_for_unknown_topic` for rdkafka2 output plugin HOT 3
- How to set topic_key with record's field in kafka2 output
- Request fluent-plugin-kafka gem version update HOT 3
- Expect a fluentd-plugin-rocketmq like this one
- ruby-kafka is no longer maintained and superseded by librdkafka via rdkafka-ruby bindings HOT 5
- Errors while sending data to Eventhub HOT 3
- source is kafka , not support kdc without ssl
- when fluentd's config graceful reload, Warn logs have occurred 'undefined method `synchronize' for nil:NilClass'
- Infinite reconnect until BufferChunkOverflowError is resolved
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluent-plugin-kafka.