Comments (6)
htgc
commented on August 28, 2024
I confirmed it can resend after reconnected connection with following log.
2015-01-18 23:14:01 +0900 [warn]: retry succeeded. plugin_id="object:3fa3f02d1148"
Perhaps, I couldn't build a same environment.(I tried stop kafka-server while td-agent is working, after exception occurred, I restart kafka-server)
Please let me know how to build this situation if I make misunderstanding.
Thanks,
from fluent-plugin-kafka.
mohitkhanna
commented on August 28, 2024
@htgc I can consistently reproduce it on my setup. I will see if I have concrete steps to reproduce it and mail them to you.
BTW... Just to add on to my observation: I can see the error happening after the max retries have been reached. So after I let the data go into the buffer file, I bring down the broker and let the agent fail more than max retries.
After that, this error starts occurring and data is never retransmitted. Lets for time being assume that that's the right behaviour, but:
- Why repeated errors about "All messages not sent"? Looks like it's trying to attempt again and again on the same data.
- What happens to the new data that comes from that point onwards? Will that atleast be sent? I have to again confirm that. I don't think new data is being sent unless i restart the agent.
For now - i have put a retry limit of 150 to take care of major downtime - but that's just a workaround.
from fluent-plugin-kafka.
htgc
commented on August 28, 2024
Sorry for being super late.
I missed your notice below.
after restarting the broker service (different server)
This plugin configures broker info only start fluentd, so it couldn't reconnect another broker server.
I'll fix this issue by next release.
Sorry for inconvenience.
from fluent-plugin-kafka.
raviprakashshahi
commented on August 28, 2024
Hi Mohit,
Did you manage to re-push the logs in the buffer file? I am facing the same problem and had to move the buffer files for td-agent to start working again.
Logs in buffer files contain the tag-name and some special character also, something like "��TagName�X�����message� �" [logs]
Is there any plugin which can reprocess these logs.
from fluent-plugin-kafka.
mohitkhanna
commented on August 28, 2024
Ravi,
It's been sometime so I cannot recall exact steps that we had to do. But the logs got pushed out to Kafka after we corrected some topic Leader election issue and restarting fluentd agent.
Besides we had the raw source data to help us recover in case that step had not worked.
Not aware of any plugin that could have done that forced.
Best reach out to the author of the plugin for that. Also Fluentd authors are very active on Slack - suggest you join that and ask this question.
Search for fluentd + Slack and you'd know how to onboard on that forum.
Regards,
Mohit
… On 23-Feb-2017, at 22:43, raviprakashshahi ***@***.***> wrote:
Hi Mohit,
Did you manage to re-push the logs in the buffer file? I am facing the same problem and had to move the buffer files for td-agent to start working again.
Logs in buffer files contain the tag-name and some special character also, something like "��TagName�X�����message� �" [logs]
Is there any plugin which can reprocess these logs.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.
from fluent-plugin-kafka.
mohitkhanna
commented on August 28, 2024
Please also upgrade to latest Kafka plugin versions.
Regards,
Mohit
… On 24-Feb-2017, at 00:47, Mohit Khanna ***@***.***> wrote:
Ravi,
It's been sometime so I cannot recall exact steps that we had to do. But the logs got pushed out to Kafka after we corrected some topic Leader election issue and restarting fluentd agent.
Besides we had the raw source data to help us recover in case that step had not worked.
Not aware of any plugin that could have done that forced.
Best reach out to the author of the plugin for that. Also Fluentd authors are very active on Slack - suggest you join that and ask this question.
Search for fluentd + Slack and you'd know how to onboard on that forum.
Regards,
Mohit
> On 23-Feb-2017, at 22:43, raviprakashshahi ***@***.***> wrote:
>
> Hi Mohit,
>
> Did you manage to re-push the logs in the buffer file? I am facing the same problem and had to move the buffer files for td-agent to start working again.
>
> Logs in buffer files contain the tag-name and some special character also, something like "��TagName�X�����message� �" [logs]
>
> Is there any plugin which can reprocess these logs.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or mute the thread.
>
from fluent-plugin-kafka.
Related Issues (20)
- Newly introduced share_producer setting in 0.18.0 for out_kafka2 plugin results in BufferOverflowError HOT 3
- rdkafka 0.12.0 is incompatible with fluent-plugin-kafka v0.18.0 or former HOT 7
- Fluent-bit failed to connecting to the Kafka brokers using Self Signed certs HOT 3
- Unexpected behavior by fluentd-collector from one Elastic search index to kafka HOT 4
- Output plugin kafka2 error: Kafka::BufferOverflow error="Cannot produce to XXX, max buffer size (1000 messages) reached HOT 3
- Support AWS MSK Authentication HOT 2
- Every buffer configuration returns Kafka::MessageSizeTooLarge or chunk bytes limit exceeds for an emitted event stream HOT 2
- Scram mechanism is not supported HOT 1
- SASL_PLAIN configuration does not work HOT 1
- encrypt / suppress sensitive data in the log HOT 2
- logging level HOT 2
- Implement `use_default_for_unknown_topic` for rdkafka2 output plugin HOT 3
- How to set topic_key with record's field in kafka2 output
- Request fluent-plugin-kafka gem version update HOT 3
- Expect a fluentd-plugin-rocketmq like this one
- ruby-kafka is no longer maintained and superseded by librdkafka via rdkafka-ruby bindings HOT 5
- Errors while sending data to Eventhub HOT 3
- source is kafka , not support kdc without ssl
- when fluentd's config graceful reload, Warn logs have occurred 'undefined method `synchronize' for nil:NilClass'
- Infinite reconnect until BufferChunkOverflowError is resolved
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluent-plugin-kafka.