Comments (8)
mensfeld
commented on September 27, 2024
1
I will close this issue once documented.
from waterdrop.
mensfeld
commented on September 27, 2024
Works correctly. Here's a minimal setup (unless you did something else) minus the noise:
producer = ::WaterDrop::Producer.new do |wd_config|
# something incorrect
wd_config.kafka[:'bootstrap.servers'] = '0.0.0.0:29092'
wd_config.kafka[:'transactional.id'] = SecureRandom.hex(6)
# Make shorter to timeout faster
wd_config.kafka[:'message.timeout.ms'] = 2_000
wd_config.kafka[:'transaction.timeout.ms'] = 3_000
end
producer.monitor.subscribe(WaterDrop::Instrumentation::LoggerListener.new(Logger.new(STDOUT)))
producer.produce_many_sync([{ topic: '4c28211d4684', payload: 'a' }])here are logs:
E, [2024-06-06T13:28:12.234512 #2347228] ERROR -- : [waterdrop-32f7954face5] Error occurred: Local: Timed out (timed_out) - transaction.begin
E, [2024-06-06T13:28:19.235389 #2347228] ERROR -- : [waterdrop-32f7954face5] Error occurred: Local: Timed out (timed_out) - transaction.begin
E, [2024-06-06T13:28:26.736377 #2347228] ERROR -- : [waterdrop-32f7954face5] Error occurred: Local: Timed out (timed_out) - transaction.begin
E, [2024-06-06T13:28:34.737464 #2347228] ERROR -- : [waterdrop-32f7954face5] Error occurred: Local: Timed out (timed_out) - transaction.begin
E, [2024-06-06T13:28:40.738384 #2347228] ERROR -- : [waterdrop-32f7954face5] Error occurred: Local: Timed out (timed_out) - transaction.abort
E, [2024-06-06T13:28:47.239256 #2347228] ERROR -- : [waterdrop-32f7954face5] Error occurred: Local: Timed out (timed_out) - transaction.abort
E, [2024-06-06T13:28:54.240035 #2347228] ERROR -- : [waterdrop-32f7954face5] Error occurred: Local: Timed out (timed_out) - transaction.abort
E, [2024-06-06T13:29:01.740825 #2347228] ERROR -- : [waterdrop-32f7954face5] Error occurred: Local: Timed out (timed_out) - transaction.abort
E, [2024-06-06T13:29:09.741818 #2347228] ERROR -- : [waterdrop-32f7954face5] Error occurred: Local: Timed out (timed_out) - transaction.abort
E, [2024-06-06T13:29:09.741982 #2347228] ERROR -- : [waterdrop-32f7954face5] Error occurred: #<Rdkafka::RdkafkaError: Local: Timed out (timed_out)> - messages.produce_many_sync
(irb):12:in `<main>': #<Rdkafka::RdkafkaError: Local: Timed out (timed_out)> (WaterDrop::Errors::ProduceManyError)from waterdrop.
mensfeld
commented on September 27, 2024
The thing that is NOT working here is the fact that the timeout should raise faster on retried transacitonal operations because reaching the timeout time. This however could cause consistency issues on retryable transactions hence I am leading towards documenting this rather than trying to re-raise on wait timeout time.
from waterdrop.
mensfeld
commented on September 27, 2024
you can see this if you run it like that (should timeout after 1 second, times out much later):
producer = ::WaterDrop::Producer.new do |wd_config|
# wd_config.kafka[:'bootstrap.servers'] = '0.0.0.0:29092'
wd_config.kafka[:'transactional.id'] = SecureRandom.hex(6)
## wd_config.kafka[:'max.in.flight'] = 5
wd_config.kafka[:'message.timeout.ms'] = 2_000
wd_config.kafka[:'transaction.timeout.ms'] = 3_000
wd_config.max_attempts_on_transaction_command = 1
wd_config.wait_backoff_on_transaction_command = 5
wd_config.max_wait_timeout = 1_000
end
producer.monitor.subscribe(WaterDrop::Instrumentation::LoggerListener.new(Logger.new(STDOUT)))
producer.produce_many_sync([{ topic: '4c28211d4684', payload: 'a' }])from waterdrop.
mensfeld
commented on September 27, 2024
@Aerdayne if you can confirm this is the case I will look into this deeper.
from waterdrop.
Aerdayne
commented on September 27, 2024
Can confirm that both of your versions with adjusted timeout values raise eventually.
from waterdrop.
mensfeld
commented on September 27, 2024
@Aerdayne ok, so it is not hanging. It just retries and (aside from time it takes) works as expected. I will document this behaviour. I do not want to have granular time tracking inside of transactions because this could cause really weird problems (especially if not able to abort) causing problems with LSO.
from waterdrop.
mensfeld
commented on September 27, 2024
added a warning in the timeout section: http://localhost/docs/WaterDrop-Transactions/#timeouts
from waterdrop.
Related Issues (20)
- Produce lifecycle events similar to karafka
- Trying to understand recommendation against at_exit HOT 2
- Revisit the librdkafka `delivery.timeout.ms` vs. max wait time
- Introduce `instrument_on_wait_queue_full` for ability to disable instrumentation on non-critical queue full errors
- Document `instrument_on_wait_queue_full` in the docs on retries
- Transactional producer config not aligned by default
- Middleware runs twice when using buffered produce
- Provide alias to `#with` named `#variant`
- Provide `#idempotent?` similar to `#transactional?`
- Enhance AppSignal and DataDog instrumentation with transactional metrics (when used)
- Check if variants can work with idempontents on and acks changes. If not, validate similar to transactional alterations to ack
- Implementation to report topic level statistics
- Transactional sync producer raises a 'Purged in queue' exception when producing messages via the same producer that previously failed to produce messages HOT 6
- Consumer consuming error: Local: Erroneous state (state) post break flow in transaction
- Raise an exception when using return/break/next in a transactional code block similar to Rails 8 HOT 1
- Replace the `throw(:abort)` API with direct `WaterDrop::AbortTransaction` error HOT 1
- Dispatched collection in the ProduceManyError contains DeliveryHandle and DeliveryReport HOT 8
- Improve transactional spec stability
- NoMethodError: undefined method `init_transactions' if `transactional.id` is specified HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from waterdrop.