Comments (10)
@gjobin Sorry for my late reply.
You can create a Fluency
instance that synchronously flushes data like this
// Same as: TCPSender sender = new TCPSender.Config().setHost("localhost").setPort(24224).createInstance();
TCPSender sender = new TCPSender.Config().createInstance();
Fluency fluency = new Fluency.Builder(sender).setFlusherConfig(new SyncFlusher.Config()).build();
I'll add an example code snippet like the above to README later. Maybe I should add a configuration API to Fluency.defaultFluency()
for AsyncFlusher
.
from fluency.
Thank you for the reply.
Sure A configuration API would do the job, but at least I can now do it manually.
from fluency.
I've been testing sync flushing and see some weird behavior that I can't explain looking at the code.
I've configured Fluency as above-- no other options. I've then set up a thread that pushes ~200 small messages into emit() per minute every minute. But fluentd isn't receiving them in a timely manner. I added a file output to fluentd and I've been watching it for 30 minutes now with no data. Is this expected?
from fluency.
@jklap With SyncFlusher, Fluency doesn't flush its buffers automatically. Can you call Fluency#flush
when you want to flush the buffers?
from fluency.
I'm closing this issue. The issue @jklap said above is being discussed on #64.
from fluency.
That is curious-- both emit() and Fluency#flush() call flushInternal() which calls buffer.flush() depending on the flag force or time passed or buffer threshold-- so am I reading this wrong as it does seem like SyncFlusher should flush during calls to emit()? And I'm not calling Fluency#flush() but it is flushing-- anywhere between 45 and 90 minutes consistently with that number of messages.
from fluency.
Fluency flushes the buffers when it tried to append an event to the buffers and runs into BufferFullException. https://github.com/komamitsu/fluency/blob/master/src/main/java/org/komamitsu/fluency/Fluency.java#L163
You kept to append events calling emit()
without a call of flush()
and the Fluency caught BufferFullException. As a result, the Fluency flushed the buffers the several ten minutes later, I think.
from fluency.
Wouldn't the flushInternal() called as part of the emit() force a flush based on time or threshold? Or am misunderstanding that logic?
from fluency.
emit(), upon catching the BufferFullException logs that fact at an error level before calling flush()-- I don't see that error message in my log files. But even so, both emit() and flush() end up calling flushInternal() which should trigger not only on the forced flag but also on time or threshold, yes?
from fluency.
Oops. Sorry, it looks like I missed another path. Let me check...
from fluency.
Related Issues (20)
- Can we customize the size of data sent? HOT 3
- Exception when ack expected but not received HOT 3
- Some logs are lost after fluentd restart HOT 4
- emit map date issure HOT 2
- ACKs with fluent-bit HOT 8
- Not getting logs on AWS EKS cluster HOT 1
- Cannot use mutual signing TLS HOT 2
- Micrometer metrics HOT 2
- java.lang.NoClassDefFoundError: on Android 6 devices HOT 1
- Introduce strict buffer retention size property HOT 5
- Broken Pipe error after 1 minute of idle HOT 3
- Provide an additional way to configure the RecordFormatter when using the FluencyBuilderForFluentd HOT 4
- Connection reset by peer errors HOT 2
- Need a way to configure fileBackupPrefix of Buffer.Config HOT 3
- Fluency - emit() failed due to buffer full. Flushing buffer. Please try again. HOT 5
- Add support for Unix domain sockets HOT 3
- ackResponseMode true - SocketReadTimeout causes infinite retry HOT 1
- A full buffer should not log any errors HOT 3
- Data loss in the Buffer after throwing an IOException inside `moveRetentionBufferToFlushable()` method HOT 2
- `fluency-fluentd` should expose `fluency-core` via `api` in `build.gradle.kts`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluency.