Comments (3)
Thanks!
Time-based flushing is possible. However, things to watch out for would be:
- Uneven distribution of log events between batches, depending on program behavior.
- In some cases, there may not even be anything to flush if the period was short enough and/or the program is quiet enough.
- The flip side of that is if the period were set too long, we risk getting OutOfMemoryError if the program produces log events too much too fast.
- At least one more background thread is needed to periodically flush.
I can try to implement one given the caveats above. I guess the user will need to monitor the host app's logging behavior to find a good setting to use, keeping in mind that there may be some unpleasant surprises if it's configured incorrectly. Size-based flushing doesn't have this problem.
Lastly, I will probably refactor an interface from the current LoggingEventCache to have a hierarchy of EventCache implementations. One will be based on size (the current behavior), and one will be based on time. This will keep the classes simple and specific. (I hesitate to call them SOLID, though.)
from log4j-s3-search.
That would be nice if you'll implement it.
I tried messing with LoggingEventCache for about two hours yesterday.
I tried to set the log files to flush if the eventQueueLength >= capacity OR a specified time interval from the latest flush had elapsed.
(not exactly the same approach as you described above)
Unfortunately, whatever I tried when testing with s3loggersample, the last log would fail to flush and the program seemed to hang and never terminate.
Possibly a deadlock or something.
from log4j-s3-search.
#15 has the first rev of the support for time-based flushing. I'll perform some functional testing before merging.
from log4j-s3-search.
Related Issues (20)
- Minio support. HOT 3
- AWS S3 - lack of data compression HOT 3
- Thread Count - keep increase HOT 2
- Cannot end publish with com.van.logging.azure.BlobPublishHelper HOT 3
- System out in S3PublishHelper.java HOT 2
- Cannot end publish with com.van.logging.aws.S3PublishHelper HOT 13
- Thread pool executer shutdown not being called. HOT 3
- Compatibility with / Benchmarking with Asynchronous logging HOT 1
- Security dependencies bump HOT 2
- Call to Exception.printStackTrace() causes Log4j2 to split stacktrace logs HOT 2
- Support AWS IRSA. HOT 3
- Flushing out remaining events in the cache on shutdown HOT 15
- Refresh "s3Path" setting for S3Appender based on configuration HOT 13
- spark-submit hangs after successful job completion HOT 11
- LoggingEventCache shutdown hangs with <stagingBufferAge> set HOT 6
- Issues preventing logging to storage grid HOT 11
- While using the log4j2.xml configurations for Gcp bucket I am getting invalid attributes error HOT 7
- Error processing element Log4j2Appender ([Appenders: null]): CLASS_NOT_FOUND HOT 5
- Elastic Search HOT 9
- Support Property Substitution on Path/Prefix. HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from log4j-s3-search.