Comments (4)
I was able to replicate the DataAlreadyAcceptedException by calling the PutLogEvents API twice using the same sequence token. The message
properties need to be exactly the same otherwise it returns a InvalidSequenceTokenException error, but interestingly the timestamp
properties can be different. If the first call is successful, the second request should respond with the DataAlreadyAcceptedException with retryable
as false
.
from bunyan-cloudwatch.
The issue is quite old and we have been happily using our forked version of the module for some time therefore I have not looked into this issue for some time, so please forgive me if this response seems a bit naive but given that it is probably possible for properties to be exactly the same for a message, surely if the timestamp
is different it should be treated as a new message? In any case, I feel like there should be a method or optional flag to choose whether to make these responses retryable
using the new sequenceToken
.
from bunyan-cloudwatch.
@clocked0ne Just took a quick look at the commits in your fork. Has your change resulted in duplicates?
I did a bit more testing and I think what happens is 1) there is a network issue while receiving the success response from the AWS API; 2) the AWS SDK automatically resends the request (up to 4-5 times) on network errors; 3) the AWS API responds with DataAlreadyAcceptedException.
If this is the case, I believe it would be safe for bunyan-cloudwatch to ignore the error and not resend the log events.
The only caveat would be that it would be better if the messages still retained the timestamp (see #17), since it would decrease the chance of messages matching by chance if multiple emitters are concurrently putting log events to the same LogStream.
from bunyan-cloudwatch.
Hi @amekkawi Thanks for taking the time to look at this, I think that seems like a sensible conclusion - these errors should probably be ignored if the timestamp is retained and used to cross-correlate the potentially duplicated request.
As I mentioned, I haven't looked at this in some time so couldn't categorically state if we have had any duplicates as a result, I will see if I can find time to investigate our Cloudwatch logs for any possible duplicates based on your reasonable assessment of what is happening.
from bunyan-cloudwatch.
Related Issues (15)
- "InvalidSequenceTokenException" thrown when used with multiple instances HOT 8
- Batch Logs HOT 1
- Error: Timestamp is null HOT 8
- No timestamp in Clouwatch logs HOT 1
- return in async functions?
- Loss of logs
- Possible v3 of bunyan-cloudwatch HOT 5
- JSON.stringify crashes application HOT 3
- Implement writable._writeev() to remove own buffer implementation
- Chunk logs to avoid hitting 1MB limit in putlogevents HOT 4
- Error thrown if optional dependency safe-json-stringify is not installed
- Logging error handling HOT 3
- Timeout on AWS Lambda HOT 5
- OperationAbortedException: A conflicting operation is currently in progress against this resource. Please try again. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bunyan-cloudwatch.