The long and short is that if you need to ensure your data in your
log messages is "never" lost, logs are not the best place for this
data. Rather you should put this information in a more formal
datastore like a database. Such systems make different
decisions/trade-offs ...
-- Alex Jackson
https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/xRi_TNo58aE/5itmAuQfQp4J
- "Realtime"
- No overhead
- Don't block
- Minimise noisy neighbour
- Favour recent over historic
- Every component makes tradeoffs
- Simplicity / operability / speed
correctness
logger.info(
String.format("Funds updated successfully for account: %s",
order.getAccountId()));
return repository.save(order);
- Don't block
- UDP - fire and forget
- TCP with (truncating) buffer
- Doppler uses UDP by default
- Doppler max_buffer_length
- Sharded - failure loses 1/shard % of data
- Lose order
- Optimised for speed, not resiliency.
- Errors - dropped
- Write throughput
- Not ACID / Eventual consistancy
- Replication for speed
- Statistical counts
- 14 - 30 days; then deleted
- No backup operational budget
- ALL components of log pipline optimised for
- Low overhead
- Low cost
- Availability
- Speed
- ==> NOT A TRANSACTIONAL DATABASE!
-
YES
- Log message searching
- Trending up / down
- Percentile measurements
- Approx counts
-
NO!
- Financial records
- Compliance audit trails