• Architecture patterns
  • Kubernetes
    • Deploying as a DaemonSet
    • Deploying as a sidecar
  • Host

• Summary of the different ways to deploy fluent-bit to be discussed

• Display a 2 nodes k8s cluster
• Show the config details of the daemonset
• Show the configs in Prometheus deployment
• Publish Prometheus and show the metrics available
• Optionally show the metrics in Grafana

• Show the config details of the sidecar
• Display sample app with a fbit sidecar grabbing logs and sending them to stdout (or external service if prep time permits)

• Installing from packages. It installs as a service
• Show prometheus configuration (target)
• Show metrics in prometheus
• Show the metrics in Grafana

*Prometheus remote write

• Formats

• Plugins

  • Records/bytes
  • Files
  • Drops
  • Errors
  • Retries

• Storage

  • Chunks
  • Memory

• Filters

  • Records/bytes
  • Add/drop
  • Totals

• Others

  • Overlimit
  • Uptime

• Show the dashboard in Grafana
• Show known patterns
  • Show images or run script to simulate
  • How to analyze the case?
  • What can be done? (briefly, this could vary depending on the environment)
• Patterns to show
  • Backpressure
  • Retries (with and without backpressure)
  • Drops
  • Ingestion drop

Cases

• Data is not flowing
• Corrupted and rejected messages
• Formatting is incorrect

Tooling

• Tap/trace
• Chunk inspector

What should be tuned?

• Are we forwarding or processing?
• What are the bottlenecks, limits, goals, constraints, tradeoffs?
• What are the risks, costs, benefits, alternatives?
• What are the assumptions, dependencies, requirements?

• Input
  • Threaded
  • Processors
• Output
  • Workers
  • Storage limits
  • Processors
• Buffering
  • Buffer_chunk_size
  • Buffer_max_size
  • Static files, others
• Filters
• Storage
• Parsers
  • Built in
  • Multiline
  • Regex
• Others