marceloboeira / crowd Goto Github PK

It would be essential to have a external configuration file that we could use to setup the route/sink adapters maps:

routes:
  - path: "/foo"
    sink: 
      adapter: redis
      uri: "redis://[USER]:[PASSWORD]@[IP]:[PORT]"
  - path: "/bar"
    sink: 
      adapter: sqs
      ...

then, we could have something like:

crowd -f crowd.yml

Which crowd.yml could be the default ...

@mauricioabreu I'll take care of this, I just wanted to figure if there is anything we should look for on matrix of tests (linux specific or even, go versions?) I'm really outdated when it comes to Golang versions and dependency management... it changes faster than javascript frameworks ahhah

Is it worth to add support for other backends? It could be achieved using go interfaces.
We could start with redis, writing a code that is easy to use different queue vendors.

• Test current SQS backend
• Implement a new backend like redis

Add support to a sink buffer, to avoid round-trips on every incoming request.

The buffer should be configurable through the configuration file: #3.

The buffer should have a configurable timeout too, something like, "100 messages OR 1 second" and it could be adjustable per sink.

routes:
  - path: "/foo"
    sink: 
      adapter: redis
      uri: "redis://[USER]:[PASSWORD]@[IP]:[PORT]"
      buffer:
        size: 100
        timeout: 1s

document that sink buffers are volatile and shouldn't be used for situations that there is no tolerance for message dropping.