Proposal: exposing the option to parallelize actions/triggers about sailor-nodejs HOT 4 OPEN

• read an optional parallelize: bool field in from component.json inside every action/trigger
• if this field exists, read in a number from component.json that represents the number of messages that should be read in parallel at once. This number should default to a value greater than 1 (2?) if not set. It should also be able to be configured as a config field by a user, as it would be dependent on the amount of computing power available

Or just have a single number that defaults to 1 if not present.

from sailor-nodejs.

It's a good idea to optimize the processing, however drawbacks of this approach are:

1. Increase memory usage - we need to keep multiple (potentially large) messages in memory
2. Decreased reliability in case of container failures - instead of potentially re-processing a single message (e.g. when container failed before rabbitmq ack.) we will have to re-process potentially multiple messages

Taking drawbacks into account I don't' think the component developer can make this decision

from sailor-nodejs.

I potentially also makes sense to allow this value to be dynamically configured at runtime based on a specific cfg. Perhaps add a way to set this in either the init() or startup() functions?

from sailor-nodejs.

Increase memory usage - we need to keep multiple (potentially large) messages in memory

Currently, our platform doesn't really support messages above ~3.5 MB in side. So even if we fetched 10 messages in parallel, at most 35 MB of RAM would be consumed which is still less than the default 256 MB allocated to a component.

Decreased reliability in case of container failures - instead of potentially re-processing a single message (e.g. when container failed before rabbitmq ack.) we will have to re-process potentially multiple messages

I don't understand why the costs of doing this would be a significant concern.

from sailor-nodejs.