hemerajs / hemera-nats-streaming Goto Github PK

Sorry if this post comes out rude in any way, but we tried to use this plugin in our upcoming project (given that it's mentioned in the Hemera documentation) and it turned into hours of trying to make sense of the things that we were seeing in logs. When I finally went looking into the source code I was surprised to say the least, and still have no idea how it's supposed to be working with more than one service. I'll give an example to illustrate what I mean.

From what I gather, this plugin more or less works in the following way (please correct me if I'm wrong at any point):
For publishing:

• On connect, add a Hemera handler on natss topic with publish command
• In this handler, emit a NATS streaming event with a subject passed as a parameter

For subscriptions:

• Create a NATS streaming subscription when hemera.natss.add method is used
• When an event is received from NATS streaming, execute Hemera.act with natss.event-subject topic from a subscription handler
• Events can be listened by adding a Hemera handler on natss.subject topic

Now, let's imagine that we have 3 independent services, all using Hemera with Hemera-nats-streaming plugin. Let's call them A, B and C. An event with a subject user-created is published by A, while B and C wish to subscribe to it.

1. On B and C we create a NATS steaming subscription via

hemera.natss.add({
  subject: 'user-created'
})

and then add a handler via Hemera

hemera.add({ topic: 'natss.user-created' }, async () => { })

2. On A, we publish the message with

hemera.act({
  topic: 'natss',
  cmd: 'publish',
  subject: 'user-created',
  data: { }
})

At this point strange things start to happen:

3. Random service (A, B or C) handles the Hemera call and emits the actual event to NATS streaming

I don't know why events are first passed through NATS with Hemera before being published in NATS streaming (is this a Hemera plugin limitation?), but since all services have handler on the same natss topic, it gets routed to a randomly selected one, instead of the one that tries to emit an event in a first place.

At this point it still works correctly, even though the network traffic is doubled and it's very confusing to debug (for example, event handler error logs end up in random services).

4. B and C both receive the event inside the NATS streaming subscription in the plugin

Looking good at this point, every service received the event just as intended

5. From a plugin, B and C both make a Hemera request that contains the event with natss.user-created topic

At this point all hell breaks loose. We have two services listening on a topic natss.user-created. This means that any of the following scenarios can happen, at random:

• A receives the event twice, B never gets it
• B receives the event twice, A never gets it
• Both A and B get the event once

You can imagine how difficult it was to debug, given that it works correctly (though not efficiently) with a single listener for a given event, but breaks as soon as you add a second one and only get worse with more.

It seems less like a bug and more like a flaw in a way that a plugin is designed, so I don't know what the solution should be here. Things that comes to my mind are:

• if we have completely misused this plugin somehow, please make it clear in the documentation on how it should be used with more than one service
• if it's not meant to be used in production, please make it explicit in the Hemera documentation

If it's possible to intercept Hemera in a way that would allow passing events in memory (both when publishing and subscribing) I think it would resolve most of the issues, but I don't know if Hemera allows it in a first place.

Hello,

I just realized that node-nats-streaming is fixed to 0.0.28 ( Semver fixes exact version for ^0.0.x , ref: https://nodesource.com/blog/semver-tilde-and-caret/#caretmajorzero ). Can we safely update to 0.0.41 (latest released version)?. This would help solve nats-io/stan.js#73 among others.

Hello,

I'm trying to perform some tests using Jest.

Jest returned an error about open Handlers, and after some debugging it seems that a nats connection is left open.

I tried to check if there was a way to close nats connection inside hemera-nats-streaming, but it seems that the onClose handler does nothing:
https://github.com/hemerajs/hemera-nats-streaming/blob/master/index.js#L35

Shouldn't this function close the connection to nats server? Is there any other way to gracefully shutdown the connection?

Error when use plugins in hemera, ERR_AVVIO_PLUGIN_TIMEOUT: plugin did not start in time.