x-cubed / event-store-client Goto Github PK

Hello,

I've tried to use this library to connect angular 5 clients to an eventstore backend, unfortunately I'm running into multiple problems. I've installed the application using npm install event-store-client and inserted import * as EventStoreClient from "event-store-client"; at the top of test-es.ts - the TS file responsible for connecting to the EventStore server (based on the provided example.js file). After building the app with ng serve, I first had a first runtime error when loading the page that calls to test-es.ts:
get http://localhost:4200/ClientMessageDtos.proto error 404 (not found)
I solved this by manually deploying the proto file in .angular-cli.json
Then I run into another runtime error:

ERROR TypeError: net.connect is not a function
    at new Connection (connection.js:49)

Therefore I was wondering if anyone had successfully managed to use this library in an Angular+typescript client-side application.

And btw, thanks for this very interesting library !

Hi, I am using this library to interface with the event store 5.0.8.

I am able to write to the streams and by checking in the web interface I can see the contents correctly, such as the event type and the data.

But when I subscribe, I see that some fields are returned as Buffer. Example of a console.log of one event retrieved shows

{
  streamId: '$ce-mystream',
  eventNumber: 54,
  eventId: 'bad6ce85-edff-4c13-a6fa-63c7f755257a',
  eventType: '$>',
  isJson: false,
  created: 2020-05-10T19:40:23.166Z,
  data: <Buffer ...

I could not find how to have access to the original value

(Disclaimer I'm not sure if this error is coming from event store client, from event store itself, or something completely different that has not yet been identified.)

We have an event handler that throws the following error when it runs for a sufficiently long time:

Error: write after end
    at writeAfterEnd (_stream_writable.js:166:12)
    at Socket.Writable.write (_stream_writable.js:217:5)
    at Socket.write (net.js:650:40)
    at Connection.sendMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:533:21)
    at receiveMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:160:22)
    at Socket.onData (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:81:21)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:172:18)
    at Socket.Readable.push (_stream_readable.js:130:10)
    at TCP.onread (net.js:542:20)
/home/app/tableservice/tableEventStore/index.js:404
            throw err;
            ^

Error: write after end
    at writeAfterEnd (_stream_writable.js:166:12)
    at Socket.Writable.write (_stream_writable.js:217:5)
    at Socket.write (net.js:650:40)
    at Connection.sendMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:533:21)
    at receiveMessage (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:160:22)
    at Socket.onData (/home/app/tableservice/node_modules/event-store-client/lib/connection.js:81:21)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:172:18)
    at Socket.Readable.push (_stream_readable.js:130:10)
    at TCP.onread (net.js:542:20)

If I restart the service running this event handler, it picks up where it died just fine and continues processing events.

Something that might be relevant - the event handler that appears to be the source of this error is subscribed to one stream. When it picks up a certain class of event, it opens up a different stream to read detailed events relating to the subscribed stream.

I'm looking to add this functionality. Are there any technical reasons why this hasn't been done? Or is it simply a missing feature?

Hi @x-cubed,

I wonder if you can provide some insight into making sure the each event is processed before moving to the next one.

( async event => { await eventHandler.handle(event); }),

but it seems to ignore that as I see other events being handled before the previous one finished.

I may be missing something here, but I can't find a way to get the most recent event in a stream without doing two calls.

I would have thought readStreamEventsBackwards with a max number of 1 would do it, but it requires a "fromEventNumber" which prevents me from getting the last one as I don't know the last event number prior to the query. I'm not sure why "fromEventNumber" is a required field on this query... what am I missing?

We are getting subscriptions dropping intermittently using this library. Although we log errors on both connection drop and on Error, the subscription just drops silently, not giving any hint as to why it has stopped receiving events. This doesn't appear to be after any set period of time, sometimes it is almost straight away, sometimes after a few hours. With DEBUG mode on, we can see the heardbeats continue on without fail after the last event before the drop.

Any idea what might be going on and how to debug this further?

Do you plan to implement transaction scope?

I'm curious, @x-cubed. Is there a wishlist for version 1.0.0? Can you post it? My team is beginning to build Node services using this client, and if we have an opportunity to further contribute (and if we find time to do so) we can continue to help.

Looks like its actively zero out metadata in the writeEvent method.

Hi, we love and use your library! One thing I can't figure out though is how to detect a connection error.
We use writeEvents and wait for the success-callback. If the event store went down for some reason I can see a log output of the connection error, but I don't see a way to catch and react to it in my code. Do you have a hint for me?

Hello

Does this client support an SSL connection to Eventstore?

Thanks
Maarten

I was having difficulties on handling circuit breaker when i lost connection due to unforeseen event (e.g., cluster node is not accessible). I tried to use different circuit breaker libraries to catch the "onError" to return the fail attempt but with no luck.

Awesome library. Thanks for building it. We are using TypeScript within our projects and identified that you are also providing a type definition file. Cool! Unfortunately there is a discrepancy between the package.json which lives in the master branch and the one that gets distributed by npm. The one in the master references the type definition file correctly and the actual package on npm does not has a types attribute at all. Would it be possible to publish a patch version 0.0.11 with the correct package.json in it?

Edit: It also seems that the typings within the master are old. Would it also be possible to update them?

The onData and receiveMessage functions in connection.js need to use streams to support large messages.

messageLength: 4294967299
UNCAUGHT EXCEPTION!!!!!
[RangeError: Invalid typed array length]
RangeError: Invalid typed array length
at new Uint8Array (native)
at createBuffer (buffer.js:23:15)
at allocate (buffer.js:98:12)
at new Buffer (buffer.js:53:12)
at Socket.onData (/Users/Corey/GitHub/UpdecSync/node_modules/event-store-client/lib/connection.js:92:47)
at emitOne (events.js:77:13)
at Socket.emit (events.js:169:7)
at readableAddChunk (_stream_readable.js:153:18)
at Socket.Readable.push (_stream_readable.js:111:10)
at TCP.onread (net.js:531:20)

Hi,

There seems to be a small bug in the Typescript definition file for the createGuid method.

I believe this needs to be marked static: like this:

    /***
     * Helper function to create a new v4 UUID to use for event IDs or correlation IDs
     */
    static createGuid(): Buffer;

Hi,

I've been investigating issue where my js code intermittenly stop pushing event to eventstore. Once I put some logging i found these:

[PID:01424:012 2016.08.13 15:07:30.341 TRACE TcpConnectionManager] Closing connection 'external-normal' [127.0.0.1:61047, L127.0.0.1:1113, {7c3c6a87-ec80-404c-b0f5-916f7ef81e72}] cleanly. Reason: HEARTBEAT TIMEOUT at msgNum 3742004
[PID:01424:012 2016.08.13 15:07:30.341 INFO  TcpConnection       ] ES TcpConnection closed [15:07:30.341: N127.0.0.1:61047, L127.0.0.1:1113, {7c3c6a87-ec80-404c-b0f5-916f7ef81e72}]:Received bytes: 284729333, Sent bytes: 6448674876
[PID:01424:012 2016.08.13 15:07:30.341 INFO  TcpConnection       ] ES TcpConnection closed [15:07:30.341: N127.0.0.1:61047, L127.0.0.1:1113, {7c3c6a87-ec80-404c-b0f5-916f7ef81e72}]:Send calls: 3741924, callbacks: 3741924
[PID:01424:012 2016.08.13 15:07:30.341 INFO  TcpConnection       ] ES TcpConnection closed [15:07:30.341: N127.0.0.1:61047, L127.0.0.1:1113, {7c3c6a87-ec80-404c-b0f5-916f7ef81e72}]:Receive calls: 3742005, callbacks: 3742004
[PID:01424:012 2016.08.13 15:07:30.341 INFO  TcpConnection       ] ES TcpConnection closed [15:07:30.341: N127.0.0.1:61047, L127.0.0.1:1113, {7c3c6a87-ec80-404c-b0f5-916f7ef81e72}]:Close reason: [Success] HEARTBEAT TIMEOUT at msgNum 3742004

I found in .NET client library have the functionality:

connectionSettingsBuilder.KeepReconnecting();
connectionSettingsBuilder.KeepRetrying();

Is there a workaround for this issue ? or should I reconnect when onError / onClosed event called ?

Since the catchup subscription is not currently supported, I assume that competing consumer subscriptions are also not supported?

Cannot find facilities for subscription to all events being written to EventStore. In C# official client we have SubscribeToAllFrom(lastCheckpoint, ...) for such purposes.

Are there any alternatives in event-store-client?

In the type definition:
subscribeToStreamFrom(streamId: string, fromEventNumber: number, ...)

Should be:
subscribeToStreamFrom(streamId: string, fromEventNumber: number | null, ...)

It is missing the | null, for the fromEventNumber parameter. Without it, you cannot read from the beginning of the stream.

How does one start a catchup subscription through this client?

After a few hours I always get the error

Error: This socket has been ended by the other party
    at Socket.writeAfterFIN [as write] (net.js:278:12)
    at Connection.sendMessage (/.../node_modules/event-store-client/lib/connection.js:504:21)
    at Connection.writeEvents (/.../node_modules/event-store-client/lib/connection.js:423:10)
    at /.../eventstore.js:43:18

My code (the line 43 from the stacktrace is the one with eventStore.writeEvents):

let connected = false;
const eventStore = new EventStore.Connection({
  host: config.eventStore.host,
  port: config.eventStore.port,
  debug: config.eventStore.debug,
  onError: (error) => {
    connected = false;
    logger.error(error);
  },
  onConnect: () => {
    logger.info('connected to eventstore');
    connected = true;
  },
});

function emitToEventStore(message) {
  if (!connected) {
    return Promise.reject(new Error('EventStore not connected'));
  }
  return Promise.resolve()
  .then(() => {
    const event = {
      ...
    };
    return new Promise((resolve, reject) => {
      eventStore.writeEvents('streamname', EventStore.ExpectedVersion.Any, true, [event], null, (result) => {
        if (result.message) {
          return reject(new Error(result.message, result));
        }
        return resolve();
      });
    });
  })
  .then(() => logger.info(`posted message # ${message.properties.messageId} to eventstore`))
  .catch((error) => {
    logger.error(`could not post message # ${message.properties.messageId}`, error);
  });
}

Is there any way to react to a disconnect?

The scenario was ... (am migrating crud sql to aggregates)
using one connection
Maybe 35 rows of data that - per row - converts into 8 calls into an aggregate root. This in turn would per call load 1-2 aggregates (no more than 8 events per stream) by reading stream forwards and then potentially write an event to each of those streams.

If i limit the rows of data to about 24 it all works - at this point i have put nothing in to monitor the number of concurrent read stream forwards i am doing.
If i operate on the rows sequentially rather than a ton of promises in parrallel it all works.

The problem is not the failure - its the lack of feedback.

When i fails i do get onClose(false) getting called at the connection level - but nothing that causes each aggregate to fail. The callback simply isn't made as far as i can tell.

Still investigating. Posted this incase someone can point out how i'm being silly!
Otherwise i guess a solution would be to end up adding some kind of timeout into the read forward subscription chunks?

Noticed that when consuming events from a stream built by a projection, with the "resolveLinkTos" option set to true, the deserializer appears to ignore what's in the link: { } structure entirely. This means that I can't get at the event numbers from the projection stream, just the ones from the stream that the link-resolved event comes from. So there's no way to track sequential event numbers in a projection stream if I also want the linked events resolved.

Relevant code: lib/connection.js - unpackResolvedEvent(resolvedEvent) -> return unpackEventRecord(resolvedEvent.event);

See Google Groups issue I raised: https://groups.google.com/forum/#!topic/event-store/MIsb87kjvzE