datalust / seq-logging Goto Github PK

Exit logging does not work on pino-seq which is crucial for production code. This causes important error logs to get lost on unhandled exceptions. When using exit handlers I get the following error:

Error: final requires a stream that has a flushSync method, such as pino.destination and pino.extreme

These issues pinojs/pino-pretty#37 and pinojs/pino#542 seem to be related. We need seq-logging to support synchronous requests for this to work so we can call it from flushSync implementation in pino-seq and probably other implementors.

When using seq-logging on a client side application, the flush doesn't work in unload or pagehide events. The sequence of promises and waiting doesn't actually get waited for and the network event doesn't start.

An alternative is to get a final POST data and set up a navigator.sendBeacon(url, data). There is no opportunity to set headers, but setting the url to use ?apiKey= should work. I just need the data string out of the SeqLogger somehow.

What are your thoughts?

Make it possible for client libraries to send these values.

https://docs.datalust.co/docs/built-in-properties-and-functions

The recently published version doesn't package the code correctly, e.g. missing the seq_logger.js file that is imported via index.js:
https://www.npmjs.com/package/seq-logging?activeTab=code

The code is published and working correctly for earlier versions, e.g.:
https://www.npmjs.com/package/seq-logging/v/2.0.0?activeTab=code

AC:
As a developer using VSCode (or any TS-aware IDE), I would like the intellisense for functions and class methods (e.g. seq_logger.js) to be displayed with the accompanying comments/jsdocs. This will save me having to look into the source code for definitions, e.g. what is the difference between flush and flushToBeacon

Notes:
The package seq-logging definition file (node_modules/seq-logging/index.d.ts) does not have the SeqLogger method comments.

I think perhaps this problem is two-fold:

1. The methods on node_modules/seq-logging/seq_logger.js are not using the jsdoc format
2. The type definition file generation process (tsc index.js --declaration --emitDeclarationOnly) should then be run

Ever since webpack 4->5 removed node polyfills for builtin packages like url, http and https, there are a number of hoops you need to jump through in order to be able to use packages that use them in the browser.

Any chance you are considering a browser-native version of the client?
I suppose you could release a client that just uses the polyfills locally?

Hi Nicolas

We have some code that handles requests from a set of services. It doesn't know what type of service it's handling for and before it does something logs (through bunyan and bunyan-seq) the objects it'll work on.

This is fine (in-general) except for edge cases where the messages will not ever got to seq because of the max limits. In the code below, it first JSON.stringifys the object to do a length detection. In our edge-cases, the objects are big enough that the stringify blocks the thread for a significant time.

How could we handle this better?

https://github.com/datalust/seq-logging/blob/master/seq_logger.js#L180

An event with properties that have circular structures makes _dequeBatch fail every time, stopping all logging.

Failing code: https://github.com/datalust/seq-logging/blob/dev/seq_logger.js#L212

I propose that in the case JSON.parse fails we try to build a new object but with a string indicating circular structures.

Example event:

const a = {};
a.a = a;
const event = {
  level: "Error",
  timestamp: new Date(),
  messageTemplate: 'Circular dependency issue!',
  properties: { a }
};`

See datalust/bunyan-seq#47

seq-logging causes Node to publish a MaxListenersExceeded warning because it registers a 'timeout' event handler each time the request object emits a 'socket' event.

node: 8.7.0,
bunyan: 1.8.12,
bunyan-seq: 0.2.0

I want to log to a local Seq instance which is using a self-signed certificate. The only way to get it working now is to add NODE_TLS_REJECT_UNAUTHORIZED=0 but I want to avoid this at all cost.

Would it be possible to add an option so we could enable (or actually disable) the strictSSL option in the Request library?

The following PR #64 has removed the functionality introduced by #23. Which potentially allows seq to control the logging level.

Was this intended?

Hello i am getting the following error while trying to run a script with seq-logging attached

seq_logger.js:9 const HEADER_FOOTER_BYTES = (new SafeGlobalBlob([HEADER])).size + (new SafeGlobalBlob([FOOTER])).size; ^

TypeError: SafeGlobalBlob is not a constructor at Object.<anonymous> (node_modules\seq-logging\seq_logger.js:9:30) at Module._compile (internal/modules/cjs/loader.js:1063:30) at Module._extensions..js (internal/modules/cjs/loader.js:1092:10) at Object.require.extensions.<computed> [as .js] (\node_modules\ts-node\src\index.ts:1608:43) at Module.load (internal/modules/cjs/loader.js:928:32) at Function.Module._load (internal/modules/cjs/loader.js:769:14) at Module.require (internal/modules/cjs/loader.js:952:19) at require (internal/modules/cjs/helpers.js:88:18) at Object.<anonymous> (node_modules\seq-logging\index.js:3:14) at Module._compile (internal/modules/cjs/loader.js:1063:30)
printFatalErrorUponExit                               	source-map-support.js:726
process.emit.sharedData.processEmitHook.installedValue	source-map-support.js:753
exit                                                  	per_thread.js:169
process.emit.sharedData.processEmitHook.installedValue	source-map-support.js:750
(anonymous function)                                  	execution.js:163

I have no idea what is this and google is not helping. Any idea how tor resolve this?

Hi, we are using pino and pino-seq library. Performance tests show that opening and closing TCP connections impacts server performance negatively in 2 ways:

1. Unnecessary CPU overhead.
2. Diminishes available TCP ports for the server (connections closed are not available to the OS immediately).

Do you plan to allow via configuration to keep sockets around even when there are no outstanding requests, so they can be used for future requests without having to reestablish a TCP connection like creating a custom http.Agent with keepAlive and other required options?
Currently it sets HTTP header Connection to close.

If you have a fork or local clone from before 7 Dec 2020, you may need to run the following commands to update your forked/cloned repo.

//you might need to do: git fetch origin
git checkout master
git branch -m master main
git fetch
git branch --unset-upstream
//if fork: git push -u origin main
git branch -u origin/main
git symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/main

Currently, batches of log events that cannot be sent to Seq for any reason will be dropped.

If the returned status code from the HTTP request indicates that the server is down (0) or unavailable (503) then some number of retries should be attempted.

Ref: datalust/bunyan-seq#1