Log client IP address as a custom X-FORWARDED-FOR header. See https://github.com/pbojinov/request-ip

Research and create middleware that works from the client side, to record calls made to a remote API.

axios.interceptors.response.use(new AxiosMiddleware(url, rules));

axios.interceptors.response.use((response) => {
    return response;
  }, (error) => {
    let response = error.response;
    console.log(response.config);
    console.log(response.request);
  });

Add support for custom fields. This introduces breaking changes since the HttpMessage.send signature must be modified to accept custom fields. The custom fields themselves should be added to the message after rules have been applied in HttpLogger.submit_if_passing. See the logger-go implementation.

Use raw deflate compression by default, add skip_compression runtime property

Found this while testing with a local listener like this: http://localhost:5000/messages

This is breaking on npm version 5, and we barely depend on this. Let's copy out the code on which we depend and remove the module dependency (but keep the license notice and clearly mark the copied code as originating from valid-url).

The logger instance maintained by HttpLoggerForExpress is private but should be exposed through an accessor.

Update copyright & dependencies

Sometimes the node logger breaks like this under test:

Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

• check that logger is enabled before doing anything
• use format/submit over log (to avoid one extra enabled check)
• format code to better match HttpLoggerForServlets

Add message batching to the background submission feature, using NDJSON. We can follow the logger-go implementation, in order to keep loggers consistent across languages.

Returns true if logger can ever be enabled

Create middleware for NodeJS lambdas on AWS and other serverless environments.

The example that uses mock implementations in API.md results in an incomplete message that is ultimately dropped by the Resurface DB. This is due to the request_url field being skipped as both request.protocol and request.hostname are not present in the mock request instance.

How to reproduce?

// define request to log
const request = new HttpRequestImpl();
request.method = 'GET';
request.url = 'http://resurface.io';

// define response to log
const response = new HttpResponseImpl();
response.statusCode = 200;

// build JSON message
let msg = HttpMessage.build(request, response);

console.log(msg) // no request_url detail present in JSON

HttpMessage.send(logger, request, response); // this results in the request down below

//POST /message HTTP/1.1
//Content-Encoding: identity
//Content-Type: application/json; charset=UTF-8
//User-Agent: Resurface/2.2.1 (http_logger.js)
//Host: localhost:7701
//Connection: keep-alive
//Transfer-Encoding: chunked

//[["request_method","POST"],["response_code","200"],["now","1708555058014"],["host","MYPCNAME.local"]]

Parsing the given URL inside the HttpRequestImpl setter for the url field and setting protocol and hostname fields from there might help solve this issue.

This would allow JSON generation to be separated out for easier profiling.

Not yet adding these in BaseLogger.format()
Should guard against sending empty request/response bodies

Support monitoring APIs built on Flask, using a middleware plugin similar to Express.

Using periods as separators in JSON keys makes parsing with regular expressions more difficult than it needs to be (since period is a special character and has to be escaped).

Have integration tests for this but not local unit tests

Order params based on frequency of data:
request, response
request, response, response_body
request, response, response_body, request_body
request, response, response_body, request_body, now

SharedArrayBuffer has not been supported for a while in all major browsers (unless the resource is cross-isolated).

Normally this is not an issue for back-end services and the corresponding middleware. However, for browser-based apps that use node as runtime environment -such as chromium-based electron- the SharedArrayBuffer variable is no longer available. This results in an error when trying to initialize a logger instance. As an example, see Kong/insomnia#5078

BaseLogger init should take a timeout argument that is set to a low value by default (5 sec).

Content-Encoding header is set to deflated instead of identity when skip_compression rule is present as part of options.rules but not options itself. This is not the expected result according to the logger API.

Expected behavior

let logger = new HttpLogger({url: 'http://localhost:7701/message', rules: 'include debug\n\nskip_compression'});
console.log(logger._url_options.headers);
//{
//  'Content-Encoding': 'identity',
//  'Content-Type': 'application/json; charset=UTF-8',
//  'User-Agent': 'Resurface/2.2.1 (http_logger.js)'
//}

Current behavior

let logger = new HttpLogger({url: 'http://localhost:7701/message', rules: 'include debug\n\nskip_compression'});
console.log(logger._url_options.headers);
//{
//  'Content-Encoding': 'deflate',
//  'Content-Type': 'application/json; charset=UTF-8',
//  'User-Agent': 'Resurface/2.2.1 (http_logger.js)'
//}

How to reproduce?

const { HttpLogger } = require('resurfaceio-logger');

let logger = new HttpLogger({url: 'http://localhost:7701/message', rules: 'include debug\n\nskip_compression'});

assert.equal(logger.skip_compression, true);
assert.equal(logger._url_options.headers['Content-Encoding'], 'identity');  // fails

The header can also be inspected with a packet capture tool like wireshark, or an HTTP proxy.

Content to match other loggers

Our CodeFactor score is C+ due to a couple of code-formatting issues, which should be easy to resolve.

We should be checking for null in a lot of places where we're checking for undefined.

• passing {url: null} to BaseLogger constructor
• passing null param to request_body or response_body to HttpLogger.format

https://www.davidbcalhoun.com/2011/checking-for-undefined-null-and-empty-variables-in-javascript/

(rather than a single URL for all demo customers)

Simplify HttpLogger internals by pulling key/value generation out into new HttpMessageImpl class

Unit tests don't seem to break regardless of what status code is matched in the submit method

Log both query and form parameters as request parameters

Apparently express-session generates cookie headers that are string arrays, and these are not joined but passed along as JSON array elements, which leads to invalid JSON.

const session = require('express-session');
app.use(session({ secret: 'this-is-a-secret-token', cookie: { maxAge: 60000 }}));

Send POST requests on a separate thread with bounded queue. Introduce BaseLogger.maxQueueDepth (with default value of 128) to control the depth of the bounded queue before the response is blocked.

BaseLogger.submit is where the handoff to the background thread should be done. (Rules engine and JSON conversion will continue to be done in the calling thread)

BaseLogger needs only a single background thread for performing POST requests since a single CPU core is normally capable of saturating the network.

Use async and axios libraries to do POST submissions asynchronously on a separate thread.

const async = require('async');
const transmitQ = async.queue((task, callback) => {
    send_message(task.message, callback);  // poll until queue is reasonably sized?
}, 1);

const axios = require('axios');
function send_message(message, callback) {
    try {
    axios.post(url, message.toString())
        .then(function (response) {
            if (response.status != 204) console.log("ARRRRHHHHHHHHHHHHHHHHHHHHHHHHHHHHH");
            callback();
        })
        .catch(function (error) {
            callback();
        });
    } catch (e) {
        callback();
    }
}

Similar to rack middleware but different API