msiebuhr / node-statsd-client Goto Github PK

Any chance I help with merging with the other 4 modules I found?

https://github.com/msiebuhr/node-statsd-client
https://github.com/sivy/node-statsd
https://github.com/spreaker/nodejs-statsd-client
https://github.com/dscape/lynx

https://github.com/msiebuhr/node-statsd-client/blob/master/lib/EphemeralSocket.js#L24

Is there any purpose behind this? Ideally you should be able to log and debug what's going on when debug is set to true.

/foo/bar → foo_bar
/foo/bar/ → foo_bar_

While the trailiing slash does sometimes carry semantic significance, it mostly doesn't, and just creates two metric series for the same thing.

Debug output from statsd server:

1 Nov 14:12:46 - DEBUG: prefix.[object Object].response_time:3132|ms

This is how I'm using the module:

'use strict';

const SDC = require('statsd-client');
const sdc = new SDC({debug: true, prefix: 'prefix'});

sdc.increment('working');

const metricsMiddleware = sdc.helpers.getExpressMiddleware({
  timeByUrl: true
});

  const app = express();
  app.disable('x-powered-by');
  app.use(cors);
  app.use(bodyParser.json());
  app.use(bodyParser.urlencoded({extended: true}));
  app.use(metricsMiddleware);

Hi,

please have a look at the following code:

run();
async function run() {
    //..... doing some async / await stuff here .....//

    const sdc = new SDC({ host: appConfig.statsdHost, prefix: `test` });
    sdc.increment("testcounter", 1);

    sdc.close();
    process.exit();
};

This will cause that stats will not be sent to the server. Is there any way to await for the socket until everything is done?

When batch-programs are done, it would be nicer to call a .close() to close the socket and kill the timer, rather than having to wait ~1½ cycles of the timer for the program to end.

Create a handler of sort, that makes it easy to measure what's going on in streams;

var measureStream = sdc.getStreamMetric('key.to.report'); // Return readable+writable stream
input.pipe(measureStream).pipe(output)

This would then report how much data goes from input to output via the counter key.to.report.

An advanced version would allow one to do custom stuff on stream events; eg data, end, error and so on:

var mS = sdc.getStreamMetric('key.to.report', {
    data: function (sdc, data) { sdc.increment('data', data.length); },
    end: function (sdc) {sdc.increment('end'); }
});
...

Which would count all passing data in key.to.report.data and how many connections that end in key.to.report.end.

Hey,

Just wondering if you know how this compares to https://github.com/sivy/node-statsd as I saw you've been active on the issue tracker over there.
what would you recommend for end users?

I am receiving this error intermittently. Seems to happen most frequently when a request takes a long time. I believe the bug is in lib/statsd-client.js lines 117-126.

StatsDClient.prototype._send = function (data) {
    // Create socket if it isn't there
    if (!this._socket) {
        this._socket = dgram.createSocket('udp4');

        // Start timer, if we have a positive timeout
        if (this.options.socket_timeout > 0) {
            this._socket_timer = setTimeout(this._socket_timeout.bind(this), this.options.socket_timeout);
        }
    }

It seems like you are expecting code within the conditional to block, but when it doesn't it reaches the 'send' and then you receieve the error 'cannot call method send of null'

I'm expecting statsclient could emit some events that would help developers to observer its status.

E.g:

sdc.on('connect', (): void=>{});
sdc.on('data', (): void=>{});
sdc.on('error', (): void=> {});
sdc.on('close', (): void=>{});

workflow-regression_1 | TypeError: Cannot convert a Symbol value to a string
workflow-regression_1 | at Socket. (node_modules/zone.js/dist/zone-node.js:1665:52)
workflow-regression_1 | at enqueue (dgram.js:498:10)
workflow-regression_1 | at Socket.send (dgram.js:633:5)
workflow-regression_1 | at /opt/haven/app/node_modules/statsd-client/lib/EphemeralSocket.js:182:22
workflow-regression_1 | at EphemeralSocket._createSocket (node_modules/statsd-client/lib/EphemeralSocket.js:106:16)
workflow-regression_1 | at EphemeralSocket._send (node_modules/statsd-client/lib/EphemeralSocket.js:174:10)
workflow-regression_1 | at EphemeralSocket._flushBuffer (node_modules/statsd-client/lib/EphemeralSocket.js:150:10)
workflow-regression_1 | at EphemeralSocket._enqueue (node_modules/statsd-client/lib/EphemeralSocket.js:138:14)
workflow-regression_1 | at EphemeralSocket.send (node_modules/statsd-client/lib/EphemeralSocket.js:161:14)
workflow-regression_1 | at StatsDClient.counter (node_modules/statsd-client/lib/statsd-client.js:82:18)
workflow-regression_1 | at StatsDClient.increment (node_modules/statsd-client/lib/statsd-client.js:91:10)

suggest code change, from:
EphemeralSocket.prototype._send = function _send(data) {
debug("_send(", data, ")");
// If we don't have a socket, or we have created one but it isn't
// ready yet, we need to enqueue data to send once the socket is ready.
var that = this;

this._createSocket(function () {
    that._socketUsed = true;

    // Create message
    var message = new Buffer(data);

    debug(message.toString());

    that._socket.send(message, 0, message.length, that._port, that._hostname);
});

};

to:

EphemeralSocket.prototype._send = function _send(data) {
debug("_send(", data, ")");
// If we don't have a socket, or we have created one but it isn't
// ready yet, we need to enqueue data to send once the socket is ready.
var that = this;

this._createSocket(function () {
    that._socketUsed = true;

    // Create message
    var message = Buffer.from(data);

    debug(message.toString());

    that._socket.send(message.toString(), 0, message.length, that._port, that._hostname);
});

};

See http://docs.datadoghq.com/guides/dogstatsd/

In particular they have histograms (key |h), allows sampling all inputs and has support for key/value tagging (|#key:value;key2).

If we want to broaden the scope a bit, events and service-checks could be supported too.

When running just npm test, the StatsDClient-tests fail. But when the Ephemeral-socket tests are skipped, the very same tests work fine.

I suspect I've messed up something in either EphemeralSocket-tests or FakeServer.

It would be nice to have support for an automatically added prefix, eg. to put appname.subsystem in front of all sent metrics.

The express-helper is quickly becoming a somewhat complex part of the code-base, and is completely devoid of testing. (But, as we've found out, not bugs...)

Hi

I have installed nodejs - statsd Client on server 1. Graphite installed on Server 2

What are the changes required to display server1 statsd UDP points on Server 2 graphs ??

Well this is not an issue. Its a query. Please suggest..

Either explicitly or using something like Nagle's algorithm.

The timeout-system in the ephemeral socket uses setTimeout(...) to check if it's not used. This should be changed to a setInterval(...).

Both with the project as-is, and the ideas I (and, probably, others) have for new features, there really should be a test-suite.

Ideally, a small faux statsd-server would be ideal, but other ideas are welcome too.

There are regular PR's to support different variants of the statsd protocol:

• InfluxDB: #74
• "Datadog-style" (#69) which ex. Prometheus also seem to support. Has some slightly different names from "standard" statsd.
• Telegraf (#76), which apparently requires sample-rates on some types.

It's all getting quite fragmented, and we can't possibly support all of it without having some kind of strategy.

So: What servers are popular out there (i.e. which should we support) and any ideas besides throwing more arguments on the functions we have?

statsd-client

For whatever reason, the 0.15 module currently on NPM does not have the fix from commit 2273d5a. This prevents the node process from terminating:

var Client = require('statsd-client');
var stats = new Client({
  host: '',
  port: null

});
stats._ephemeralSocket.log = console.log;

stats.increment('blah');
stats.close();

This code from the readme:

var sdc = new require('statsd-client')({host: 'statsd.example.com'});

var timer = new Date();
sdc.increment('some.counter'); // Increment by one.
sdc.gauge('some.gauge', 10); // Set gauge to 10
sdc.timing('some.timer', timer); // Calculates time diff

sdc.close(); // Optional - stop NOW

doesn't work:

$ node test.js 

test.js:4
sdc.increment('some.counter'); // Increment by one.
    ^
TypeError: Cannot call method 'increment' of undefined
    at Object.<anonymous> (/Users/josephg/src/livedb/test.js:4:5)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:901:3

The problem is this new require('statsd-client')({host: 'statsd.example.com'}); is actually interpretted as (new require('statsd-client'))({host: 'statsd.example.com'}); or something like that, and it returns undefined.

This works:

var SDC = require('statsd-client');
var sdc = new SDC({host: 'localhost'});

var timer = new Date();
sdc.increment('some.counter'); // Increment by one.
sdc.gauge('some.gauge', 10); // Set gauge to 10
sdc.timing('some.timer', timer); // Calculates time diff

sdc.close(); // Optional - stop NOW

Just tried to define routes like:

.get(
      [ '/aroute', '/broute', '/croute'],
      (req, res, next) => doAThing();
    )

According to http://expressjs.com/de/api.html#path-examples arrays are supported. Sadly I get an error in here: https://github.com/msiebuhr/node-statsd-client/blob/master/lib/helpers/getExpressMiddleware.js#L4

We need to collect statistics in several corners of our app; we should have some kind of support for managing this a lá namespacing.

Hi. Currently experiencing an issue where when our statsd collector is restarted, it stops receiving metrics from all clients until they are restarted.

I looked through the code and found a curious spot.

https://github.com/msiebuhr/node-statsd-client/blob/master/lib/EphemeralSocket.js#L117-L120

Why does the client bind to a socket and listen for udp packets before sending them?

What I expected:

statsd.set('names', 'A');
statsd.set('names', 'B');
statsd.set('names', 'C');

… with names:3|s emitted.

Instead, I need to statsd.set('names', 3)… in which case, keeping track of the set and when to emit is left to me.

See #69

Streams will change significantly in Node 0.10, so these will be worthless as-is.

And does anybody actually use the stream-helpers?

From NewRelic's Node API:

asyncFuncToMeasure(some, arguments, statsd.timeAsyncFunc("foo.bar.baz", callback(...))

It should basically start the timer when it is initially called wrap the given function in a small shim that will submit the time taken to execute.

This is all about getting data out the door. Let's make sure we don't have any obvious performance issues around.

Ex. we bind() the UDP-socket for no apparent reason, which might not be a good idea...

• Find some benchmarking library we can use
• Write some basic benchmarks

In complex setups, having mostly-working defaults all over makes debugging quite hard.

• It should be in-active when no configuration is given (at least no host is given).
• Some kind of logging. Consider giving a console-like object when setting up, to do some logging.
• Assert-guards to make sure stuff is set up correctly.

Anything else?

Hi,

I'm using statsd-client in an express application which exposes a REST API with several endpoints (URLs).

I use statsdUrlKey to distinguish between endpoints and it works great with response times.

Is there any reason why statsdUrlKey is ignored when pushing response_code metrics to statsd?
It could be useful to distinguish the route names for response codes also. Example: compute and display the number of requests for a given endpoint and not the total for all endpoints ...

Thanks for your (future) answer.

Per statsd/statsd@817738f

StatsD now support delta gauges on the form foo.bar:-10|g and foo.bar:+10|g.

https://github.com/etsy/statsd/blob/master/docs/metric_types.md#gauges

With a fresh install from npm, I can't get the client to post to StatsD server.

 npm install statsd-client

Slightly modified example to post to StatsD server running on localhost. Result - no stats get posted.

Adding debug code to EphemeralSocket.js, it's definitely trying to send. Server is running on default port 8125 (and I tried another statsd client that is posting, so I know server is working).

var sdc = require('statsd-client'),
SDC = new sdc({ debug: 1, prefix: "statsd-client"}),
SDCTest = SDC.getChildClient('test');

var begin = new Date();
setTimeout(function () {
    // Set 'statsd-client.test.gauge'
    SDCTest.gauge('gauge', 100 * Math.random());

    // Icrement 'statsd-client.test.counter' twice
    SDCTest.increment('counter');
    SDC.increment('test.counter');

    // Set some time
    SDC.timing('speed', begin);

    // Close socket
    SDC.close();
}, 100 * Math.random());

Support sampling (the |@some-interval) in the client,

Hi,
I'm using this library to send data to telegraf-statsd server. All seems to work fine except when I attach tags to the metrics.

Given this sentence sdc.increment('my_counter', 2, { service: 'value' }); the library send the message my_counter:2|c|#service:value while telegraf-statsd expects something like my_counter,service=value:+3|c.

Does the library supports telegraf?

Otherwise it is difficult to see what changed to consider an upgrade