Allow different configs to be passed into same broadcast script to broadcast different log files.

I'm working on a project that is using hapi and good and I have leakDetection enabled with ops outputting to console.

When I run my app, here's some output after a few mins:

17 Dec 16:06:08 - [nodemon] starting node server.js
131217/160624.849, ops, memory: 86M uptime (seconds): 16
131217/160639.850, ops, memory: 86M uptime (seconds): 31
131217/160654.850, ops, memory: 88M uptime (seconds): 46
131217/160709.850, ops, memory: 88M uptime (seconds): 61
131217/160724.850, ops, memory: 91M uptime (seconds): 76
131217/160739.851, ops, memory: 91M uptime (seconds): 91
131217/160754.851, ops, memory: 93M uptime (seconds): 106

Clearly, there is something going on here because the longer the app is running, the more memory it uses.

Shouldn't leakDetection be spitting something out? Where should I go from here to hunt down the source of the leak?

Required module does not work under node 0.12 and has compilation issues on many platforms. Will look for a way to bring those features back via native JS code later.

Errors on npm install or npm update when hapi 3.0.x is installed.
npm ERR! notarget No compatible version found: good@'>=2.0.0-0 <3.0.0-0'

> node -v
v0.10.24

> make test-cov-html

/Users/jan/Work/good/node_modules/lab/node_modules/chai/lib/chai/assertion.js:106
      throw new AssertionError(msg, {
            ^
AssertionError: expected '1469328953222-42369-62002' to equal '1369328752975-42369-3828'
    at Assertion.assertEqual (/Users/jan/Work/good/node_modules/lab/node_modules/chai/lib/chai/core/assertions.js:395:12)
    at Assertion.ctx.(anonymous function) [as equal] (/Users/jan/Work/good/node_modules/lab/node_modules/chai/lib/chai/utils/addMethod.js:40:25)
    at IncomingMessage.<anonymous> (/Users/jan/Work/good/test/broadcast.js:205:53)
    at IncomingMessage.g (events.js:180:16)
    at IncomingMessage.EventEmitter.emit (events.js:92:17)
    at _stream_readable.js:920:16
    at process._tickDomainCallback (node.js:459:13)
make: *** [test-cov-html] Error 8

I have a PR coming up that adds a native graphite subscriber, but I can’t test it because of the above :)

• show timestamp in human readable (just for dev mode)
• pretty printed
• non-json style, space separated lines

Hi, I'd like to propose another type of subscriber for functions.

I've sketched it out in my fork https://github.com/dannycoates/good/commit/473d84b718f788cf0c3ad2d6d58246f963a94e06

The api might be something like:

var options = {
    subscribers: {
        statsd: {
            tags: ['stat'],
            events: ['log', 'request'],
            handler: function (event) {
                // TODO: send a metric to statsd
            }
        }
    }
};

Would you be interested in a pull request for this?

Measure the time diff between the expected timeout and actual and publish event queue delay.

In our app, we use Hapi composer to initialize our server and load our plugins. Our database plugin, for example, may return an error if it cannot connect to the database.

What we're finding is that when that err bubbles up to the composer callback, we need to wrap throwing the error in a setTimeout in order to wait for Good to finish broadcasting the error.

We've lowered the broadcast interval and made the setTimeout bigger to try and mitigate the issue, but this isn't full proof. Is there a better way to ensure good has finished broadcasting before throwing the error?

var Hapi = require('hapi');
var manifest = require('./config/manifest');

var composer = new Hapi.Composer(manifest, { pack: {requirePath : __dirname} });
composer.compose(function (err) {
  if(err) { 
    // wait for good to complete log broadcast
    setTimeout(function() { throw err; }, 5000);
  }

  composer.start();
});

We use the OPS logs extensively for our apps in all of different envs. The problem we have is that the events don't have any context on the app that emitted the ops event, like the node_env or hostname for example.

It'd be nice to be able to add additional context to the OPS logs to address this, unless there's a better way.

We're spontaneously getting this error from Good:

TypeError: Converting circular structure to JSON
    at Object.stringify (native)
    at /my/path/node_modules/good/lib/monitor.js:267:102
    at Array.forEach (native)
    at internals.Monitor._broadcastHttp (/my/path/node_modules/good/lib/monitor.js:250:46)
    at internals.Monitor._broadcastRemotes (/my/path/node_modules/good/lib/monitor.js:241:10)
    at wrapper [as _onTimeout] (timers.js:252:14)
    at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)

The issue appears to be in Line 262 in monitor.js, where the events are being added to the outgoing broadcast envelope. When I comment that line out, the error no longer occurs.

It's possible I'm doing something wrong but I can't get any ops event to output to the console. Request events output just fine but not ops. I've tried explicitly subscribing to the 'ops' event as well as following the defaults but no difference.

Upgrade to newer version of async

This creates too much information, and the information is already available in the request logs.

I started seeing load: undefined print out when I moved to hapi 2.x + good 1.x

140205/221945.132, ops, memory: 72M uptime (seconds): 2762.54082292784load: undefined
140205/222000.133, ops, memory: 73M uptime (seconds): 2777.541325445287load: undefined
140205/222015.141, ops, memory: 72M uptime (seconds): 2792.5491037797183load: undefined
140205/222030.142, ops, memory: 72M uptime (seconds): 2807.550354591571load: undefined
140205/222045.143, ops, memory: 69M uptime (seconds): 2822.5512211993337load: undefined

For reference, my server logic is:

if (process.env.NODE_ENV === 'production') {
  subscribers = {
    'console': ['ops', 'log', 'error'],
    '/path/tp/error.log':  ['error']
  };
} else {
  subscribers = {
    'console': ['ops', 'request', 'log', 'error']
  };
}

var goodOptions = {
  subscribers: subscribers
};
server.pack.require('good', goodOptions, function (err) {
  if (!err) { /* Plugin loaded successfully */ }
});

I'm using hapi 2.1.2 and good 1.0.0

Need to use setImmediate() instead.

I don't have any experience with good, so I checked the examples:

good/examples/log.js:28
    Log.event('server', 'started at http://localhost:8080/');
        ^
TypeError: Object #<Object> has no method 'event'
    at Object.internals.startServer (good/examples/log.js:28:9)
    at Object.<anonymous> (good/examples/log.js:32:11)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:901:3

Instead of hard coding 'node' as the spawn name for the node executable, use process.execPath such that execution is portable and works with people testing against newer builds of node

Need to have examples of using the monitors

At the moment, the console doesn't display any disk details. Therefore, the monitor should be smart enough to not perform any ops related monitoring when the only subscriber is the console.

Hi,

try to use good 0.5.10 with hapijs 1.8.1:

var server = Hapi.createServer('localhost', 8000, options);

var options2 = {
  subscribers: {
    console: ['ops', 'request', 'log'],
    'http://localhost/logs': ['log'],
    '/tmp/logs/': ['request', 'log']
  }
};

server.plugin.require('good', options2, function (err) {

  if (!err) {
    // Plugin loaded successfully
  }
});

but when run, got:

server.plugin.require('good', options2, function (err) {
              ^
TypeError: Cannot call method 'require' of undefined

Am I missing something? Thanks

Ensure that all request headers are logged with the request event data.

Hello,

I'm using request.log in my handler but good doesn't handle this logging way.

edit : forgot 'debug' server param

Using hapi 6.0.1 and good 2.1.2.

server.pack.register(good, {
  subscribers: {
    console: ["ops", "request", "log", "error"]
  }
}, function() {});

Got the following error:

Error: Invalid register options  {
  "subscribers" [1]: {
    "console": [
      "ops",
      "request",
      "log",
      "error"
    ]
  }
}

[1] subscribers is not allowed
  at Object.exports.assert (/Users/Steve/Git/skimpage/node_modules/hapi/node_modules/hoek/lib/index.js:421:11)
  at Object.exports.assert (/Users/Steve/Git/skimpage/node_modules/hapi/lib/schema.js:15:10)
  at [object Object].internals.Pack._plugin (/Users/Steve/Git/skimpage/node_modules/hapi/lib/pack.js:263:12)
  at /Users/Steve/Git/skimpage/node_modules/hapi/lib/pack.js:252:14
  at iterate (/Users/Steve/Git/skimpage/node_modules/hapi/node_modules/async/lib/async.js:149:13)
  at Object.async.eachSeries (/Users/Steve/Git/skimpage/node_modules/hapi/node_modules/async/lib/async.js:165:9)
  at [object Object].internals.Pack._register (/Users/Steve/Git/skimpage/node_modules/hapi/lib/pack.js:250:11)
  at [object Object].internals.Pack.register (/Users/Steve/Git/skimpage/node_modules/hapi/lib/pack.js:179:17)
  at Object.<anonymous> (/Users/Steve/Git/skimpage/server.coffee:61:13)
  at Object.<anonymous> (/Users/Steve/Git/skimpage/server.coffee:1:1)
  at Module._compile (module.js:456:26)

Using these options i don't get any output at all

options = {
  subscribers: {
    console: {
      events: ['ops', 'request', 'log', 'error'],
      tags: ['error', 'warn', 'sync', 'auth']
    }
  }
}

The following works for ops and errors, but i'm not seeing any request logs:

options = {
  subscribers: {
    console: ['ops', 'request', 'log', 'error']
  }
}

Using good 2.0.0 and hapi 5.1.0

I left this running overnight and when I looked at it in the morning I noticed my server was completely idle. The process (hapi) appeared to still be running, but it was idle. I had been running the server using forever but since hapi never actually crashes I suppose that doesn't really do anything :)

manually restarting the server brought it back to operating and I decided to remove request logging for now.

The alarming part here isn't that the logs failed to be written but that hapi seems to have stopped accepting requests once this error was encountered.

{
  "event": "error",
  "url": {
    "protocol": null,
    "slashes": null,
    "auth": null,
    "host": null,
    "port": null,
    "hostname": null,
    "hash": null,
    "search": "OMITTED",
    "query": "OMITTED",
    "pathname": "OMITTED",
    "path": "OMITTED",
    "href": "OMITTED"
  },
  "method": "get",
  "timestamp": 1389053530597,
  "message": "Uncaught error",
  "stack": [
    "Error: EMFILE, too many open files",
    "    at Object.internals.ProcessMonitor.memory [as psmem] (/OMITTED/node_modules/good/lib/process.js:71:26)",
    "    at /OMITTED/node_modules/good/node_modules/async/lib/async.js:521:25",
    "    at /OMITTED/node_modules/good/node_modules/async/lib/async.js:108:13",
    "    at Array.forEach (native)",
    "    at _each (/OMITTED/node_modules/good/node_modules/async/lib/async.js:32:24)",
    "    at Object.async.each (/OMITTED/node_modules/good/node_modules/async/lib/async.js:107:9)",
    "    at _parallel (/OMITTED/node_modules/good/node_modules/async/lib/async.js:520:20)",
    "    at Object.async.parallel (/OMITTED/node_modules/good/node_modules/async/lib/async.js:536:9)",
    "    at opsFunc (/OMITTED/node_modules/good/lib/monitor.js:172:23)",
    "    at wrapper [as _onTimeout] (timers.js:252:14)"
  ]
}

for reference, my options are rougly:

var good_options = {
  subscribers: {
    'console':             ['ops', 'log', 'error'],
    '/OMITTED/access.log': ['request'],
    '/OMITTED/error.log':  ['error']
  },
  leakDetection: true,
  maxLogSize: 1073741824
};

system

ulimit -n
1024

I tried setting my ulimit very low to force the exception; nothing happened. I tried setting my ulimit very high to avoid the exception; still occurred.

Per your examples, I'm trying the following:

var options = {
    subscribers: {
        'http://localhost/logs': ['log']
    }
};

And I have a route for /logs which just dumps request.payload to console.

handlers.logs = function(req) {
   console.log(req.payload); 
   console.log('hello');
};

However, the handler is never getting the request because my console.logs are never fired. Why is this?

Also, won't setting a localhost to subscribe also trigger log entries itself?

Maybe I misunderstand how the Hapi logging and good are supposed to interact.

When I do a request.log() call in a hapi route handler, the request does this.server.emit('request', this, item, tagsMap); But good doesn't listen for 'request' events (at least not by default)-- its notion of 'request' logging seems to creating a record of a request and not logging various events pertaining to a request. So now I'm doing request.server.log() in my handler, which is fine. But I'm curious what request.log() is supposed to be used for?

Need to try to get code coverage back up to 100%

https://github.com/spumko/good/blob/master/lib/monitor.js#L562

Wrap in try

hapijs/hapi#1664

Just need to add attributes.

It would be nice if we could log request.auth.credentials (or some properties thereof) for request logs as a way to track requests by user.

Send to the console.error with the error message and stack trace

Good: v0.12.0
Node: v0.10.21
Hapi: v1.14.0
OS: Mac OS 10.8.5

After spinning up a site I hadn't worked on for a bit locally, I started getting this error. Never had it before. I downgraded my version of node recently, wondering if that's the problem.

/Users/user/htdocs/project/node_modules/good/lib/process.js:107
    var bench = new Hoek.Bench();
                ^
TypeError: undefined is not a function
    at Object.internals.ProcessMonitor.delay [as psdelay] (/Users/user/htdocs/project/node_modules/good/lib/process.js:107:17)
    at /Users/user/htdocs/project/node_modules/async/lib/async.js:521:25
    at /Users/user/htdocs/project/node_modules/async/lib/async.js:108:13
    at Array.forEach (native)
    at _each (/Users/user/htdocs/project/node_modules/async/lib/async.js:32:24)
    at Object.async.each (/Users/user/htdocs/project/node_modules/async/lib/async.js:107:9)
    at _parallel (/Users/user/htdocs/project/node_modules/async/lib/async.js:520:20)
    at Object.async.parallel (/Users/user/htdocs/project/node_modules/async/lib/async.js:536:9)
    at opsFunc (/Users/user/htdocs/project/node_modules/good/lib/monitor.js:172:23)
    at wrapper [as _onTimeout] (timers.js:252:14)

Hi,
here is the code:

var log_options = {
  subscribers: { console: ['request'] }
};

server.pack.require('good', log_options, function (err) {
  if (!err) {
    /* Plugin loaded successfully */
  }
});

when calling with http://localhost:8000//hello?test=asdf
output:

130504/115432.47, request, undefined: get /hello (8ms)

how to show the test= part ? also, what's that undefined after the request? Thanks,

Angelo

Document the HTTP payloads used to send updates.