mirkokiefer / bunyan-cloudwatch Goto Github PK

View Code? Open in Web Editor NEW

47.0 2.0 33.0 21 KB

Writable stream to write bunyan logs to AWS CloudWatch

License: MIT License

JavaScript 100.00%

bunyan-cloudwatch's Introduction

bunyan-cloudwatch

Stream to write bunyan logs to AWS CloudWatch.

This is actually a plain Node.js Writable object stream so could be used without bunyan.

Usage

var bunyan = require('bunyan');
var createCWStream = require('bunyan-cloudwatch');

var stream = createCWStream({
  logGroupName: 'my-group',
  logStreamName: 'my-stream',
  cloudWatchLogsOptions: {
    region: 'us-west-1'
  }
});

var log = bunyan.createLogger({
  name: 'foo',
  streams: [
    {
      stream: stream,
      type: 'raw'
    }
  ]
});

To avoid raising exceptions when stringifying circular object logs, install the optional dependency 'safe-json-stringify'.

Avoid logging from multiple sources to the same CloudWatch log stream. The module has a fallback by requesting the latest log sequence but this can result in a major performance decrease due to additional requests. It is recommended to e.g. add the machines IP address to the log stream name when logging from multiple instances.

API

createCWStream(opts)

With opts of:

logGroupName (required)
logStreamName (required)
cloudWatchLogsOptions (optional): options passed to the AWS.CloudWatchLogs constructor

On write of the first log, the module creates the logGroup and logStream if necessary.

We use the aws-sdk to write the logs - the AWS credentials have therefore to be configured using environment variables (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY).

Configuring the aws-sdk
CloudWatchLogs.putLogEvents is the method we use to write logs

Contributors

This project was created by Mirko Kiefer (@mirkokiefer).

bunyan-cloudwatch's People

Contributors

Stargazers

Watchers

bunyan-cloudwatch's Issues

Timeout on AWS Lambda

It works fine on my local environment (my laptop). But the same code on AWS Lambda would cause a timeout exception, and no log event was created in the target log group.

'use strict';
const bunyan = require('bunyan');
const bunyanCloudWatch = require('bunyan-cloudwatch');
const PrettyStream = require('bunyan-prettystream');
const os = require('os');

const prettyStdOut = new PrettyStream();
prettyStdOut.pipe(process.stdout);

const prettyStdErr = new PrettyStream();
prettyStdErr.pipe(process.stderr);

module.exports = bunyan.createLogger({
  name: 'nexus',
  streams: [
    {
      level: 'error',
      stream: bunyanCloudWatch({
        logGroupName: 'herald',
        logStreamName: os.hostname() + '-' + Date.now(),
      }),
      type: 'raw',
    },
    {
      level: 'debug',
      type: 'raw',
      stream: prettyStdOut,
      reemitErrorEvents: true,
    },
  ],
});

CloudWatch log group of the Lambda function (not the target log group)

REPORT RequestId: 5de9d2b8-1296-11e9-a802-8bf9a9168cb1	Duration: 10010.15 ms	Billed Duration: 10000 ms Memory Size: 128 MB	Max Memory Used: 53 MB	
2019-01-07T16:07:54.192Z 5de9d2b8-1296-11e9-a802-8bf9a9168cb1 Task timed out after 10.01 seconds
2019-01-07T16:07:54.748Z	5de9d2b8-1296-11e9-a802-8bf9a9168cb1	Getting value AWS_REGION from environmental variable with value us-west-2 overriding us-west-2

JSON.stringify crashes application

Because this module passes log input through JSON.stringify before passing it to bunyan, it runs the high risk of causing an exception due to the circular structure of the object passed to the logger.

bunyan itself avoids this problem, so all our logging calls were fine before we introduced this module.

IMHO, at the very least, the same stringification approach from bunyan should be utilized.

Implement writable._writeev() to remove own buffer implementation

https://nodejs.org/api/stream.html#stream_writable_writev_chunks_callback

Logging error handling

It seems that now to handle errors during logging we need to define an onError function for the stream object.

IMO, it should provide a default onError function that swallow errors, since logging should never make your application to crash, which is the current behavior if you haven't defined this function.

Also, maybe since the stream inherits from Writeable, error handling should be performed by using a stream.on('error', function(err) {...}) instead of using this onError function. Or otherwise, we could pass the error function as part of the options. It doesn't feel right to have to define that function directly over the stream.

Opinions?

Data Already Accepted Exception

I can find very little information on this error in the AWS documentation and do not know the root cause, so for now I have forked the repo and used a method similar to https://github.com/mirkokiefer/bunyan-cloudwatch/pull/2/files to handle the error:
DataAlreadyAcceptedException: The given batch of log events has already been accepted. The next batch can be sent with sequenceToken: 49567539988883386692933872174268000661736578965150110610

The problem is that this appears to halt all further sending of logs to Cloudwatch until the app is restarted.

Can you suggest a cause or solution to this, as it does not appear that bunyan-cloudwatch currently receives this as a retryable error?

Error thrown if optional dependency safe-json-stringify is not installed

Loading the optional dependency safe-json-stringify is not properly handled when it is not installed which results in an error being thrown (see stack trace below).

Error: Cannot find module 'safe-json-stringify'
    at Function.Module._resolveFilename (module.js:489:15)
    at Function.Module._load (module.js:439:25)
    at Module.require (module.js:517:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/app/node_modules/bunyan-cloudwatch/index.js:5:25)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)

PR on its way.

No timestamp in Clouwatch logs

The timestamp field is not logged in Cloudwatch. I only see this much in my CW logs. Is this a configuration issue?

{
    "name": "web-monitor",
    "hostname": "my-foo-host",
    "pid": 3411,
    "level": 20,
    "msg": "Error count is 6",
    "v": 0
}

Possible v3 of bunyan-cloudwatch

Thanks for the great library. I've recently rewritten bunyan-cloudwatch to overcome a few issues I encountered:

time prop removed from bunyan JSON (#17)
Unable to recover logs that failed to send (#19)
Memory leak after onError (stream does not clear the queue and continues adding to it)

I also added support for recovering from AWS errors and followed AWS limts on PutLogEvents (i.e. max 10000 log events and 1 MB per call) just to be safe.

Since I had a hard deadline I've published it as cwlogs-writable.

However, I kept the constructor options (and methods to a degree) the same so it could act as a drop in replacement of bunyan-cloudwatch.

If you're still maintaining this library, I'd love to collaborate to try to work these changes into bunyan-cloudwatch.

Thanks again!

Error: Timestamp is null

Hi there,

I am seeing this error when running in AWS:

InvalidParameterException: 1 validation error detected: Value null at 'logEvents.1.member.timestamp' failed to satisfy constraint: Member must not be null
    at Request.extractError (/api/node_modules/aws-sdk/lib/protocol/json.js:43:27)
    at Request.callListeners (/api/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/api/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/api/node_modules/aws-sdk/lib/request.js:596:14)
    at Request.transition (/api/node_modules/aws-sdk/lib/request.js:21:10)
    at AcceptorStateMachine.runTo (/api/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /api/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/api/node_modules/aws-sdk/lib/request.js:37:9)
    at Request.<anonymous> (/api/node_modules/aws-sdk/lib/request.js:598:12)
    at Request.callListeners (/api/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at Request.emit (/api/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/api/node_modules/aws-sdk/lib/request.js:596:14)
    at Request.transition (/api/node_modules/aws-sdk/lib/request.js:21:10)
    at AcceptorStateMachine.runTo (/api/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /api/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/api/node_modules/aws-sdk/lib/request.js:37:9)

I have configured the logger like so:

module.exports = function() {

    var bunyan = require('bunyan'),
        Bunyan2Loggly = require('bunyan-loggly').Bunyan2Loggly,
        bformat = require('bunyan-format'),
        formatOut = bformat({ outputMode: 'short' }),
        log
    ;

    if (process.env.NODE_ENV === 'local') {
        log = bunyan.createLogger({'name': 'LOCAL', stream: formatOut});
    } else {
        log = bunyan.createLogger({
            name: process.env.LOGGLY_ALIAS.replace('.%s', ''),
            streams: [
                {
                    level: 'error',
                    type: 'raw',
                    stream: new Bunyan2Loggly({
                        token: process.env.LOGGLY_TOKEN,
                        subdomain: process.env.LOGGLY_SUBDOMAIN
                    })
                },
                {
                    level: 'fatal',
                    type: 'raw',
                    stream: require('bunyan-cloudwatch')({
                        logGroupName: process.env.OPSGENIE_CW_GROUP,
                        logStreamName: 'service-api-ops-cloudwatch-stream',
                        cloudWatchLogsOptions: {
                            accessKeyId: process.env.AWS_KEY,
                            secretAccessKey: process.env.AWS_SECRET,
                            region: process.env.AWS_REGION
                        }
                    })
                }
            ]
        });
    }

    return log;

};

Using the logger:

log.fatal('TESTING CLOUDWATCH');

Thanks

Loss of logs

at https://github.com/mirkokiefer/bunyan-cloudwatch/blob/master/index.js#L44, the log queue is being cleared however there is a possibility the writelogs() would fail afterward. In such event, that results into the loss of log entries.

Batch Logs

Feature Request.

I haven't been able to use this because I need to call the log event more than 5x/second. Would love to see periodic uploads added to this library.

"InvalidSequenceTokenException" thrown when used with multiple instances

When pushing logs from multiple machines to the same stream, the following error is encountered:

{ 
  message: 'The given sequenceToken is invalid. The next expected sequenceToken is: 49540113951436919075269750063470750655650281036628124738',
  code: 'InvalidSequenceTokenException',
  time: Wed Jun 10 2015 19:36:17 GMT+0000 (UTC),
  statusCode: 400,
  retryable: false,
  retryDelay: 30 
}

OperationAbortedException: A conflicting operation is currently in progress against this resource. Please try again.

When I fired up the server using the example I got bunyanCW is undefined. I changed the request var from createCWStream to bunyanCW and it fired up and worked just fine reporting to cloudwatch. However, when I fired off an event I got this error.

OperationAbortedException: A conflicting operation is currently in progress against this resource. Please try again.

Here is my current setup

var bunyan = require('bunyan');
var bunyanCW = require('bunyan-cloudwatch');
var extend = require('extend');

/**
 * Provide a common base logger for our application
 *
 * @param  {string} component The (sub)component we are logging for
 * @param  {Object} defaults  Any 'default' or 'common' values to include in the logger
 * @return {Object}           The logger
 */
module.exports = function (component, defaults) {

  var stream = bunyanCW({
    logGroupName: 'reactor-group',
    logStreamName: 'reactor-stream',
    region: 'us-east-1'
  });
  // var streams = clone(config.streams);
  var options = {
    name: 'reactor.core',
    streams: [
      {
        stream: process.stdout,
        level: 'debug'
      },
      {
        stream: stream,
        type: 'raw'
      }
    ]
  };
  if (typeof component === 'string') {
    options.component = component;
  }
  if (typeof defaults === 'object') {
    options = extend(false, options, defaults);
  }

  var logger = bunyan.createLogger(options);

  var logLevel;
  if (process.env.LOGLEVEL) {
    logLevel = process.env.LOGLEVEL;
  } else {
    logLevel = (process.env.NODE_ENV === 'development' ? 'debug' : 'info');
  }
  logger.level(logLevel);

  return logger;
};

return in async functions?

I see few return statements in async functions.

For example https://github.com/mirkokiefer/bunyan-cloudwatch/blob/master/index.js#L102, IMO the callback should be called with an error as argument because this method already returned a value upon stating (not finishing) the execution of cloudwatch.createLogGroup().

Same comment on getSequenceToken() and _writeLogs() function

Chunk logs to avoid hitting 1MB limit in putlogevents

From the docs, http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/CloudWatchLogs.html#putLogEvents-property

The maximum batch size is 1,048,576 bytes, and this size is calculated as the sum of all event messages in UTF-8, plus 26 bytes for each log event.

It would be nice to limit/chunk the amount of data uploaded, since a request over the limit returns error and crashes nodejs.

mirkokiefer / bunyan-cloudwatch Goto Github PK

bunyan-cloudwatch's Introduction

bunyan-cloudwatch

Usage

API

createCWStream(opts)

Contributors

bunyan-cloudwatch's People

Contributors

Stargazers

Watchers

Forkers

bunyan-cloudwatch's Issues

Recommend Projects

Recommend Topics

Recommend Org