jandre / always-tail Goto Github PK

node module for always tailing a file -- recovers gracefully if it gets moved, deleted, or rolled-over.

JavaScript 100.00%

always-tail's Introduction

always-tail

Node.js module for continuously tailing a file.

It differs from other tail modules in that it survives truncates, file rollovers (e.g. mv /var/log/test /var/log/test.1), and unlink.

It does this by monitoring the filename, and when the inode changes, it will continue to read to the end of the existing file descriptor, then automatically read from the newly created file with the same name.

It emits a 'line' event when a new line is read.

Installation

npm install always-tail

Example

var Tail = require('always-tail');
var fs = require('fs');
var filename = "/tmp/testlog";

if (!fs.existsSync(filename)) fs.writeFileSync(filename, "");

var tail = new Tail(filename, '\n');

tail.on('line', function(data) {
  console.log("got line:", data);
});


tail.on('error', function(data) {
  console.log("error:", data);
});

tail.watch();

// to unwatch and close all file descriptors, tail.unwatch();

Usage

var tail = new Tail(filename, separator, options);

filename - filename to monitor

separator - optional separator for each line (default: \n)

options.interval - optional interval to check for changes

options.start - optional start byte to start reading from

options.blockSize - maximum reading block size (default is 1 MB)

Credits

Code is heavily modified from the node-tail module (https://github.com/forward/node-tail)

License

MIT

always-tail's People

Contributors

Stargazers

Watchers

Forkers

djcas9 knolleary pekeler anandpathak longloop tareq101 cloudhomebr douglastkawano clearvox melvinto veedeo alainbartmandilaw emms007 jbenguira lankesh kinvey

always-tail's Issues

Can we have the index.coffee that produces index.js?

Hi!
It seems that index.js is an output of some coffee script. I was wondering if you are open to share that original coffee script?

Thanks

Getting 'TypeError [ERR_INVALID_CALLBACK]: Callback must be a function. Received undefined' on tail.unwatch()

Here's stack trace:

TypeError [ERR_INVALID_CALLBACK]: Callback must be a function. Received undefined
    at makeCallback (fs.js:168:11)
    at Object.close (fs.js:427:14)
    at Tail.unwatch (/home/komal/projects/zestl/source_code/TwigMeServerApplication/docker/clients/python_error_logs/api/node_modules/always-tail/index.js:164:10)
    at Socket.<anonymous> (/home/komal/projects/zestl/source_code/TwigMeServerApplication/docker/clients/python_error_logs/api/services/tail_file.js:22:14)
    at Socket.emit (events.js:327:22)
    at TCP.<anonymous> (net.js:673:12) {
code: 'ERR_INVALID_CALLBACK'
}

Here's what I was trying to do

const TailFile = (req, res, next, filename) => {
    var Tail = require('always-tail');
    var fs   = require('fs');
    if (!fs.existsSync(filename)) {
        fs.writeFileSync(filename, "")
    };
    var tail = new Tail(filename, '\n');
    tail.on('line', function (data) {
        console.log("got line:", data);
        res.write(data)
    });
    tail.on('error', function (data) {
        console.log("error:", data);
    });
    tail.watch();

    res.header('Content-Type', 'text/event-stream');

    req.socket.on('close', () => {
        // This is not working
        tail.unwatch();
    });
};

I am getting error on tail.unwatch();

I wandered around the always-tail code and found that by changing following lines in always-tail/index.js's Tail.prototype.unwatch function, the error message is gone and my application stops crashing
Previous code
fs.close(self.fd);
Fixed code
fs.close(self.fd, function(){});

Previous code
fs.close(item.fd);
Fixed code
fs.close(item.fd, function(){});

Get line number of current line

Is there any way to get the line number of the current line?

Update debug

I note that the version of debug that always-tail relies on has a minor vulnerability filed against it. Can you update the dependency please?

Here is the audit message. The formatting is not perfect but is readable:

`
│ Low │ Regular Expression Denial of Service │

│ Package │ debug │

│ Patched in │ >= 2.6.9 < 3.0.0 || >= 3.1.0 │

│ Dependency of │ always-tail │

│ Path │ always-tail > debug │

│ More info │ https://nodesecurity.io/advisories/534 │
`

truncated log file not read correctly

When a tailed log file is being truncated, always-tail gets confused about its current location.

always-tail is tailing a large log file, and is at the end
the log file gets truncated or overwritten with an empty string
the log file gets its first line
always-tail will emit a line event for this first line (so far so good)
the log file gets its second line
always-tail will emit a line event for the first line (again, this is the bug) and then for the second line
the log file gets its third line
always-tail will emit a line event for the first line (bug), the second line (bug), and then for the third line
and so on

When Tail gets constructed with a start option value larger than the actual log file, we'll get the same symptom.

tail files without polling

Just wondering if there would be a way to implement this library without polling for file changes?
Would be a lot more performant, I think, thanks!

REQUEST: Tail multiple files

Is there a way to tail multiple files? Or could that functionality be added easily to this project?

what type of data is returned by 'line'-event

return only the last line

Hi Everybody,

I have noticed that the always-tail package reads the entire file. Is there a way to only return the last line of the file or like an option where one could set how many lines one wants to be returned. Just like the tail command on Linux which allows me to set the lines returned with the -n flag.

ERR_INVALID_CALLBACK when logfile is deleted

Hello,
i have an exception with the code from readme when logfile get deleted.

#> node --version
v10.15.1
#> node tail.js
got line: aaa
got line: abc
fs.js:137
throw new ERR_INVALID_CALLBACK();
^

TypeError [ERR_INVALID_CALLBACK]: Callback must be a function
at makeCallback (fs.js:137:11)
at Object.close (fs.js:395:20)
at next (/root/scripts/logpusher/node_modules/always-tail/index.js:27:12)
at /root/scripts/logpusher/node_modules/always-tail/index.js:52:31
at FSReqWrap.oncomplete (fs.js:155:5)

It should handle non-existing file

It throws if the file does not exist. One improvement would be to accept missing file and simply wait for it to appear (since that is often the case with some log files). It makes it more robust in a server environment.

Oh and I think it should by default start at the end of file, not at position 0. I believe these two features are related.