mjackson / bufferedstream Goto Github PK

A robust stream implementation for node.js and the browser

JavaScript 100.00%

bufferedstream's Introduction

BufferedStream is a robust stream implementation for node.js and the browser based on the initial version of the stream API in Node.js. All data that is written to a BufferedStream is buffered until the next turn of the event loop. This greatly enhances the usability of streams by making it easy to setup listeners in the same turn of the event loop before data is emitted.

The implementation follows the first version of the node streams API, which is powerful because of its simplicity. Node has since moved on to other, much more complex streams implementations, but there never was a problem with the initial API. The only problems were with node's implementation. For example, streams did not always wait until the next tick to emit data. Also, some streams did not respect pause/resume semantics.

BufferedStream addresses these problems by providing a well-tested, performant implementation that preserves the original streams API and works in both node.js and browsers.

Usage

The key feature of this class is that anything you write to the stream in the current turn of the event loop is buffered until the next one. This allows you to register event handlers, pause the stream, etc. reliably without losing any data.

var BufferedStream = require('bufferedstream');

var stream = new BufferedStream;
stream.write('Hello ');
stream.pause();

setTimeout(function () {
  stream.write('IHdvcmxkLg==', 'base64');
  stream.resume();
  stream.on('data', function (chunk) {
    console.log(chunk.toString()); // Hello world.
  });
}, 10);

The BufferedStream constructor may also accept a "source" which may be another stream that will be piped directly through to this stream or a string. This is useful for wrapping various stream-like objects and normalizing their behavior across implementations.

var stream = new BufferedStream(anotherStream);

Please see the source code for more information. The module is small enough (and well-documented) that it should be easy to digest in a quick skim.

Installation

Using npm:

$ npm install bufferedstream

Or, include dist/BufferedStream.min.js in a <script> tag:

<script src="BufferedStream.min.js"></script>

Issues

Please file issues on the issue tracker on GitHub.

Tests

To run the tests in node:

$ npm install
$ npm test

To run the tests in Chrome:

$ npm install
$ npm run test-browser

License

MIT

bufferedstream's People

Contributors

Stargazers

Watchers

Forkers

mvolkmann jeffbski utricularian smillaedler lawlmart kriszyp alexjeffburke

bufferedstream's Issues

Premature ending while paused

In the flush function, if the "stream.ended" check passes while the stream is paused, BufferedStream will emit an end and wipe out the internal buffer.

The fix seems simple, don't call the end function if the stream is paused (allowing it to resume and flush first)

if (stream.ended && !stream.paused) {
end(stream);
return;
}

Thanks

'end' is never emitted when piping an empty stream to the http response stream

when piping an empty bufferedstream to the http response stream the 'end' event is never emitted and the request hangs until the socket timeout occurs. piping it to a file stream however does work.

it looks like the empty bufferedstream gets paused through the 'pipe' logic and this prevents the 'end' event being emitted. this seems to be a regression of issue #5.

test case:

var http = require('http'),
  BufferedStream = require('bufferedstream');

http.createServer(function (req, res) {
  var stream = new BufferedStream();
  stream.end('');
  stream.pipe(res);
  // 'end' is never emitted, request hangs...
}).listen(8080, '127.0.0.1');

Doesn't have the unpipe implementation

I am using this package to create a buffered stream that is then consumed by one of the unzip packages. The unzip package has an 'unpipe' call which fails since there is no implementation of 'unpipe' in bufferedstream. Is there a way to implement our own 'unpipe' method on the client?

duplex stream support?

Hi,

See example code below. I was trying to use this module to create simple proxy. However ssh complained of corruption. When I manually connected to server I see an 'echo' of what I type, which is definitely not expected. Seems like duplex streams (like socket) are not supported with pipe?

var net = require('net');
var BufferedStream = require('bufferedstream');


//recieves the socket of the remote end of the tunnel                                                                                                                                                       
function Proxy( remoteSocket, localPort ) {


    // this buffer insures no data is lost before we have a local socket                                                                                                                                    
    var bufferedRemoteSocket = BufferedStream( remoteSocket )                                                                                                                                             
    bufferedRemoteSocket.pause()                                                                                                                                                                          

    var onServerConnected = function( localSocket ) {

        console.log( 'local connection' )
        //duplex pipe                                                                                                                                                                                       
        bufferedRemoteSocket.pipe( localSocket ).pipe( bufferedRemoteSocket )   
         bufferedRemoteSocket.resume()                                                                                                                  

    }
    var server = net.createServer( onServerConnected )
    server.listen( localPort, 'localhost' )
    console.log( 'listening on 22000' )
}
module.exports.Proxy = Proxy

var onConnected = function() {
    //console.log('client connected')                                                                                                                                                                       
    proxy = new Proxy( client, 22000 )
}
var client = net.connect({port: 22}, onConnected )

with node 0.10+, shouldn't we use setImmediate() rather than process.nextTick() ?

I was reading the changes for node 0.10 and it has changed the way process.nextTick works so that it always returns before any IO (although it has a failsafe which allows IO after recursion causes it to hit process.maxTickDepth)

So since bufferedStream is supposed to be triggering flush after IO shouldn't we be using the recommended setImmediate() rather than process.nextTick() ?

Otherwise bufferedStream will put things into a tight loop (where no IO is occuring, until it hits process.maxTickDepth, then it will bail out and do some IO).

See
http://blog.nodejs.org/2013/03/11/node-v0-10-0-stable/ (scroll down to 'faster process.nexttick' section)

Ref to setImmediate doc:
http://nodejs.org/api/timers.html#timers_setimmediate_callback_arg

Here is the code I am referencing:
https://github.com/mjijackson/bufferedstream/blob/master/buffered-stream.js#L153

mjackson / bufferedstream Goto Github PK

bufferedstream's Introduction

Usage

Installation

Issues

Tests

License

bufferedstream's People

Contributors

Stargazers

Watchers

Forkers

bufferedstream's Issues

Premature ending while paused

'end' is never emitted when piping an empty stream to the http response stream

Doesn't have the unpipe implementation

duplex stream support?

with node 0.10+, shouldn't we use setImmediate() rather than process.nextTick() ?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent