crispyconductor / node-zstreams Goto Github PK

Add an easy interface to create a correctly behaving stream from request()

We should only include useful abstractions.
Befunge is not useful.

Or something similar.
The issue I'm having is that I can't use the 'to' convention for String, since '' + readSrtream will implicitly call the toString method on `readStream.

Need the following JSON streams:

• JSONStringifyStream - Generate (configurable) either delimiter-separated JSON blobs or a continuous single JSON array from a stream of objects
• JSONParseStream - Streaming JSON generation

Similar things have been implemented by other existing projects; it may be possible to use those and just wrap them in zstreams.

We should add a grunt file for running tests and the linter at the same time.

We will also want it in the future for Browserify/Bower builds.

Add usage/existence of helper methods to readme

More specifically, it returns a stream, rather than follow the conventions of the other into functions

Add a stream that will retry RequestStream connections on error and utilize the Range header to only pull data that still needs to be pulled.

There is a confusing error if anything bombs out in a through function about no writecb in the Transform.

We want this to be useful. Making the user download a parser for an esoteric language isn't useful.

It makes it harder to how/when features are added.

Readables are supposed to continue emitting data as long as .push() returns true and it has data to emit. FunctionStream's current implementation will result in unnecessary delays and pauses. It needs to repeatedly call the generator function until push() returns false. There should also be checks added for if _read() is called while a generator function is already running.

Use to encapsulate a Readable/Writable chain.

This could be done with CompoundDuplex, but it would leave unusable functions on the instance.

This will be a breaking change. Make helpers like .through() synchronous by default, unless they return a Promise . Add variants like throughAsync() for the callback-style interface.

Ex., if I call readable.intoString() without supplying a callback, it should return a promise. Same with the other applicable into...() functions.

Need to look in to this more for an exact cause, but I have a very large file being written, which had it's callback called before it's done writing.

As seen here.

For all the things.

This is an empty file.

On Duplex/Transform streams, the highWaterMark needs to be explicitly set on the readable and writable states.
This is caused by a bug that is fixed in Streams3

Browser compatibility will be nice.

e.g. each()

https://github.com/crispy1989/node-zstreams/blob/8fa37cacb25197ba4b6c34a7f39f018f5f61828e/lib/index.js#L16

browserify does not support fs.readdir or fs.readdirSync

browserify/brfs#19

Currently, if a stream has finished or errored already, and .intoPromise() or .intoCallback() are called, the promise never resolves/callback is never called. This should do the expected of returning a resolved/rejected promise or calling the callback immediately.

Should return a promise

I'm seeing some odd behavior when using SplitStream with windows-style line endings, specifically when the input stream ends in one. This seems like it might indicate a larger problem with how trailing delimiters are handled.

const { SplitStream, StringReadableStream } = require('zstreams');

let unix = new StringReadableStream('foo\nbar\n');
let win = new StringReadableStream('foo\r\nbar\r\n');

unix.pipe(new SplitStream()).intoArray()
	.then((array) => {
		console.log(array); // Logs [ 'foo', 'bar' ] as expected.
		return win.pipe(new SplitStream()).intoArray();
	})
	.then((array) => {
		console.log(array); // Logs [ 'foo', 'bar', '\n' ].
	});

Hi, i am piping a request to FileBufferStream, then i pipe to passtroughtstream, and consume with fluent-ffmpeg.

some times i got this error line 173:

// We got an EOF but we've written more than we've read ?
throw new Error('Unexpected EOF reading buffer file');

The consumer (ffmpeg) is to fast? how i can resolve this? thanxs.

Need the additional event streams:

• Stream to convert normal object/data stream to events (ex. an XML parser)
• Stream to convert events to normal object/data stream
• Subclass of above stream to listen to a single event and convert its payload into an object stream

As well as added a good .jshintrc

Document helper functions from the mixins like .each, .skip, etc.

The zstream(...) conversion doesn't work with certain types of streams, particularly older streams that are not streams2 compliant. These need to be deep-wrapped in a way similar to RequestStream . Ideally this would be detected and done automatically in the conversion code.