Add NodeJS stream support about lz-string HOT 6 CLOSED

LZ-String, as its name implies, is supposed to work on strings as input and strings as output. I am sure there are plenty of stream compressors out there that work much better than LZ-Strnig to compress a stream.

What usage did you have in mind for this feature ?

from lz-string.

@pieroxy, I'm currently working on a custom analytics solution that has to support partially offline devices (browsers) and be cross-platform. I've used IndexedDB (with a shim for iOS) to store the metrics.

Since there is a lot of redundancy in the data on the device that has to be posted back to the backend once online, I used your library to compress the data (using compressToUTF16 stored into a Uint16Array to be posted via xhr as binary data). Since the data is basically a huge JSON array, I was hoping to be able to nicely stream it to the LevelDB backend. Using something like on NodeJS :

var dbWriteStream = db.createWriteStream();
app.post('/collect', function(req, res) {
  req.pipe(lzUncompress).pipe(JSONStream.parse('rows.*')).pipe(dbWriteStream);
});

from lz-string.

Well... I don't really have time to do this these days, but I'm always glad to accept pull requests.

On an unrelated note, if you store your stuff in a Uint16Array, you probably don't need to worry about compressToUTF16 (which uses 15bits per char) and could achieve better encoding ratio by using compress directly.

from lz-string.

I was using compressToUTF16 since I'm using this on the server side:

    var string = buffer.toString('utf16le');
    string = LZString.decompressFromUTF16(string);

Did not manage to get it work using raw binary encoding.
Do you think I can still safely use compress directly with this setup?

I'm still getting my head around streams, but I might have a try, not sure how your algorithm works but do you think it could be easily adapted to handle streams? Or would it require to rewrite the algo?

from lz-string.

Sorry to have waited that long... Did you get your issue resolved ?

from lz-string.

Well, it's been a while. If I don't hear from you I'll just close this issue.

from lz-string.