Getting Unexpected EOF when decompressing from UTF-16 about lz-string HOT 4 CLOSED

Can you tell me what column type you use to store the result of the compression? Also on what type of device does it fail?

from lz-string.

I think I solved the problem. It seems that the compressToUTF16 function sometimes generates \u2028 and \u2029 characters which are parsed as line terminators (see here for details).

I ended up replacing all occurrences of the 2 characters after compression and before storing to the db, then after retrieval and before decompression. Something like this:

before storage

var dbvalue = LZString.compressToUTF16(JSON.stringify(value));
dbvalue = dbvalue.replace(/\u2028/g, '\\u2028').replace(/\u2029/g, '\\u2029');

after retrieval

var utfdata = dbvalue.replace(/\\u2028/g,'\u2028').replace(/\\u2029/g,'\u2029');
var value = JSON.parse(LZString.decompressFromUTF16(utfdata));          

Of course, it would be better if these 2 characters can be escaped inside LZString instead.

from lz-string.

LZString itself is a hack about how to store bits in UTF-16 strings. The only guarantee you get here is that the produced string is a valid UTF-16 string. Now, if your particular storage mechanism doesn't support certain UTF characters, the best way is to replace them with others. But LZString cannot (and should not) accommodate all quirks of all storage mechanisms.

Now, about your code, the replace you've used is not safe. Nothing prevents LZString from outputting a string made of "\u2028" so your replace could conflict with this sequence.

You should rather replace both characters with other characters in the UTF range that are not used by LZString. LZString will use 15 bits, meaning char 0 to 32767 but shifted by 32 because chars below 32 aren't valid UTF. So the range used effectively is 32 to 32799. Just replace both your characters with 32800 and 32801 and you should be fine.

from lz-string.

I see your point. I guess it is better to have that part implemented outside of the library.

Thanks for the suggestion! I'll try working that into my code

from lz-string.