Comments (11)
https://github.com/nodeca/pako#example--api second example is for objects and binstrings.
from pako.
Thank you for your reply.
I already read this official doc, similar stackoverflow questions and googled related problems.
pako.deflate(data, {level:"9"})
Creates 58 kB data.
pako.deflate(data, {level:"9", to:"string"})
Creates almost twice larger data, 104 kB to store.
Due to such a huge difference in compressed data size, option "to string" is not a solution in my case.
I tried many functions to convert standard pako deflate output to JavaScript string, but all of them returns different data from original string.
from pako.
pako core works with byte arrays. uit8 encoder/decoder is just a wrapper for convenience. You can use any other preparation methods, optimal for you task. Just send typed (if possible) array to pako and receive new array back. Everything else is on your choice and out of pako goals.
from pako.
I tested many packers. I choosed Pako because it is incredibly fast with large data (counted in MB). Almost every packer takes it too long on larger data, or just freezes/crashes browser. Thats why I choose Pako.
Its awesome work. I will try to find the way to solve my problem. If original data was correct JavaScript binary string and compression and decompression was correct for making binary files, then there must be the way to convert it back to binary string. I will keep on research and thanks for such a great tool.
from pako.
Note, that term binary string
means that every character has code 0..255. If you are sure, your data is really binary string, pako will process it as array of bytes. Also note:
- deflate
- input - array, binary string, JS string (utf16, autoconverted to utf8 first) - autodetected
- output { to: 'string' } - binary string
- inflate
- input - array, binary string
- output { to: 'string' } - expects that unpacked data is utf8 bytes stream and convert it to utf16 (js string)
That's done for the most used cases. But as i said, encoder is optional. If you use arrays on input and output, you can be sure, that pako works exactly as described in inflate/deflate specs, and result is binary equal to zlib.
from pako.
I can not guarantee, that built-in recoder is optimal for all cases. But is solves another task. Very often beginners ask "how should i send my object to server", they don't understand that JS strings are utf16, and server works with utf8. So, with built-in methods they can use json stringify/parse without big ass pain, and without writing special recoder on server side.
from pako.
Thank you for your reply.
After 3 days of fighting and testing anything, it seems that none of JavaScript expert is able to do it.
I created minimal code with problem as possible.
Question is how to convert back x3 to x2 or x1?
x1=getFile('filename');
// raw 16 bit binary string. not 0-255 but 0-65535 (not text)
x2=x1.split('').map(function(x){return x.charCodeAt(0);});
// changing characters to array with char codes
// first 4 elements: 63367 63457 95 22
x3=new Uint8Array(x2);
// converting char codes array to 8 bit array
// first 4 elements: 135 225 95 22
// then deflate with pako
// and then inflate with pako
from pako.
What is x2
for? You should use getFile()
as binary (Uint8Array
) instead of string.
So start with x3=getFile('filename')
// this should be a raw Uint8Array
Check http://www.html5rocks.com/en/tutorials/webgl/typed_arrays/#toc-fileapis for more information
from pako.
Because I need to work with 16 bit strings (which are not just text, but raw binary data taken with XMLHttpRequest).
It is later used with few other JS apps which work with 16 bit strings only (as input), not arrays.
To save size of loading data (which for example can be compressed from 4 MB to 200 kB), I use pako (because it has the best performance - other packers freeze browser on large data).
So, I convert these strings to arrays to not lost special characters while compressing and decompressing with pako, and to get best possible compression (I tested it out, and "to string" produces much larger files, so 8 bit array is best).
Later, in other script, I decompress 8 bit array back with paco and I must send this as 16 bit string to few other JS apps (as input) that do not support arrays.
Thats all the story.
from pako.
@ociebieda so x2
is a JS array containing integers in the 16 bit range, but then you store it a Uint8Array
. That will truncate all your higher numbers. Yes this will give you very small compressed results, but you're only storing half your data! It's impossible to inflate back to the original 16-bit ranges if you only ever give pako half the data in the first place.
What you should do, assuming your string really contains char codes in the 16-bit range, is store the bytes in an Uint16Array
and then create a Uint8Array
from the buffer of that, not the array.
// string contains raw 16-bit binary data encoded as 16-bit char codes, not unicode data.
function string16ToArray8(string16) {
var length = string16.length;
var array16 = new Uint16Array(length);
for (var i = 0; i < length; ++i) {
array16[i] = string16.charCodeAt(i);
}
// Reuse the memory buffer, but as an array of 8-bit values that's twice as long.
return new Uint8Array(array16.buffer);
}
Then to do the reverse:
function array8ToString16(array8) {
var array16 = new Uint16Array(array8.buffer);
var string16 = "";
var length = array16.length;
for (var i = 0; i < length; ++i) {
string16 += String.fromCharCode(array16[i]);
}
return string16;
}
from pako.
Closed - not a bug
from pako.
Related Issues (20)
- "Error: -3" when inflating chunks of data HOT 2
- How to decompress blob? HOT 2
- Deflate() creating broken zip file
- Use classes to improve tree-shaking HOT 8
- No way to read Deflate data in chunks. HOT 4
- CI: set minimal permissions on GitHub Workflow HOT 2
- Cannot install pako HOT 1
- library initialization slowing down app start
- TypeError: input.subarray is not a function HOT 1
- Docs update: mention what `inflate` and `deflate` actually return
- Incorrect JSDoc type information for optional parameters
- Expecting a new version release HOT 1
- Different 10th byte gzip header between zlib and pako on Mac OS 13 HOT 1
- License declaration in package.json and LICENSE are not the same
- GZIP pako compressed string results in "Not in GZIP format" in JAVA on decompressing HOT 1
- Inconsistency and `undefined` return in `inflate` method between versions v1 and v2
- Using static dictionary, inflateRaw "Uncaught invalid stored block lengths"
- how to gzip multiple files? HOT 1
- Not compatible with Angular v17
- Getting corrupted data on decompressing HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pako.