Comments (7)
In case anyone else needs to do this, I'm using this snippet to inflate concatenated gzip files using pako:
function inflateConcatenatedGzip(buffer) {
var position = 0;
var blocks = [];
do {
var inflator = new pako.Inflate();
inflator.push(buffer.slice(position));
if (inflator.err) { throw inflator.msg; }
blocks.push(inflator.result);
position += inflator.strm.total_in;
} while (inflator.strm.avail_in > 0);
return blocks;
}
from pako.
You try to use high-level api for things, that it's not designed for. You need to use files from ./lib/zlib
directly and write your own wrapper.
from pako.
In short, zlib interface is very complicated for regular users. Pako API is not designed to return multiple files. It's designed to be easy and small. Explanation is the same as for #34. I'm not sure that it's technically possible to fit all zlib usage patterns into such simple api.
But you can alwayas use zlib api directly, and make wrapper with another logic. Or, if you know how to fit additional features in pako api - create a pull request.
from pako.
modified code from @danvk, to inflate concatenated gzip arrayBuffer in modern browser.
{
const loadPako = () => window.pako || fetch(`https://unpkg.com/[email protected]/dist/pako.min.js`)
.then((response) => response.text())
.then((text) => eval(text))
.then(() => window.pako)
const concatArrayBuffer = (arrayBufferList = []) => {
const resultTypedArray = new Uint8Array(arrayBufferList.reduce((o, arrayBuffer) => o + arrayBuffer.byteLength, 0))
let byteOffset = 0
arrayBufferList.forEach((arrayBuffer) => {
const { byteLength } = arrayBuffer
resultTypedArray.set(new Uint8Array(arrayBuffer), byteOffset)
byteOffset += byteLength
})
return resultTypedArray.buffer
}
const gunzipAll = (PAKO, arrayBuffer) => {
const arrayBufferList = []
let byteOffset = 0
let byteLeft = 1
while (byteLeft > 0) {
const inflator = new PAKO.Inflate()
inflator.push(new Uint8Array(arrayBuffer, byteOffset))
if (inflator.err) throw inflator.msg
arrayBufferList.push(inflator.result)
byteOffset += inflator.strm.total_in
byteLeft = inflator.strm.avail_in
}
return concatArrayBuffer(arrayBufferList)
}
}
from pako.
Seems like a regular thing for the interface to this library needing a wrapper of some sort, and everyone needing the same wrappers.
Does that project exist somewhere?
I can't seem to find it. But I have a lot of code that is derived from things people posted in github issues. And am learning more about zlib internals than would be ideal.
Right now looking for something that has done this with a gz file so big the content needs to be streamed. I can't find that yet.
from pako.
The above code worked for me in browser but not Node.JS. It seems that strm.avail_in
never decreases in Node. With the added sanity check that position < buffer.byteLength
the code completes with corrupted results. A comparison of browser and Node.JS execution shows different successive position
values for the same file. However on Node.JS you can use stream.pipe(zlib.createGunzip())
(but not zlib.inflateSync
).
from pako.
@wholenews if you're using code from #35 (comment), then my guess is you used Node.js Buffer.buffer
directly as ArrayBuffer
, the problem would be some Node.js Buffer is internally shared, and need code like toArrayBuffer to convert them correctly.
from pako.
Related Issues (20)
- "Error: -3" when inflating chunks of data HOT 2
- How to decompress blob? HOT 2
- Deflate() creating broken zip file
- Use classes to improve tree-shaking HOT 8
- No way to read Deflate data in chunks. HOT 4
- CI: set minimal permissions on GitHub Workflow HOT 2
- Cannot install pako HOT 1
- library initialization slowing down app start
- TypeError: input.subarray is not a function HOT 1
- Docs update: mention what `inflate` and `deflate` actually return
- Incorrect JSDoc type information for optional parameters
- Expecting a new version release HOT 1
- Different 10th byte gzip header between zlib and pako on Mac OS 13 HOT 1
- License declaration in package.json and LICENSE are not the same
- GZIP pako compressed string results in "Not in GZIP format" in JAVA on decompressing HOT 1
- Inconsistency and `undefined` return in `inflate` method between versions v1 and v2
- Using static dictionary, inflateRaw "Uncaught invalid stored block lengths"
- how to gzip multiple files? HOT 1
- Not compatible with Angular v17
- Getting corrupted data on decompressing HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pako.