Comments (5)
Can you see from the stack
which file/line is throwing the cb is not a function
error?
I think the lack of end
might be a problem with using on('data')
which is an event emitter and might not actually pull from the pullStream. So if you wanted to collect the data without an emitter you could do something like:
.pipe(filterArray)
.pipe(new Transform({
objectMode: true,
transform: (d,e,cb) => {
text += d;
cb();
}
})
.on('finish', () => resolve(text));
Here are some other hints:
parseOne(regex)
parses the first file that matches a regex..promise()
returns a promise on all the data as an array of the packets unzipped.
So you could do this instead of using the SelectFile
transform:
var streamPromiseArr = fileNames.map(async fileName => {
var stream = await sftp.get(fileName, true, 'utf8', { highWaterMark: 256 * 1024 });
var data = await stream.pipe(unzipper.ParseOne('DailyPunch')).promise();
// data = filterarray(data) ?
return data.join('');
});
from node-unzipper.
Thanks for the feedback, that gives me some food for thought. In my case, there are actually multiple files that match 'DailyPunch' inside the zip folders, so I'm guessing that eliminates the ability to use unzipper.ParseOne()? Feature request for a ParseMultiple(regex)?
I'm still experimenting with the event triggers, I did try using 'end' instead of 'finish' but they both have the same result. There is something strange happening that I can't pinpoint when downloading large zips that get broken into multiple chunks. I'll keep investigating.
This is the stack trace.
TypeError: cb is not a function
at afterWrite (_stream_writable.js:454:3)
at onwrite (_stream_writable.js:445:7)
at /Users/jonlamb/projects/node-test/node_modules/unzipper/lib/PullStream.js:59:60
at afterWrite (_stream_writable.js:454:3)
at _combinedTickCallback (internal/process/next_tick.js:144:20)
at process._tickCallback (internal/process/next_tick.js:180:9)
from node-unzipper.
I have never experienced this before so I'm not sure exactly how this happens. But it seems that the _write
method in Pullstream ends up being called without a callback (presumably by the pipe from the sftp).
I just published a version where we ignore the missing callback. Can you please check it out and see if you have any issues? You could also try to put a PassThrough
stream between the sftp client and the unzipper and see if node internally does some magic to fix the piping. Also you need to ensure that the sftp stream actually ends when the file is done.
Unrelated: You can always do .buffer().then(...)
to get the buffered output instead of your SelectedFile custom transform.
from node-unzipper.
@JonnyBoy333 did you check out the new version ?
npm install [email protected]
from node-unzipper.
Hey @JonnyBoy333 this has probably been fixed here
Latest version is: + [email protected]
Can you please verify and let me know? If this issue persist please reopen the ticket
from node-unzipper.
Related Issues (20)
- Not getting all files extracted HOT 13
- Attempt to decompress latest Maxmind GeoLite archive causes the error HOT 1
- `Open.file()` returns unicode replacement character(�) for unicode filenames HOT 1
- stream is not async iterable (DuplexWrapper)
- Corrupted unzipped files when using Extract HOT 15
- 支持哪几种格式的解压? HOT 1
- unzipper failed. Emitted 'error' event on DuplexWrapper instance at HOT 2
- Issue with Incorrect Character Display in `fileName` HOT 2
- Open s3 very slow HOT 1
- How to use Open.url with native "fetch"? HOT 8
- buffers has no license HOT 4
- Byte Range for File Within an Archive HOT 5
- uncompressedSize always return 0 if zip file is download from google drive HOT 1
- 是否可以在 arkts 中使用
- Is there a way to pause reading entries the unzipper stream? HOT 2
- TypeError: Promise.map is not a function after upgrade to 0.11.3 HOT 4
- v0.11.3 is broken ? HOT 7
- Consider bounding the stream request when streaming entries HOT 3
- Error handling with customSource not hooked up correcly? HOT 3
- unzipper 0.11.5 completely breaks reading of docx files HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from node-unzipper.