Comments (6)
Maybe it's an idea to add function that start multiple threads
from xlsxgrep.
I guess multiple threads is not going to work since XLSX is very heavy on memory for some reason.
$ dmesg -T| grep -E -i -B100 'killed process'
[Mon Jan 2 14:37:27 2023] Out of memory: Killed process 347179 (xlsxgrep) total-vm:30210216kB, anon-rss:30086148kB, file-rss:2228kB, shmem-rss:0kB, UID:1000 pgtables:59176kB oom_score_adj:0
from xlsxgrep.
xlsxgrep is not session limited, you can run as many sessions as your system can handle.
There might some optimization can be done to increase the speed but not too much because as you already figured out xlsx files can be very heavy on system resources sometimes even with native processing applications and there's not much can be done at the moment.
Anyway, I'll perform some test when I find some time and see.
from xlsxgrep.
Haha yes I see. 28GB RAM for 3 xlsxgrep sessions. With this little sessions it takes me a long time to get through all files. Anyway, thank you for the tool, I couldn't do my work without!
from xlsxgrep.
Yeah, 28 GB RAM is way too much. This memory issue is actually related with issue #13 you reported already.
I will hopefully provide a fix for this soon.
Cheers.
from xlsxgrep.
Memory issues are now fixed in new version.
Cheers.
from xlsxgrep.
Related Issues (12)
- Support for searching ods (libreoffice files) HOT 1
- UserWarning on searches HOT 9
- support xlsm ? HOT 1
- -l doens't work when searching through a big amount of files HOT 4
- -Z option HOT 2
- A few warning are written to stdout instead of ignored HOT 4
- Unsupported format, password protected or corrupted file HOT 4
- Tag releases HOT 2
- Request: Print to stderr? HOT 2
- Using python3.9 and xlsxgrep results in errors HOT 2
- NewFeature: Support for the -l and -L HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from xlsxgrep.