Comments (3)
This is similar to your other issue, but I think it may be out of scope. Being able to display progress would require a careful design so it works well for a number of different use cases. I consider the folder support of ECT to be a convenience feature and not its main function, so adding progress information to see how long it takes to compress one big folder is not what I plan to focus on.
If you are using ECT on larger amounts of data, I would recommend to use a script to process files one at a time, which allows you to print progress or any additional information as you go through the folder and makes it more flexible. When compressing larger amounts of data with a high compresssion setting, the run time can certainly be high, so you can run the program on a subset of the files or with a lower setting (e.g. -3) first to get a sense of how long it might take.
from efficient-compression-tool.
I use xargs to help,
find . -type f -name "*.png" -o -name "*.PNG" -o -name "*.jpg" -o -name "*.JPG" | xargs -n1 ect -3 -recurse -strip -keep --mt-file
but I have not figure out yet, to show the file names in line before optimize result output
from efficient-compression-tool.
I use this:
find . -type f -iname "*.png" -o -iname "*.jpg" -print0 | sort -z | xargs -0r -n 1 -P $(nproc) -I '{}' sh -c 'OUT=$(ect -3 -strip -keep "$1" | grep -v "^Processed") && echo "$1 --> $OUT" || echo "Error processing $1"' sh '{}'
Files are delimited by a null character to avoid any funny business with weird characters in filenames. nproc
returns the number of logical processors in your system to maximize performance, adjust as necessary. From my short testing it appears it's more efficient to run more single-threaded processes than fewer multi-threaded processes. I sort the results of find
to help give a sense of progress but mainly so it can help me locate approximately which file caused issues (with an error margin +/- of the value used for -P
) in case the error isn't catched by || echo "Error processing $1"
. The output for each file is converted to a single line because the multiple processes running at the same time could mix their multi-line output making it difficult to know to which result corresponds to which file.
However, I find the command a monstrosity and I think it would be better if ect
just showed what file(s) it was working on.
from efficient-compression-tool.
Related Issues (20)
- Could NOT find Threads (missing: Threads_FOUND) HOT 1
- Use CI to build release binaries
- How to only strip the metadata from PNG files? HOT 1
- ECT built from source for Apple Silicon is slower than release x86/64 build HOT 6
- `-strip` with gzip does not drop original name in metadata HOT 2
- Suggestion for lossless confirmation HOT 1
- Infinite loop via FileOptimizer since 0.9.2 HOT 44
- Using '-zip' option leaves temporary '..zip' file when used on folder HOT 2
- Request: Document what the `--strict` option does HOT 1
- Is building on arm64 / aarch64 supported? HOT 2
- Access violation in merge.h in windows release build only HOT 1
- [Enhancement] Animated PNG (APNG) support HOT 4
- --disable-png and --disable-jpg are ignored when -gzip is given. HOT 2
- Per file and per block multithreading don't work correctly HOT 3
- replace functions with Array
- "bad png file" if Japanese character is in path HOT 1
- Possible equivalent of `--mt-file` when recompressing files contained zip and gzip archives HOT 1
- Please add unicode support HOT 3
- Request: Distribute via homebrew HOT 1
- Prebuilt binary with PGO here HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from efficient-compression-tool.