Comments (24)
I run Win10 x64 & via @javiergutierrezchamorro's FileOptimizer, which masks the errors, but when I find an applicable file that exhibits it, I can run it directly. How shall I proceed?
@javiergutierrezchamorro What parameters does FO use?
from efficient-compression-tool.
@fhanau Here you go (I apologize in advance for thread spam. If there's duplication, I can't tell.):
from efficient-compression-tool.
Perhaps it’s the very time to release a new Windows binary? Some recent commits look yummy.
from efficient-compression-tool.
Not sure if the previous issues with crashes are still present, but I have published a new release and compiled binaries for it. Enjoy!
from efficient-compression-tool.
Relatively up-to-date Windows binaries are provided in this thread.
from efficient-compression-tool.
Anything official? Problem is, 64-bit binaries seem to be Intel-only, which isn't helpful for the other ½ of us. 😞
from efficient-compression-tool.
They aren't Intel-only. SSE4 is available on many AMD processors, and AVX2 is supported by Zen. Also, x86-SSE2 in my tests was as fast as x64 default; you shouldn't have an issue unless you are running out of memory.
If you are having actual issue with binaries please describe in more detail + your CPU.
from efficient-compression-tool.
As mentioned in https://sourceforge.net/p/nikkhokkho/discussion/fileoptimizer/thread/9cd8548c/#6d83, I'm trying to speed up progress on 17 48 Mpixel PNGs on an AMD A8-6410, which apparently does support even SSE4a. How do I debug for y'all?
from efficient-compression-tool.
How exactly does the problem appear? Does it say something like "illegal instruction"?
Make sure your OS is not ancient as SSE needs OS support.
I won't be able to test this myself as I don't have AMD hardware.
from efficient-compression-tool.
Using the following very large (17Mpixel) PNG (you've been warned!):
I tried ECT64.Exe -9 -strip -progressive -recurse -keep --strict --allfilters-b --pal_sort=120 --mt-deflate
-v 16777216colors.png
EventVwr (after quite some time) gives:
Faulting application name: ECT64.Exe, version: 0.0.0.0, time stamp: 0x59784ab5
Faulting module name: ECT64.Exe, version: 0.0.0.0, time stamp: 0x59784ab5
Exception code: 0xc00000fd
Fault offset: 0x0000000000024d22
Faulting process id: 0x1df8
Faulting application start time: 0x01d31445999e1462
Faulting application path: z:\xxx\yyy\ECTTest\ECT64.Exe
Faulting module path: z:\xxx\yyy\ECT64.Exe
Report Id: 0415bdd6-dcb1-4d00-9a2a-4b72e6bf391b
Faulting package full name:
Faulting package-relative application ID:
I can find no other crash data, particularly from ECT itself (dies silently). Does someone know a better crash logger I can use?
from efficient-compression-tool.
It appears to be a stack overflow, but I don't really know what causes it.
The command line options look odd, "-v" is not an option and should cause ECT to exit.
I use GDB to debug ECT. It is particulary useful as it identifies the function which caused a crash.
You can download a windows binary at ftp://ftp.equation.com/gdb/64/gdb.exe.
To use it, just type gdb.exe path/to/ect.exe to start GDB.
Then you need to give it the following command:
run (i. e. "run -9 -strip -progressive -recurse -keep --strict --allfilters-b --pal_sort=120 --mt-deflate 16777216colors.png")
This should either complete successfully or stop if ECT crashes. If it crashes, please enter "bt full" to print the function call stack and send me the output starting at the crash.
from efficient-compression-tool.
It is strange, because -v is not comming from FileOptimizer.
from efficient-compression-tool.
Sorry, folks, -v
is a typo from my efforts to make ECT
more verbose. Edited above.
@javiergutierrezchamorro I just made up the command-line based on available options. What does FO use?
from efficient-compression-tool.
@TPS, enable Log Level to maximum in FO; and FIleOptimizer.log will contain the exact arguments used.
from efficient-compression-tool.
[New Thread 11232.0x2afc]
[New Thread 11232.0x1488]
[New Thread 11232.0x1bbc]
warning: Can not parse XML library list; XML support was disabled at compile time
[New Thread 11232.0x2ee8]
[New Thread 11232.0x14c8]
[New Thread 11232.0x2e94]
[New Thread 11232.0x2cc0]
[Thread 11232.0x1488 exited with code 0]
[Thread 11232.0x1bbc exited with code 0]
[Thread 11232.0x14c8 exited with code 0]
[Thread 11232.0x2e94 exited with code 0]
from efficient-compression-tool.
@fhanau Here you go (I don't think this is working right 😕):
#0 0x0000000000424e5a in ?? ()
No symbol table info available.
#1 0x0000000000424e5f in ?? ()
No symbol table info available.
#2 0x0000000000424e5f in ?? ()
No symbol table info available.
#3 0x00000000004250d0 in ?? ()
No symbol table info available.
#4 0x00000000004219bc in ?? ()
No symbol table info available.
#5 0x0000000000412e3c in ?? ()
No symbol table info available.
#6 0x0000000000422341 in ?? ()
No symbol table info available.
#7 0x0000000000422308 in ?? ()
No symbol table info available.
#8 0x0000000000422308 in ?? ()
No symbol table info available.
#9 0x0000000000422308 in ?? ()
No symbol table info available.
#10 0x0000000000422308 in ?? ()
No symbol table info available.
...
#1784 0x0000000000422308 in ?? ()
No symbol table info available.
...
I quit after that. What am I doing wrong?
from efficient-compression-tool.
Probably nothing - the binary appears to not have debug symbols. I have uploaded a debug binary over here, please try this again. There may still incomplete data as the binary was compiled with optimization but this will at least reveal which function caused the bug.
As using very aggressive settings on this big image is very slow, you can try to either omit the -9 or the --allfilters-b arguments, even if you need to run ECT twice, this will speed up the process.
As for the actual bug, it seems to be caused infinite recursion as the call stack shouldn't be 1784 functions. If it crashes again, you can omit any function that calls itself and only send me the top and bottom of the call stack - you don't need to send me thousands of files!
And please add the output immediately before "bt full", i. e. the output at the time of the crash.
from efficient-compression-tool.
OK, now running as before, but, since I'm just running overnight, I'll run with the full args. So far, there's:
Starting program: z:\xxx\yyy\ECTTest\ECT_x64_SSE4.exe -9 -strip -progressive -recurse -keep --strict --allfilters-b "--pal_sort=120" --mt-deflate 16777216colors.png
[New Thread 3976.0x28c0]
[New Thread 3976.0xe54]
[New Thread 3976.0xe10]
warning: Can not parse XML library list; XML support was disabled at compile time
[New Thread 3976.0x484]
[New Thread 3976.0x294c]
[New Thread 3976.0x10b0]
[New Thread 3976.0x2684]
[Thread 3976.0xe54 exited with code 0]
[Thread 3976.0xe10 exited with code 0]
More tomorrow.
from efficient-compression-tool.
@javiergutierrezchamorro I think
gives how the ECT command-line is compiled, right? Those options don't seem nearly as aggressive as mine above, even @ max FO optimization level, but they produce the same crash.
from efficient-compression-tool.
So the gdb output you posted indicates that it crashes while sorting a stack-allocated array. Unless there is an error in MinGW's std::sort function, it is likely that ECT runs out of stack space or the stack gets corrupted. This should be reproducible on with a 32-bit binary and on Intel cpus too, however.
I will try to reduce stack usage and see if this fixes the problem.
from efficient-compression-tool.
@fhanau Yet I've been running 32-bit ECT
on that file & many of its 17 Mpxl permutations for >10 days, & it's never crashed. But 64-bit ECT
will — on every 1! It's quite strange, as you say.
Would giving me some sort of 32-bit debug version give you any comparative insight?
from efficient-compression-tool.
I'm having trouble compiling this on Linux, I'd also be interested in a binary.
from efficient-compression-tool.
Is it possible to build this without xcode? Encode.ru thread does not seem to have ECT 0.8.2 binaries for osx and linux. Are they available for download anywhere?
from efficient-compression-tool.
I was able to compile this on debian v9. Binary attached.
from efficient-compression-tool.
Related Issues (20)
- Add progress information HOT 3
- compression > 10000? HOT 2
- build fail on ubuntu (google colab) HOT 2
- Could NOT find Threads (missing: Threads_FOUND) HOT 1
- Use CI to build release binaries
- How to only strip the metadata from PNG files? HOT 1
- ECT built from source for Apple Silicon is slower than release x86/64 build HOT 6
- `-strip` with gzip does not drop original name in metadata HOT 2
- Suggestion for lossless confirmation HOT 1
- Infinite loop via FileOptimizer since 0.9.2 HOT 44
- Using '-zip' option leaves temporary '..zip' file when used on folder HOT 2
- Request: Document what the `--strict` option does HOT 1
- Is building on arm64 / aarch64 supported? HOT 2
- Access violation in merge.h in windows release build only HOT 1
- [Enhancement] Animated PNG (APNG) support HOT 3
- --disable-png and --disable-jpg are ignored when -gzip is given. HOT 2
- Per file and per block multithreading don't work correctly HOT 3
- replace functions with Array
- "bad png file" if Japanese character is in path HOT 1
- Possible equivalent of `--mt-file` when recompressing files contained zip and gzip archives HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from efficient-compression-tool.