google / zopfli Goto Github PK
View Code? Open in Web Editor NEWZopfli Compression Algorithm is a compression library programmed in C to perform very good, but slow, deflate or zlib compression.
License: Apache License 2.0
Zopfli Compression Algorithm is a compression library programmed in C to perform very good, but slow, deflate or zlib compression.
License: Apache License 2.0
Zopfli Compression Algorithm is a compression library programmed in C to perform very good, but slow, deflate or zlib compression. The basic function to compress data is ZopfliCompress in zopfli.h. Use the ZopfliOptions object to set parameters that affect the speed and compression. Use the ZopfliInitOptions function to place the default values in the ZopfliOptions first. ZopfliCompress supports deflate, gzip and zlib output format with a parameter. To support only one individual format, you can instead use ZopfliDeflate in deflate.h, ZopfliZlibCompress in zlib_container.h or ZopfliGzipCompress in gzip_container.h. ZopfliDeflate creates a valid deflate stream in memory, see: http://www.ietf.org/rfc/rfc1951.txt ZopfliZlibCompress creates a valid zlib stream in memory, see: http://www.ietf.org/rfc/rfc1950.txt ZopfliGzipCompress creates a valid gzip stream in memory, see: http://www.ietf.org/rfc/rfc1952.txt This library can only compress, not decompress. Existing zlib or deflate libraries can decompress the data. zopfli_bin.c is separate from the library and contains an example program to create very well compressed gzip files. Currently the makefile builds this program with the library statically linked in. The source code of Zopfli is under src/zopfli. Build instructions: To build zopfli, compile all .c source files under src/zopfli to a single binary with C, and link to the standard C math library, e.g.: gcc src/zopfli/*.c -O2 -W -Wall -Wextra -Wno-unused-function -ansi -pedantic -lm -o zopfli A makefile is provided as well, but only for linux. Use "make" to build the binary, "make libzopfli" to build it as a shared library. For other platforms, please use the build instructions above instead. Zopfli Compression Algorithm was created by Lode Vandevenne and Jyrki Alakuijala, based on an algorithm by Jyrki Alakuijala.
See attachment.
Original issue reported on code.google.com by [email protected]
on 2 Mar 2013 at 5:05
Attachments:
What steps will reproduce the problem?
1. In OSX, long is 8bytes-long
2. crc32's type is here unsigned long, which is problematic
Please provide any additional information below.
#if !defined(Z_U4) && !defined(Z_SOLO) && defined(STDC)
# include <limits.h>
# if (UINT_MAX == 0xffffffffUL)
# define Z_U4 unsigned
# elif (ULONG_MAX == 0xffffffffUL)
# define Z_U4 unsigned long
# elif (USHRT_MAX == 0xffffffffUL)
# define Z_U4 unsigned short
# endif
#endif
#ifdef Z_U4
typedef Z_U4 z_crc_t;
#else
typedef unsigned long z_crc_t;
#endif
Original issue reported on code.google.com by [email protected]
on 11 Jul 2013 at 6:11
https://github.com/Hello71/zopfli
Original issue reported on code.google.com by [email protected]
on 12 Nov 2013 at 7:44
What steps will reproduce the problem?
1. Run zopfli.exe on a 2gb+ file
2. Crash inside the CRC function as it tries to array-index into a null
pointer, coming from LoadFile, which failed to allocate a 2gb memory region.
What is the expected output? What do you see instead?
Expect: Error message or whatever.
Crash.
What version of the product are you using? On what operating system?
Latest build. Windows x86.
Related to http://code.google.com/p/zopfli/issues/detail?id=15.
Original issue reported on code.google.com by [email protected]
on 30 Oct 2014 at 8:53
What steps will reproduce the problem?
1. Run zopfli -zlib
2. Examine second byte of output
3. It's 01, indicating minimal compression
What is the expected output? What do you see instead?
Expect that maximal compression value is shown.
zlibcontainer.c line 56 is:
unsigned flevel = 0;
should be:
unsigned flevel = 3;
https://www.ietf.org/rfc/rfc1950.txt
FLEVEL (Compression level)
These flags are available for use by specific compression
methods. The "deflate" method (CM = 8) sets these flags as
follows:
0 - compressor used fastest algorithm
1 - compressor used fast algorithm
2 - compressor used default algorithm
3 - compressor used maximum compression, slowest algorithm
Original issue reported on code.google.com by [email protected]
on 13 Oct 2014 at 2:54
The "Invalid filename" error, more often than not is caused by a valid filename which exists in the filesystem, but is an empty file.
$ zopflipng -y aura-083.png aura-083.png
Original issue reported on code.google.com by pdknsk
on 29 Jan 2014 at 1:37
Attachments:
What steps will reproduce the problem?
1. Use zopflipng to compress up_black.png (attached) as up_black2.png:
zopflipng up_black.png up_black2.png
2. You can use GIMP (Image > Image Properties) or Mac's file info screen ⌘I
to see that the color space was changed from RGB to Grayscale.
Attached is another PNG, up_gray.png. zopflipng does not change this image's
color space:
zopflipng up_gray.png up_gray2.png
The reason why changing the color space is not desired in my case is because I
want to create a sprite using ImageMagick. However, when the sprite tiles have
different color spaces, then the colors are changed.
I have attached up.png, generated with ImageMagick 6.8.7-7 via:
convert up_gray.png up_black.png +append up.png
I have also attached up2.png, generated with ImageMagick 6.8.7-7 via:
convert up_gray2.png up_black2.png +append up2.png
(Note: ImageMagick 6.8.8-2 and ImageMagick compiled from source at revision
14367 have the same behavior.)
Notice how the colors are different in up.png and up2.png.
What is the expected output? What do you see instead?
I would like an option to *not* change the color space of the input PNG.
What version of the product are you using? On what operating system?
I am using zopflipng compiled from source @c54dc204 on Mac OS 10.9.1
'Mavericks'.
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 14 Jan 2014 at 4:33
Attachments:
After
http://code.google.com/p/zopfli/source/detail?r=337d27f25ef15a6cf34fef2acd0613fd
dc411cb1 it's no longer possible to build with MSVC. I use MSVC 2012 but my
guess is the same happens with older MSVC versions.
I'm getting:
error C3861: 'getline': identifier not
found C:\Users\xmr\Desktop\xhmikosr-zopfli\src\zopflipng\zopflipng_bin.cc 367 1
zopfli
I tried this:
src/zopflipng/zopflipng_bin.cc | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/zopflipng/zopflipng_bin.cc b/src/zopflipng/zopflipng_bin.cc
index d264c07..fd1a2b0 100644
--- a/src/zopflipng/zopflipng_bin.cc
+++ b/src/zopflipng/zopflipng_bin.cc
@@ -19,6 +19,7 @@
#include <stdlib.h>
#include <stdio.h>
+#include <string>
#include "lodepng/lodepng.h"
#include "zopflipng_lib.h"
@@ -364,7 +365,7 @@ int main(int argc, char *argv[]) {
size_t nbytes = 2;
char* text = (char*) malloc(nbytes);
printf("File %s exists, overwrite? (y/N) ", out_filename.c_str());
- int bytes_read = getline(&text, &nbytes, stdin);
+ int bytes_read = std::getline(&text, &nbytes, stdin);
confirmed = bytes_read > 0 && (text[0] == 'y' || text[0] == 'Y');
}
if (confirmed) {
but it fails:
Error 1 error C2780: 'std::basic_istream<_Elem,_Traits>
&std::getline(std::basic_istream<_Elem,_Traits>
&,std::basic_string<_Elem,_Traits,_Alloc> &)' : expects 2 arguments - 3
provided C:\Users\xmr\Desktop\xhmikosr-zopfli\src\zopflipng\zopflipng_bin.cc 368
1 zopfli
Error 2 error C2784: 'std::basic_istream<_Elem,_Traits>
&std::getline(std::basic_istream<_Elem,_Traits>
&,std::basic_string<_Elem,_Traits,_Alloc> &,const _Elem)' : could not deduce
template argument for 'std::basic_istream<_Elem,_Traits> &' from 'char
**' C:\Users\xmr\Desktop\xhmikosr-zopfli\src\zopflipng\zopflipng_bin.cc 368 1 zo
pfli
Error 3 error C2780: 'std::basic_istream<_Elem,_Traits>
&std::getline(std::basic_istream<_Elem,_Traits>
&&,std::basic_string<_Elem,_Traits,_Alloc> &)' : expects 2 arguments - 3
provided C:\Users\xmr\Desktop\xhmikosr-zopfli\src\zopflipng\zopflipng_bin.cc 368
1 zopfli
Error 4 error C2784: 'std::basic_istream<_Elem,_Traits>
&std::getline(std::basic_istream<_Elem,_Traits>
&&,std::basic_string<_Elem,_Traits,_Alloc> &,const _Elem)' : could not deduce
template argument for 'std::basic_istream<_Elem,_Traits> &&' from 'char
**' C:\Users\xmr\Desktop\xhmikosr-zopfli\src\zopflipng\zopflipng_bin.cc 368 1 zo
pfli
Original issue reported on code.google.com by XhmikosR
on 7 May 2013 at 4:09
This feature would be useful especially for older images where the timestamp
matters. PNGOUT has this feature.
What version of the product are you using? On what operating system?
Version unknown, downloaded from
http://encode.ru/threads/1717-ZopfliPNG?p=33092&viewfull=1#post33092
Windows
Original issue reported on code.google.com by [email protected]
on 28 Jul 2014 at 9:20
What steps will reproduce the problem?
1. If I compress large image (larger than 5000x5000), I have:
...
IDAT 8192 bytes 8 Kbytes
IDAT 8192 bytes 8 Kbytes
IDAT 6418 bytes 6 Kbytes
uncompress returned Z_BUF_ERROR
What is the expected output? What do you see instead?
Work
What version of the product are you using? On what operating system?
last version
Please provide any additional information below.
It's file larger than 5MB.
Original issue reported on code.google.com by [email protected]
on 18 Nov 2014 at 12:33
What steps will reproduce the problem?
zopflipng --lossy_transparent 002_Md.png
What is the expected output? What do you see instead?
Encoding error 82: color conversion to palette requested while a color isn't in
palette
Decoding error 82: color conversion to palette requested while a color isn't in
palette
There was an error
What version of the product are you using? On what operating system?
Custom build from current repository on Mac OS X 10.6.8
002_Md.png file attached
Original issue reported on code.google.com by [email protected]
on 28 Nov 2013 at 1:29
Attachments:
For example, it would be nice to be able to do:
tar -cv foo | zopfli -c > foo.tar.gz
For this example to work, Zopfli would have to support direct input as well
(not just file names), e.g.
zopfli -c -s "$(cat foo.txt)"
Original issue reported on code.google.com by [email protected]
on 29 Apr 2013 at 11:56
I've added --timelimit option to zopflipng that stops iteration after the given
number of seconds has elapsed, as measured by clock() function that's supposed
to measure CPU time used.
This allows placing hard upper limit on how long optimization will take, e.g.
when server needs to respond with optimized image on time or when impatient
user is waiting for large image to optimize.
The time limit has the same inaccuracy as number of iterations:
ZopfliLZ77Optimal() may be called several times (e.g. when strategy==3),
multiplying actual number of iterations done, and so time. I did not try to
compensate for that.
Original issue reported on code.google.com by [email protected]
on 16 Jul 2013 at 2:59
Attachments:
I am trying to get zopfli into Fedora. Fedora uses the following compiler
options by default and this is a mandatory requirement.
https://fedoraproject.org/wiki/Packaging:Guidelines#Compiler_flags
Other distros have similar packaging guidelines as well. Please fix
gcc blocksplitter.c cache.c deflate.c gzip_container.c hash.c katajainen.c
lz77.c squeeze.c tree.c util.c zlib_container.c zopfli.c -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic
/tmp/cchyEmFs.o: In function `CalculateEntropy':
/home/rahul/work/rpmbuild/SOURCES/zopfli-acc035/tree.c:79: undefined reference
to `log'
/home/rahul/work/rpmbuild/SOURCES/zopfli-acc035/tree.c:85: undefined reference
to `log'
/home/rahul/work/rpmbuild/SOURCES/zopfli-acc035/tree.c:79: undefined reference
to `log'
collect2: error: ld returned 1 exit status
Original issue reported on code.google.com by [email protected]
on 3 Mar 2013 at 8:15
What steps will reproduce the problem?
1. Run ZopfliPNG with correct parameters to optimize an image.
What is the expected output? What do you see instead?
It would be nice if some progress info was shown (e.g. similar to PNGOUT
progress info).
Currently, just "Optimizing [filename]" is shown.
What version of the product are you using? On what operating system?
Unknown version, downloaded from
http://encode.ru/threads/1717-ZopfliPNG?p=33092&viewfull=1#post33092
Windows
Please provide any additional information below.
Thanks for ZopfliPNG, I like it very much.
Original issue reported on code.google.com by [email protected]
on 28 Jul 2014 at 9:14
What steps will reproduce the problem?
1. Under CygWin when ZopfliPNG is compiled with gcc 4.8.2 or mingw-gcc 4.8.2
and executed as batch job with bash script I get core dump errors
2. When compiled with gcc 4.7.3 or mingw-gcc 4.7.3 I don't have these errors
3. Also, in make file, you use CFLAGS for CXX instead CXXFLAGS
What is the expected output? What do you see instead?
assertion "expected_data_size == 0 || testlength == expected_data_size" failed:
file "src/zopfli/deflate.c", line 272, function: AddLZ77Data
What version of the product are you using? On what operating system?
Latest source c54dc204ef42
Please provide any additional information below.
assertion "expected_data_size == 0 || testlength == expected_data_size" failed:
file "src/zopfli/deflate.c", line 272, function: AddLZ77Data
assertion "expected_data_size == 0 || testlength == expected_data_size" failed:
file "src/zopfli/deflate.c", line 272, function: AddLZ77Data
Stack trace:
Frame Function Args
00288D98 610304E2 (000000CC, 0000EA60, 000000A4, 00288DE8)
00288EA8 610DA8AD (80052C08, 80052C18, 00000000, 80052D40)
Original issue reported on code.google.com by [email protected]
on 14 Apr 2014 at 1:35
What steps will reproduce the problem?
1. compress with Zopfli
2. run DeflOpt on its output... it's sometimes further compressed
What is the expected output?
Zopfli is supposed to produce the smallest Deflate stream
What do you see instead?
It's not the case since DeflOpt can still reduce Zopfli output
What version of the product are you using?
Current source
On what operating system?
Linux
On enwik8:
*** DeflOpt V2.07 ***
*** Built on Wed Sep 5 18:56:30 2007 ***
*** Copyright (C) 2003-2007 by Ben Jos Walbeehm ***
"Z:/home/caveman/zopfli/enwik8.gz"
Number of bytes saved: 1,745 (34,995,756 --> 34,994,011) (13,960 bits)
File rewritten.
Number of files processed : 1
Number of files rewritten : 1
Total number of bytes saved: 1,745
DeflOpt is avaible here:
http://www.walbeehm.com/download/
It can be run on Linux using Wine.
Defluff is a similar tool:
http://encode.ru/threads/1214-defluff-a-deflate-huffman-optimizer
If you need a tool to compare Deflate streams, defdb can be downloaded from
here:
http://encode.ru/threads/1428-defdb-a-tool-to-dump-the-deflate-stream-from-gz-an
d-png-files
Original issue reported on code.google.com by [email protected]
on 1 Mar 2013 at 5:04
To suppress all status output. Thanks.
Original issue reported on code.google.com by pdknsk
on 19 Jun 2013 at 10:17
It would make sense to convert zopfli into a library so that, it could
be integrated into other applications, which want to compress using zopfli,
without embedded the code.
The avoids all the issues which follow embedded libraries.
I am working on a patch for it. I will post it as soon as it is done.
Original issue reported on code.google.com by [email protected]
on 5 Mar 2013 at 6:16
What steps will reproduce the problem?
1. Clone repo
2. ls
3. What the?
What is the expected output? What do you see instead?
Makefile
makefile
What version of the product are you using? On what operating system?
master
Please provide any additional information below.
Petty, but GNU likes it that way and these people in this Stackoverflow Q&A
seem to prefer it that way too. Sticking to convention here would be good.
http://stackoverflow.com/questions/12669367/should-i-name-makefile-or-makefile
Original issue reported on code.google.com by [email protected]
on 11 Jan 2014 at 7:45
What steps will reproduce the problem?
1. use verbose mode -v
2. reported block size does not include Huffman tables, is reported in bytes
(less accurate than bits).
Block size reported in verbose mode doesn't make sense, blocks are measured in
bits and not bytes and from what I've seen compared to defdb they do not
encompass the Huffman tables found at the beginning of type 2 blocks.
Original issue reported on code.google.com by [email protected]
on 2 Mar 2013 at 6:53
While playing with Zopfli v1.0.0 on various systems, I was surprised to learn
that using the same input files with the same Zopfli settings on different
systems may lead to slightly different results.
Here’s the results for an Ubuntu box:
File `tmp/test-1.js.gz` created.
Original: 83059 bytes.
Compressed: 26957 bytes.
File `tmp/test-3.js.deflate` created.
Original: 86916 bytes.
Compressed: 19156 bytes.
File `tmp/test-4.js.deflate` created.
Original: 86916 bytes.
Compressed: 19038 bytes.
File `tmp/test-5-a.js.gz` created.
Original: 86916 bytes.
Compressed: 19077 bytes.
File `tmp/test-5-b.js.gz` created.
Original: 83059 bytes.
Compressed: 26957 bytes.
On OS X:
File `tmp/test-1.js.gz` created.
Original: 83059 bytes.
Compressed: 27082 bytes.
File `tmp/test-3.js.deflate` created.
Original: 86916 bytes.
Compressed: 19132 bytes.
File `tmp/test-4.js.deflate` created.
Original: 86916 bytes.
Compressed: 19037 bytes.
File `tmp/test-5-a.js.gz` created.
Original: 86916 bytes.
Compressed: 19068 bytes.
File `tmp/test-5-b.js.gz` created.
Original: 83059 bytes.
Compressed: 27082 bytes.
This is using the official v1.0.0 release, on both systems. Zopfli seems to
create even smaller files on the Ubuntu box for some reason.
Any idea why this is happening? Can something be done about this, or is it a
WONTFIX?
Original issue reported on code.google.com by [email protected]
on 25 May 2013 at 6:29
zopfli does not include a man page
Original issue reported on code.google.com by [email protected]
on 3 Mar 2013 at 8:17
What steps will reproduce the problem?
1. The current makefile has -lm hardcoded in, this should be added optionally
What is the expected output? What do you see instead?
Haiku and perhaps other Operating Systems, have their math library built in and
thus don't require the use of -lm when building.
What version of the product are you using? On what operating system?
git master on Haiku.
Please provide any additional information below.
For now to build on Haiku I just use "sed -i 's/\-lm//' makefile"
Original issue reported on code.google.com by [email protected]
on 16 Mar 2013 at 5:53
Using the latest cppcheck git,
[zopfli\katajainen.c:192]: (error) Uninitialized variable: leaves
[zopfli\katajainen.c:228]: (error) Uninitialized struct member: pool.nodes
[zopflipng\lodepng\lodepng_util.cpp:403]: (error) Uninitialized variable: ct
[zopflipng\lodepng\lodepng_util.cpp:403]: (error) Uninitialized variable:
decoded
Original issue reported on code.google.com by XhmikosR
on 28 Jun 2013 at 7:44
I couldn't find any guidelines for contributing to Zopfli, so I have attached
my 3 patches as here on the issue tracker.
Patch #1 introduces build support for building ZopfliPNG as a shared library.
Patch #2 fixes a permissions bug; source files shouldn't be +x.
Patch #3 introduces a C API to libzopflipng.
To elaborate a big, I believe having a C API is beneficial in that it allows
the library to be used from a lot more places and languages (e.g. Go, other C
projects, Ruby, etc.), much easier (without SWIG madness). It also creates a
stable and define ABI for the library, something which C++ lacks.
All patches generated via git format-patch, and apply cleanly to master.
Comments welcome and thanks for your time!
Original issue reported on code.google.com by [email protected]
on 7 Nov 2014 at 3:44
Attachments:
zopfli -i1000000 some_file
executes almost instantaneously (as opposed to the expectation of it taking a very long time due to the high number of iterations). It is expecting --i
rather than -i
, but it doesn't make this clear when invoked.
Hi!
I was wondering if anyone could lend any guidance for how one could use zopfli
to (re)compress a zip file? I'd like to use it to repack jar files, but I don't
know of any way to combine the zopfli raw output with the proper headers.
libarchive's bsdtar can produce zip files and use an external compression
program, but it expects the program to be able to stream from stdin to stdout I
think.
I tried creating a wrapper (lets call it zopflinout):
#!/bin/sh
zopfli /dev/stdin --deflate -c
... and using
bsdtar -cv --format zip --use-compress-program zopflinout foo.zip *
... but this quickly segfaults and tells me 'Can't write to filter.'
Also (not really related, but...) here is a patch for the makefile to:
- Allow for environmental overrides for compiler variables.
- Build a static library.
- Build a shared library on Darwin.
- Use LDFLAGS.
- Actually use CXXFLAGS when building zopflipng (you have a variable for it but
don't use it in the target).
... if you want?
Original issue reported on code.google.com by [email protected]
on 1 Nov 2013 at 9:26
Attachments:
Zopflipng has no --
option, so there is no way to treat options beginning with a dash as filenames.
Please consider tagging a release every now and then as per http://semver.org/.
This would be useful for various obvious reasons, but I’m asking this
specifically because it would allow me to create a
[Homebrew](http://mxcl.github.io/homebrew/) formula so people using OS X can
install Zopfli even more easily. (All they’d have to do is enter `brew
install zopfli` in their shell.)
How about tagging the current release as v0.1.0 and then go from there?
Note: you may also want to display the version number in the help screen
(`zopfli -h`).
Original issue reported on code.google.com by [email protected]
on 19 Apr 2013 at 7:53
Currently, `gcc` is hardcoded into the Makefile. It would however be nice to be
able to specify a CC on the command-line.
Original issue reported on code.google.com by [email protected]
on 30 Apr 2013 at 3:01
I had a bug in a function calculating iterations based on file size, which
caused this.
src/zopfli/deflate.c:272: AddLZ77Data: Assertion `expected_data_size == 0 ||
testlength == expected_data_size' failed
It's not necessarily a bug, but maybe libzopfli should protect against it still.
Original issue reported on code.google.com by pdknsk
on 4 Jun 2014 at 3:20
Since I don't see it on the site, and I know that gzip uses a different crc32
from the standard "deflate" algorithm, and since I want to use it for
precompressed data, I was wondering if the deflate based output is similar
enough to the zip based output that I could use it on the site.
What I'm doing is quite simple, I have static resources that are precompressed,
and I store the filename.gz, filename.zip, and filename. With an nginix
extension, I'm able to tell it to use the precompressed ones. Since zopfli is
_better_ than even advzip(which uses 7zip's deflate) by ~1-2%, I am planning on
using it. Advzip is able to recompress gzip/zip streams to see if it's smaller
and still make it work the same to the decompressor. So does zopfli's deflate
output work for browsers that don't suppot gzip compression?
I know this is very runon, and I couldn't think of a better way to word this,
I'm sorry for it being very run-ony but it looks to be a very intersting
project, and I'd love to start using it but I need to know that one thing.
Finally as an extra question, do you guys have plans to recompress png files?
Since it's just deflate, ala advpng with your tool? Or is that out of the
question currently, because that would be an interesting extra application of
the technology. Anyway thanks for making the tool open source, and I await the
answers.
Original issue reported on code.google.com by [email protected]
on 1 Mar 2013 at 1:44
Gzip has a parallel implementation called 'pigz'.
Since zopfli needs a lot more processing power, it would be great if it could
be parallellized (if the algorithm allows it, of course).
Original issue reported on code.google.com by [email protected]
on 1 Mar 2013 at 9:05
When zopflipng is compiled 32 bit no optimization with gcc 3.4.6 20060404 (Red
Hat 3.4.6-11) or 4.4.6 20120305 (Red Hat 4.4.6-4), it cores on the attached
test file, problem.png. I am running off of the latest head,
c54dc204ef4278f949a965dc90e693799b6aae41.
zopflipng generates the following error message:
zopflipng: src/zopfli/deflate.c:272: AddLZ77Data: Assertion `expected_data_size
== 0 || testlength == expected_data_size' failed.
Aborted (core dumped)
The error does not appear in 32 bit mode when zopflipng is compiled with -O2
(as is its default in its makefile) or at all when zopflipng is compiled 64
bit. That is, the error doesn't show up in 64 bit regardless of the
optimization level.
I have done some debugging and the error appears to originate with lodepng
stomping on the stack of the CustomPNGDeflate function during a call to
lodepng_realloc (lodepng.cpp:68).
The memory I noticed being stomped is options.numiterations in
CustomPNGDeflate. To see lodepng stomping the stack, compile a 32 bit
unoptimized zopflipng, fire up gdb and set a breakpoint at zopflipng_lib.cc:50.
Grab the address of options.numiterations with 'p &options.numiterations' and
set a hardware watchpoint on the memory 'watch *(int *)0xffffcbb8'. Then let
the program run until something writes to that memory (press 'c'). You should
see some output like the following:
Hardware watchpoint 3: *(int *)0xffffcbb8
Old value = 15
New value = 80
_int_realloc (av=0xf7e943a0, oldp=0x8081a20, oldsize=80, nb=1512)
at malloc.c:5253
5253 if (__builtin_expect (oldp->size <= 2 * SIZE_SZ, 0)
Issuing the backtrace command ('bt') will show you the call stack:
(gdb) bt
#0 _int_realloc (av=0xf7e943a0, oldp=0x8081a20, oldsize=80, nb=1512)
at malloc.c:5253
#1 0xf7d761ac in __libc_realloc (oldmem=0x8081a28, bytes=1506)
at malloc.c:3826
#2 0x080510a3 in lodepng_realloc (ptr=0x8081a28, new_size=1506)
at src/zopflipng/lodepng/lodepng.cpp:68
#3 0x08055b8d in lodepng_chunk_create (out=0xffffcdc4, outlength=0xffffcdc8,
length=1418, type=0x8071fc0 "IDAT",
data=0x8081a78 "x\001\354Wan\263:\020\264\201\003\060\220\003\340$\a0Q\016") at src/zopflipng/lodepng/lodepng.cpp:2457
#4 0x0805c32f in addChunk (out=0xffffcdc4, chunkName=0x8071fc0 "IDAT",
data=0x8081a78 "x\001\354Wan\263:\020\264\201\003\060\220\003\340$\a0Q\016", length=1418) at src/zopflipng/lodepng/lodepng.cpp:4879
#5 0x0805c7ab in addChunk_IDAT (out=0xffffcdc4, data=0x808c648 "",
datasize=40385, zlibsettings=0xffffcff8)
at src/zopflipng/lodepng/lodepng.cpp:4989
#6 0x0805ed56 in lodepng_encode (out=0xffffce24, outsize=0xffffce20,
image=0xf7c5f008 "\377\377\377", w=392, h=205, state=0xffffcfd0)
at src/zopflipng/lodepng/lodepng.cpp:5771
#7 0x0805ffdf in lodepng::encode (out=std::vector of length 0, capacity 0,
in=0xf7c5f008 "\377\377\377", w=392, h=205, state=...)
at src/zopflipng/lodepng/lodepng.cpp:6214
#8 0x080600c9 in lodepng::encode (out=std::vector of length 0, capacity 0,
---Type <return> to continue, or q <return> to quit---
If you continue to run the program the watchpoint will trigger more as the
stack of CustomPNGDeflate is further abused.
If I come up with a patch I'll post it.
Original issue reported on code.google.com by [email protected]
on 31 Jul 2013 at 10:57
Attachments:
What steps will reproduce the problem?
1. Download the attached PNG.
2. On a terminal, execute: zopflipng preview.png preview.min.png
3. Use an image viewer application to determine the image DPI of `preview.png'
versus `preview.min.png'. For example, open the images in Mac's Preview.app
and press ⌘I to show the inspector.
What is the expected output? What do you see instead?
The image DPI changed from 144 DPI to 72 DPI. I expected the image DPI to
remain unchanged.
What version of the product are you using? On what operating system?
I am using zopflipng compiled from source @c54dc204 on Mac OS 10.10 'Yosemite'.
Please provide any additional information below.
Not changing the resolution of the PNG is important for creating assets for
high-DPI displays such as a Retina display.
Original issue reported on code.google.com by [email protected]
on 29 Oct 2014 at 11:12
Attachments:
haw@meikes:~/ > ls home-arbeit.tar -lah
-rw-r--r-- 1 haw haw 3.5G Apr 16 14:52 home-arbeit.tar
haw@meikes:~/ > zopfli -v home-arbeit.tar
Saving to: home-arbeit.tar.gz
zsh: segmentation fault zopfli -v home-arbeit.tar
Debian 6.0 with own Kernel
haw@meikes:~/ > uname -a
Linux meikes.wurpts-online.de 3.7.1amd64 #2 SMP Fri Jan 4 13:35:13 CET 2013
x86_64 GNU/Linux
Need me Information?
Original issue reported on code.google.com by [email protected]
on 16 Apr 2013 at 1:01
I am making use of a project called AdvanceCOMP, in it's compression options
allows zopfli (-4, or --shrink-insane). While going through PNG's for
Torchlight there are dozens of png's that end up larger. I've attached the
gloves so you can review and test it yourself as appropriate.
advpng.exe -z -4 */*.png
43653 43653 100% wardrobe/dragon_gloves.png (Bigger 45713)
89125 89125 100% wardrobe/heavyleather_boots.png (Bigger 96720)
11412 10817 94% wardrobe/heavyleather_boots_alt01.png
90564 90564 100% wardrobe/heavyleather_chest.png (Bigger 98420)
89328 89328 100% wardrobe/heavyleather_gloves.png (Bigger 97619)
At worst the compression should max at the same size; This means the problem's
root comes from some sections of the data are compressing better while others
end up being left as uncompressed. I noted this in some of my own compression
experiments years ago.
Viable solution:
With sections that have no compression on them (causing expansion) they should instead compress and find an identical matching length. This is most likely part of another match. If this at the beginning or end of another compressed section, that section will truncate to allow the uncompressed section to compress at a 1:1 rate (so long as the other match remains long enough to retain compression).
I am not aware of the full details of zlib compression, so there needs to be an
additional rule.
If in the instance that a match is found in the middle of another match, then
it should split the match into two matches avoiding the middle (on purpose) to
give the inner match to the uncompressed section. This should only happen in
the case that these 3 matches takes less space than 1 match & 1 non-match.
How much more complexity this will give I'm not sure, nor how much extra time
it will take.
Original issue reported on code.google.com by [email protected]
on 2 Nov 2013 at 9:00
Attachments:
Seems it's smaller when use RGBA all 0.
Original issue reported on code.google.com by [email protected]
on 19 Dec 2014 at 6:11
What steps will reproduce the problem?
1. Run ZopfliPNG.
What is the expected output? What do you see instead?
Product version should be shown.
Product version is not shown.
What version of the product are you using? On what operating system?
Downloaded from
http://encode.ru/threads/1717-ZopfliPNG?p=33092&viewfull=1#post33092
Windows
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 28 Jul 2014 at 8:56
There are currently two places to get version information in the zopfli
baseline:
1. The latest annotated tag via 'git describe --abbrev=0' (currently 1.0.0)
2. The zopfli shared library build target's version number (currently 1.0.1)
It seems like the shared library version number is the most up to date but it's
not particularly clear where to expect to find the current version. Could we
get a VERSION file (and cat it into the shared library filename)?
Original issue reported on code.google.com by [email protected]
on 25 Jul 2013 at 5:24
Mark Adler who wrote Zlib with Mark Nelson just expressed its feelings about
Zopfli on the [Zlib-devel] mailing list:
"That's cool, but it seems like an awful lot of effort for a small improvement.
Perhaps it's time to add a better compression method to HTTP's
accept-encoding."
I can only agree, Deflate was written around 1994 (when operating systems like
Windows 95 required 24MB of main memory and Unicode was a toddler), its
definitely time to move on!
Even ZIP archives can handle better compression algorithms (Deflate64, bzip2,
PPMD, LZMA) since years.
WebP has a search window that is way larger than Deflate (32k), HTML5 only
recommends UTF-8 encoding and Deflate is still single byte oriented... imagine
how poorly it performs on pages holding text written in Devanāgarī (3 bytes
per Unicode code point in UTF-8, main script used by nearly half of the
population in India).
Original issue reported on code.google.com by [email protected]
on 3 Mar 2013 at 7:13
I'm trying to make use of zopfli library in advanceCOMP, and I found that I
can't alloc memory by caller. If I do so, zopfli segfaults in
ZOPFLI_APPEND_DATA(0, out, outsize)
I wonder if you can change it to make use the memory allocated by caller
directly?
Original issue reported on code.google.com by [email protected]
on 18 Mar 2013 at 8:37
i have a file that beyond the memory sum.
i want't to know if i can compress the file use stream based compression
if it works how?
thanks
Original issue reported on code.google.com by [email protected]
on 11 Apr 2013 at 7:17
Same input and options should be same result for algorithm's reliability.
But squeeze.c uses random function ``Ran()'', so sometimes gets another result.
I suggest reset variables m_w and m_z at beginning of Deflate() in deflate.c.
Original issue reported on code.google.com by [email protected]
on 4 Mar 2013 at 12:30
zopfli make file hard codes compiler options. Must respect CFLAGS instead.
Original issue reported on code.google.com by [email protected]
on 3 Mar 2013 at 8:17
What steps will reproduce the problem?
1. Download the file
2. Run the command
3. Watch it crash
What is the expected output? What do you see instead?
I expect it not to crash.
What version of the product are you using? On what operating system?
Latest git on windows(cygwin).
Please provide any additional information below.
I have attached two files which cause the issue. The actual errors are:
for wisp.png:
$ zopflipng --splitting=3 --filters=01234mepb wisp.png wisp.png
Optimizing wisp.png
assertion "sublen[*length] == s->lmc->dist[lmcpos]" failed: file
"src/zopfli/lz77.c", line 187, function: TryGetFromLongestMatchCache
Aborted (core dumped)
for wisp2.png:
$ zopflipng --splitting=3 --filters=01234mepb wisp.png wisp.png
Optimizing wisp.png
Filter strategy zero: 2193428 bytes
Filter strategy one: 2265039 bytes
Filter strategy two: 2367388 bytes
assertion "s->lmc->length[lmcpos] == 1 && s->lmc->dist[lmcpos] == 0" failed:
file "src/zopfli/lz77.c", line 220, function: StoreInLongestMatchCache
Aborted (core dumped)
Original issue reported on code.google.com by [email protected]
on 22 Jun 2013 at 6:27
Attachments:
I'm having a SIGBUS error with zopfli-1.0.0 on OpenBSD/loongson (which is a
little-endian MIPS64 architecture, with strict alignment).
The following hello.c file:
#include <stdio.h>
int main(void)
{
printf("hello, world\n");
return 0;
}
triggers a SIGBUS error in zopfli:
$ zopfli hello.c
Starting program: /usr/local/bin/zopfli hello.c
Program received signal SIGBUS, Bus error.
0x0000000006987cb8 in ZopfliFindLongestMatch (s=0x7ffdca40, h=0x7ffdc988,
array=0x5336c200 "#include <stdio.h>\n\nint main(void)\n{\n\tprintf(\"hello, world\\n\");\n\n\treturn 0;\n}\n", pos=40, size=78, limit=38,
sublen=0x7ffdc780, distance=0x7ffdc9d4, length=0x7ffdc9d6)
at src/zopfli/lz77.c:126
126 while (scan < safe_end && *((size_t*)scan) == *((size_t*)match)) {
#0 0x0000000006987cb8 in ZopfliFindLongestMatch (s=0x7ffdca40, h=0x7ffdc988,
array=0x5336c200 "#include <stdio.h>\n\nint main(void)\n{\n\tprintf(\"hello, world\\n\");\n\n\treturn 0;\n}\n", pos=40, size=78, limit=38,
sublen=0x7ffdc780, distance=0x7ffdc9d4, length=0x7ffdc9d6)
at src/zopfli/lz77.c:126
same0 = Variable "same0" is not available.
#0 0x0000000006987cb8 in ZopfliFindLongestMatch (s=0x7ffdca40, h=0x7ffdc988,
array=0x5336c200 "#include <stdio.h>\n\nint main(void)\n{\n\tprintf(\"hello, world\\n\");\n\n\treturn 0;\n}\n", pos=40, size=78, limit=38,
sublen=0x7ffdc780, distance=0x7ffdc9d4, length=0x7ffdc9d6)
at src/zopfli/lz77.c:126
#1 0x0000000006988490 in ZopfliLZ77Greedy (s=0x7ffdca40,
in=0x5336c200 "#include <stdio.h>\n\nint main(void)\n{\n\tprintf(\"hello, world\\n\");\n\n\treturn 0;\n}\n", instart=1, inend=78, store=0x7ffdca60)
at src/zopfli/lz77.c:399
#2 0x0000000006982d78 in ZopfliBlockSplit (options=0x7ffdcd90,
in=0x5336c200 "#include <stdio.h>\n\nint main(void)\n{\n\tprintf(\"hello, world\\n\");\n\n\treturn 0;\n}\n", instart=0, inend=78, maxblocks=15,
splitpoints=0x7ffdcb00, npoints=0x7ffdcaf8)
at src/zopfli/blocksplitter.c:310
#3 0x00000000069855f4 in DeflateSplittingFirst (options=0x7ffdcd90, btype=2,
final=1,
in=0x5336c200 "#include <stdio.h>\n\nint main(void)\n{\n\tprintf(\"hello, world\\n\");\n\n\treturn 0;\n}\n", instart=0, inend=78, bp=0x7ffdccd7 "",
out=0x7ffdcdb0, outsize=0x7ffdcda8) at src/zopfli/deflate.c:569
#4 0x0000000006985904 in ZopfliDeflatePart (options=0x7ffdcd90, btype=2,
final=1,
in=0x5336c200 "#include <stdio.h>\n\nint main(void)\n{\n\tprintf(\"hello, world\\n\");\n\n\treturn 0;\n}\n", instart=0, inend=78, bp=0x7ffdccd7 "",
out=0x7ffdcdb0, outsize=0x7ffdcda8) at src/zopfli/deflate.c:668
#5 0x0000000006985adc in ZopfliDeflate (options=0x7ffdcd90, btype=2, final=1,
in=0x5336c200 "#include <stdio.h>\n\nint main(void)\n{\n\tprintf(\"hello, world\\n\");\n\n\treturn 0;\n}\n", insize=78, bp=0x7ffdccd7 "", out=0x7ffdcdb0,
outsize=0x7ffdcda8) at src/zopfli/deflate.c:687
#6 0x0000000006985ef8 in ZopfliGzipCompress (options=0x7ffdcd90,
in=0x5336c200 "#include <stdio.h>\n\nint main(void)\n{\n\tprintf(\"hello, world\\n\");\n\n\treturn 0;\n}\n", insize=78, out=0x7ffdcdb0, outsize=0x7ffdcda8)
at src/zopfli/gzip_container.c:96
#7 0x000000000698b52c in ZopfliCompress (options=Variable "options" is not
available.
) at src/zopfli/zopfli_lib.c:33
#8 0x000000000698b174 in main (argc=2, argv=0x7ffdce60)
at src/zopfli/zopfli_bin.c:96
Original issue reported on code.google.com by [email protected]
on 22 Jun 2013 at 11:37
What steps will reproduce the problem?
1. Run ZopfliPNG with input filename that contains non-English chars.
What is the expected output? What do you see instead?
The image should be read correctly, no matter of what chars are used in its
filename. (The same applies to writing output.)
Decoding error 48 is shown instead.
What version of the product are you using? On what operating system?
Unknown, downloaded from
http://encode.ru/threads/1717-ZopfliPNG?p=33092&viewfull=1#post33092
Windows
Please provide any additional information below.
As an example, the attached screenshot shows:
1) an error with a filename containing all capital letters from the French
alphabet that are not in the English alphabet,
2) comparison with PNGOUT (although incorrect chars are shown when processing,
the input is read and the output is saved with a correct name),
3) correct filenames in the title of the window.
There are 3 levels of failure, depending on chars used:
1) The file is read by both ZopfliPNG and PNGOUT. Example: Filename contains
"u" with umlaut.
2) The file is not read by ZopfliPNG but it is read by PNGOUT. Example: The
case described with the attached screenshot.
3) The file is read neither by ZopfliPNG nor by PNGOUT. Example: Filename
contains all Czech letters (https://en.wikipedia.org/wiki/Czech_alphabet).
Original issue reported on code.google.com by [email protected]
on 28 Jul 2014 at 10:37
Attachments:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.