Lizard (formerly LZ5) is an efficient compressor with very fast decompression. It achieves compression ratio that is comparable to zip/zlib and zstd/brotli (at low and medium compression levels) at decompression speed of 1000 MB/s and faster.
License: Other
Hi inikep,
as a future enhancement, could you consider adding bigger windows than 256MB to your superb Lizard?!
I just did it, despite the general trend to work with smaller windows (and more threads) and the lack of computational power for gigantic windows.
The toy is called 'Ryuugan+' - a strong and fully functional one, it inherits all the good stuff from 'Washigan+' - my mainstay. Haven't been happy with the limited Eagleeye's 512MB sliding window, so the successor Dragoneye adds to all previous windows and the 37bit one - 128GB. The underlying notion is to have (in the future) one compressor able to see deeply in the whole English Wikipedia XML dump from start to the end - it steadily is growing 2GB per year, now it is 60GB.
The idea is, when the right time comes, the compile&source of such performer to be available - not to rush and write when the machine nomenclature matches the requirements for 15GB+60GB=75GB RAM.
Looking back, say, 10 years, many tools are blind to the rich resources that machines of today offer. It is good, for a change, to have the software ready beforehand, no?
Nakamichi_Ryuugan+_(32bit_64bit_executables_source).zip 5.87 MB (6,160,495 bytes):
https://drive.google.com/file/d/0BzKgu_YpO6uZYnpGY0d0NHNHQWc/view?usp=sharing
Booklet_Ryuugan+.pdf 763 KB (781,446 bytes):
https://drive.google.com/file/d/0BzKgu_YpO6uZb1VsSC02cE9WUlk/view?usp=sharing
F:\TEXTUAL_MADNESS\OLD_DOCs\Nakamichi_Ryuugan+_(32bit_64bit_source)>"Nakamichi_Ryuugan+_(1xQWORD+1xXMM)_Intel_15.0_32bit_SSE41.exe" University_of_Canterbury_The_Calgary_Corpus.tar
Nakamichi 'Ryuugan+', written by Kaze, based on Nobuo Ito's LZSS source, babealicious suggestion by m^2 enforced, muffinesque suggestion by Jim Dempsey enforced.
Note0: Nakamichi 'Dragoneye' is 100% FREE, licenseless that is.
Note1: Hamid Buzidi's LzTurbo ([a] FASTEST [Textual] Decompressor, Levels 19/29/39) retains kingship, his TurboBench (2017-Apr-07) proves the supremacy of LzTurbo, Turbo-Amazing!
Note2: Conor Stokes' LZSSE2 ([a] FASTEST Textual Decompressor, Level 17) is embedded, all credits along with many thanks go to him.
Note3: 'Ryuugan' predecessors are Washigan, Okamigan, Zato, Tsubame, Tengu-Tsuyo, Tengu, Rakka, Kokuen, Kinroba, Yoko, Kinutora, Jiten, Butsuhira, Suiken, Keigan, Kumataka, Washi, Aratama, Hitomi, Nekomata, Kitsune, Kinezumi, Sanbashi, Kaiko, Inazuma, Zangetsu, Hanabi, Hanazakari, Sanshi and Kaidanji.
Note4: This compile can handle files up to 5120MB.
Note5: The matchfinder/memmem() is 'Railgun_Trolldom'.
Note6: Instead of '_mm_loadu_si128' '_mm_lddqu_si128' is used.
Note7: The lookahead 'Tsuyo' heuristic which looks one char ahead is applied thrice, still not strengthened, though.
Note8: The compile made 2017-May-04, the decompression time measuring is done in 16x8 passes choosing the top score from 64 back-to-back runs - the goal - to enter [maximal] Turbo Mode.
Note9: Just to reduce the codesize, the 3xQWORD become (in this compile) 1xQWORD+1xXMMWORD.
NoteA: Please send me (at [email protected]) decompression results obtained on machines with fast CPU-RAM subsystems.
Current priority class is REALTIME_PRIORITY_CLASS.
Allocating Source-Buffer 3 MB ...
Allocating Source-Buffer 3 MB (REVERSED) ...
Allocating Target-Buffer 35 MB ...
Allocating Verification-Buffer 3 MB ...
Compressing 3,265,536 bytes ...
/; Each rotation means 64KB are encoded; Speed: 000,588B/s; Done 100%; Compression Ratio: 2.67:1; Matches(24/32/48): 4,233/317/1,483; Washigan Matches(5bytes offsets): 8
NumberOfFullLiterals (lower-the-better): 8276
Tsuyo_HEURISTIC_APPLIED_thrice_back-to-back: 14
NumberOf(Tiny)Matches[Micro]Window (4)[16B]: 3852
NumberOfMatches[Bheema]Window [128GB window]: 1
RAM-to-RAM performance: 588 B/s.
Compressed to 1,223,800 bytes.
Source-file-Hash(FNV1A_YoshimitsuTRIAD) = 0xe6d1,9b78
Target-file-Hash(FNV1A_YoshimitsuTRIAD) = 0xeacf,b4c9
Decompressing 1,223,800 (being the compressed stream) bytes ...
RAM-to-RAM performance: 494 MB/s.
Verification (input and output sizes match) OK.
Verification (input and output blocks match) OK.
LZSSE2: Compressing with LZSSE2 (level 17) 3,265,536 bytes ...
LZSSE2: Compressed to 1,142,536 bytes.
LZSSE2: RAM-to-RAM performance: 376 KB/s.
LZSSE2: Decompressing 1,142,536 bytes (being the compressed stream) ...
LZSSE2: RAM-to-RAM performance: 671 MB/s.
LZSSE2: Verification (input and output sizes match) OK.
LZSSE2: Verification (input and output blocks match) OK.
LZSSE2 vs Nakamichi 'Ryuugan', c.size: 0.93x
LZSSE2 vs Nakamichi 'Ryuugan', d.rate: 1.36x
07/02/2017 10:06 AM 3,265,536 University_of_Canterbury_The_Calgary_Corpus.tar
07/02/2017 10:06 AM 667,746 University_of_Canterbury_The_Calgary_Corpus.tar.cc.nz
07/02/2017 10:06 AM 768,579 University_of_Canterbury_The_Calgary_Corpus.tar.co.nz
07/02/2017 10:06 AM 1,142,536 University_of_Canterbury_The_Calgary_Corpus.tar.L17.LZSSE2
07/02/2017 10:06 AM 853,356 University_of_Canterbury_The_Calgary_Corpus.tar.L9Dict512.xz
07/02/2017 10:06 AM 1,028,558 University_of_Canterbury_The_Calgary_Corpus.tar.M49Block256.lz5
07/02/2017 10:06 AM 1,079,861 University_of_Canterbury_The_Calgary_Corpus.tar.method29.zpaq
07/02/2017 10:06 AM 683,957 University_of_Canterbury_The_Calgary_Corpus.tar.method59.zpaq
07/02/2017 10:06 AM 1,015,733 University_of_Canterbury_The_Calgary_Corpus.tar.MX9.zip
07/02/2017 10:06 AM 853,789 University_of_Canterbury_The_Calgary_Corpus.tar.MX9Dict512.7z
08/27/2017 04:42 PM 1,223,800 University_of_Canterbury_The_Calgary_Corpus.tar.Nakamichi
07/02/2017 10:06 AM 784,803 University_of_Canterbury_The_Calgary_Corpus.tar.O16.PPMd_varI
07/02/2017 10:06 AM 796,337 University_of_Canterbury_The_Calgary_Corpus.tar.O6.PPMd_varI
07/02/2017 10:06 AM 825,124 University_of_Canterbury_The_Calgary_Corpus.tar.ST6Block512.bsc
04/06/2017 08:44 PM 152,089 alice29.txt
08/27/2017 12:48 PM 56,526 alice29.txt.L17.LZSSE2
08/27/2017 12:48 PM 67,579 alice29.txt.Nakamichi
04/06/2017 08:44 PM 260,569 Fahrenheit_451_-_Ray_Bradbury.txt
08/27/2017 12:50 PM 100,387 Fahrenheit_451_-_Ray_Bradbury.txt.L17.LZSSE2
08/27/2017 12:50 PM 115,818 Fahrenheit_451_-_Ray_Bradbury.txt.Nakamichi
04/06/2017 08:44 PM 1,820,160 Fleurs_du_mal.tar
08/27/2017 10:58 AM 557,422 Fleurs_du_mal.tar.L17.LZSSE2
08/27/2017 10:58 AM 414,512 Fleurs_du_mal.tar.Nakamichi
04/06/2017 08:44 PM 8,798 IBM_Deep_Blue_vs_Kasparov.pgn
08/27/2017 12:49 PM 4,803 IBM_Deep_Blue_vs_Kasparov.pgn.L17.LZSSE2
08/27/2017 12:49 PM 5,153 IBM_Deep_Blue_vs_Kasparov.pgn.Nakamichi
04/06/2017 08:44 PM 3,680,256 Resident_Evil_(10-books).tar
08/27/2017 02:49 PM 1,343,243 Resident_Evil_(10-books).tar.L17.LZSSE2
08/27/2017 02:49 PM 1,289,461 Resident_Evil_(10-books).tar.Nakamichi
04/06/2017 08:44 PM 698,072 The_Death_Ship_-_B.Traven.pdf.txt
08/27/2017 12:55 PM 258,683 The_Death_Ship_-_B.Traven.pdf.txt.L17.LZSSE2
08/27/2017 12:55 PM 277,889 The_Death_Ship_-_B.Traven.pdf.txt.Nakamichi
07/02/2017 10:06 AM 4,445,260 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt
08/27/2017 12:44 PM 1,390,894 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt.L17.LZSSE2
08/27/2017 12:44 PM 1,356,427 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt.Nakamichi
Hi,
I am a happy user of the Lizard codec (really shine in some of my use cases inside Blosc). However, lately I am getting some complains from valgrind (they do not happen at all with the other codecs inside Blosc, like LZ4, Zlib or Zstd). The final result is fine; do you think this is something to be worried about? Using Lizard v1.0 here on a Linux box (Ubuntu 16.04).
Thanks for your fine work!
$ valgrind tests/test_btune
==30189== Memcheck, a memory error detector
==30189== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==30189== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==30189== Command: tests/test_btune
==30189==
==30189== Thread 5:
==30189== Conditional jump or move depends on uninitialised value(s)
==30189== at 0x4E926B8: Lizard_compress_fastBig (lizard_parser_fastbig.h:85)
==30189== by 0x4E926B8: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E926B8: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
==30189== Conditional jump or move depends on uninitialised value(s)
==30189== at 0x4E92AC6: Lizard_compress_fastBig (lizard_parser_fastbig.h:138)
==30189== by 0x4E92AC6: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E92AC6: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
==30189== Thread 4:
==30189== Conditional jump or move depends on uninitialised value(s)
==30189== at 0x4E92AD1: Lizard_compress_fastBig (lizard_parser_fastbig.h:138)
==30189== by 0x4E92AD1: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E92AD1: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
==30189== Conditional jump or move depends on uninitialised value(s)
==30189== at 0x4E926BF: Lizard_compress_fastBig (lizard_parser_fastbig.h:85)
==30189== by 0x4E926BF: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E926BF: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
==30189== Thread 3:
==30189== Conditional jump or move depends on uninitialised value(s)
==30189== at 0x4E92AE5: Lizard_compress_fastBig (lizard_parser_fastbig.h:138)
==30189== by 0x4E92AE5: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E92AE5: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
==30189== Conditional jump or move depends on uninitialised value(s)
==30189== at 0x4E92AF0: Lizard_compress_fastBig (lizard_parser_fastbig.h:140)
==30189== by 0x4E92AF0: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E92AF0: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
==30189== Conditional jump or move depends on uninitialised value(s)
==30189== at 0x4E92B02: Lizard_compress_fastBig (lizard_parser_fastbig.h:142)
==30189== by 0x4E92B02: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E92B02: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
==30189== Use of uninitialised value of size 8
==30189== at 0x4E92B08: Lizard_compress_fastBig (lizard_parser_fastbig.h:143)
==30189== by 0x4E92B08: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E92B08: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
==30189== Conditional jump or move depends on uninitialised value(s)
==30189== at 0x4E926D1: Lizard_compress_fastBig (lizard_parser_fastbig.h:85)
==30189== by 0x4E926D1: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E926D1: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
==30189== Conditional jump or move depends on uninitialised value(s)
==30189== at 0x4E926D8: Lizard_compress_fastBig (lizard_parser_fastbig.h:87)
==30189== by 0x4E926D8: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E926D8: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
==30189== Conditional jump or move depends on uninitialised value(s)
==30189== at 0x4E926EB: Lizard_compress_fastBig (lizard_parser_fastbig.h:89)
==30189== by 0x4E926EB: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E926EB: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
==30189== Use of uninitialised value of size 8
==30189== at 0x4E926ED: Lizard_compress_fastBig (lizard_parser_fastbig.h:90)
==30189== by 0x4E926ED: Lizard_compress_generic (lizard_compress.c:515)
==30189== by 0x4E926ED: Lizard_compress_extState (lizard_compress.c:592)
==30189== by 0x4E93367: Lizard_compress (lizard_compress.c:602)
==30189== by 0x4E5025D: lizard_wrap_compress (blosc.c:398)
==30189== by 0x4E5025D: blosc_c (blosc.c:733)
==30189== by 0x4E50F0A: t_blosc (blosc.c:2155)
==30189== by 0x54CA6B9: start_thread (pthread_create.c:333)
==30189==
I'm not realy sure if this is issue at all, but I've got idea one may sometimes want to do a progressive block decoding. Say, specs say block could be fairly large and for e.g. networking it implies considerable latency, well over of network packet size. So I could imagine someone may want to try progressive block decoding to reduce latency a bit.
However, as far as I understand, current stream format assumes receiver have to receive complete block before being able to decode it reasonably, because some data required to begin decoding could be quite close to end of block. I'm not really sure if it actually a problem if such uses were not primary goal and it is somewhat advanced/uncommon usage scenario. I'm also not sure how it could affect entropy coding, IIRC you've told it is easier the way you do it right now.
Looking at the repo's past releases, I saw it does not quite follow either semantic versioning or calendar versioning, i.e. v1.0 is newer than v2.0.
If we are ever going to release another version, please bump the version to something like v3.0.
Thanks!
Hey there,
I have come across a vulnerability in the lizard decompressor, whereby a negative size integer is passed to memcpy(). This occurs in lib/lizard_decompress_liz.h:207.
memcpy(op, ctx->literalsPtr, length);
Since memcpy() takes the length and treats it as a size_t, this results in an excessive copy from ctx->literalsPtr into op.
void *memcpy(void *dest, const void *src, size_t n);
This was found while fuzzing commit 6a1ed71 (current latest as of this date).
By controlling the number of bytes passed as length, this results in memory corruption which can lead to execution control.
Find the POC file here: poc.zip.
As follows is the ASAN output:
$ ./lizard_afl -d crash.file /dev/null
=================================================================
==26522==ERROR: AddressSanitizer: negative-size-param: (size=-8)
#0 0x4ae273 in __asan_memcpy (/root/lizard/lizard_afl+0x4ae273)
#1 0x51a9b2 in Lizard_decompress_LIZv1 /root/lizard/programs/../lib/lizard_decompress_liz.h:207:5
#2 0x51a9b2 in Lizard_decompress_generic /root/lizard/programs/../lib/lizard_decompress.c:241
#3 0x51a9b2 in Lizard_decompress_safe_usingDict /root/lizard/programs/../lib/lizard_decompress.c:362
#4 0x634386 in LizardF_decompress /root/lizard/programs/../lib/lizard_frame.c:1197:31
#5 0x645fc1 in LizardIO_decompressLizardF /root/lizard/programs/lizardio.c:676:26
#6 0x645fc1 in selectDecoder /root/lizard/programs/lizardio.c:767
#7 0x645fc1 in LizardIO_decompressSrcFile /root/lizard/programs/lizardio.c:803
#8 0x644379 in LizardIO_decompressDstFile /root/lizard/programs/lizardio.c:831:5
#9 0x643b80 in LizardIO_decompressFilename /root/lizard/programs/lizardio.c:850:30
#10 0x64caf8 in main /root/lizard/programs/lizardcli.c:560:32
#11 0x7f4d32b69f44 in __libc_start_main /build/eglibc-ripdx6/eglibc-2.19/csu/libc-start.c:287
#12 0x419cb5 in _start (/root/lizard/lizard_afl+0x419cb5)
0x7f4d33c1e86e is located 110 bytes inside of 131072-byte region [0x7f4d33c1e800,0x7f4d33c3e800)
allocated by thread T0 here:
#0 0x4c418c in calloc (/root/lizard/lizard_afl+0x4c418c)
#1 0x636a0b in LizardF_decodeHeader /root/lizard/programs/../lib/lizard_frame.c:840:33
SUMMARY: AddressSanitizer: negative-size-param (/root/lizard/lizard_afl+0x4ae273) in __asan_memcpy
==26522==ABORTING
In the example above, with a memcpy size of -8, this value is interpreted as an unsigned integer and the memcpy continues until there is a page fault due to unmapped memory and the process terminates:
root at s127422 in ~/lizard (lizard●●)
$ ./lizard -d crash.file /dev/null
[1] 17128 segmentation fault ./lizard -d crash.file /dev/null
To demonstrate that the memcpy size value can be controlled by an attacker, I have attached a series of crashes that result from a differing memcpy size value.
lizard_crashes.zip
After calling:
if(Lizard_stream_t *lizard=Lizard_createStream(Mid(compression_level, LIZARD_MIN_CLEVEL, LIZARD_MAX_CLEVEL)))
I get some uninitialized values, like lizard->end
which caused crashes in subsequent call to 'Lizard_compress_continue'
After I changed the code to zero the context, the crashes disappeared
Lizard_stream_t* Lizard_initStream(Lizard_stream_t* ctx, int compressionLevel)
{
..
if (!ctx)
{
size_t size = sizeof(Lizard_stream_t) + hashTableSize + chainTableSize + LIZARD_COMPRESS_ADD_BUF + LIZARD_COMPRESS_ADD_HUF;
ctx = (Lizard_stream_t*)malloc(size);
if (!ctx) { printf("ERROR: Cannot allocate %d MB (compressionLevel=%d)\n", (int)(size)>>20, compressionLevel); return 0; }
memset(ctx, 0, size); // <----------HERE
LIZARD_LOG_COMPRESS("Allocated %d MB (compressionLevel=%d)\n", (int)(size)>>20, compressionLevel);
ctx->allocatedMemory = size;
Also your codes calculated this "sizeof(Lizard_stream_t) + hashTableSize + chainTableSize + LIZARD_COMPRESS_ADD_BUF + LIZARD_COMPRESS_ADD_HUF" several times, it's better to compute this once into a helper variable first.
Also I suggest removing all "printf" calls from your library, and either use some error code result enums, or an optional callback which accepts the messages.
Default stack size on Windows is 1 MB
and size of LZ5_stream_t exceeds that limit
I had to change:
to
in order to prevent crashes.
I suggest this should be set to 1 by default
Some symbols like HUF_decompress, FSE_decompress have identical names to the same symbols in ZSTD library.
This prevents using Lizard and ZSTD in the same binary.
Hi,
now that I have access to two modern and fast (more in the link) machines, could @inikep you provide the lzbench binary, please.
It would be great if updated with latest Brotli (the 30bit window) and Zstd!
Wanna benchmark 2+GB testfiles:
http://www.sanmayce.com/Nakamichi/index.html#2020Dec19
For reasons unknown to man, x86 Windows is still a thing. I didn't notice any issues when building the 32bit version. Please add x86 builds to the releases for people who don't want to build from source.
Thanks!
In this document, extra length of literals should be read from Lengths_Stream, but if we look at source code, extra length seems to be read from Literals_Stream.
Is this correct specification?
/* get literal length */
token = *ctx->flagsPtr++;
if ((length=(token & RUN_MASK_LZ4)) == RUN_MASK_LZ4) {
if (unlikely(ctx->literalsPtr > iend - 5)) { LIZARD_LOG_DECOMPRESS_LZ4("0"); goto _output_error; }
length = *ctx->literalsPtr;
if unlikely(length >= 254) {
if (length == 254) {
length = MEM_readLE16(ctx->literalsPtr+1);
ctx->literalsPtr += 2;
} else {
length = MEM_readLE24(ctx->literalsPtr+1);
ctx->literalsPtr += 3;
}
}
length += RUN_MASK_LZ4;
ctx->literalsPtr++;
if (unlikely((size_t)(op+length)<(size_t)(op))) { LIZARD_LOG_DECOMPRESS_LZ4("1"); goto _output_error; } /* overflow detection */
if (unlikely((size_t)(ctx->literalsPtr+length)<(size_t)(ctx->literalsPtr))) { LIZARD_LOG_DECOMPRESS_LZ4("2"); goto _output_error; } /* overflow detection */
}The headers say:
compressionLevel : Recommended values are between 4 and 9, although any value between 0 and 16 will work.
0 means "use default value" (see lz5hc.c).
Values >16 behave the same as 16.
However the LZ5HC_MAX_CLEVEL is defined as 15
Also:
* Default value is 14, for 16KB, which nicely fits into Intel x86 L1 cache
*/
#define LZ5_MEMORY_USAGE 20
it says default is 14 but defined as 20
The install logic in lib/Makefile seams to forget to install the lizard_decompress.h header, is that one public?
Seen with Lizard 1.0.0.
Reproduce:
make
make PREFIX=$HOME/lizard installinclude contains:
lizard_common.h lizard_compress.h lizard_frame.h
Hi, @inikep !
I really appreciate your works and I want to ask you:
Do you have any clear explanations of ZSTD, LZ4 and Lizard?
I want to explain their internals in my coursework.All references would be included.
Hello.
I found a invalid memory access in LZ5 v2.0.
Please confirm.
Thanks.
Summary
OS: ubuntu-14.04-64bit
version: Lz5 2.0
POC Download: invalid memory access
test code: examples/blockStreaming_lineByLine.c,compile with ASan.
details:
AddressSanitizer:DEADLYSIGNAL
=================================================================
==181087==ERROR: AddressSanitizer: SEGV on unknown address 0x7f920ce206be (pc 0x7f914d4092c0 bp 0x7ffeaa32fd10 sp 0x7ffeaa32f4b8 T0)
==181087==The signal is caused by a READ memory access.
#0 0x7f914d4092bf /build/eglibc-ripdx6/eglibc-2.19/string/../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:2714
#1 0x4b9852 in __asan_memcpy /local/mnt/workspace/clang_nightly/plain/llvm/utils/release/final/llvm.src/projects/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cc:23:3
#2 0x5eea7f in MEM_read32 /home/yangq/testapp/lizard-2.0/examples/../lib/entropy/mem.h:173:14
#3 0x52350e in LZ5_compress_fastSmall /home/yangq/testapp/lizard-2.0/examples/../lib/lz5_parser_fastsmall.h:94:25
#4 0x52350e in LZ5_compress_generic /home/yangq/testapp/lizard-2.0/examples/../lib/lz5_compress.c:513
#5 0x52350e in LZ5_compress_continue /home/yangq/testapp/lizard-2.0/examples/../lib/lz5_compress.c:586
#6 0x5f0cad in test_compress /home/yangq/testapp/lizard-2.0/examples/blockStreaming_lineByLine.c:67:34
#7 0x5f098b in main /home/yangq/testapp/lizard-2.0/examples/blockStreaming_lineByLine.c:177:9
#8 0x7f914d2d4f44 in __libc_start_main /build/eglibc-ripdx6/eglibc-2.19/csu/libc-start.c:287
#9 0x419eeb in _start (/home/yangq/testapp/lizard-2.0/examples/lineCompress_asan+0x419eeb)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /build/eglibc-ripdx6/eglibc-2.19/string/../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:2714
==181087==ABORTING
It seems 'match' or 'base' (lz5_parse_fastsmall.h:90) access invalid memory in some cases.
It may cause a crash or denial of service.
I tried to add lizard to Far manager multirac and Total/Double commander Multi-Arc plugin/s. These plugins utilize original compressors/archivers to list, extract and manipulate archives. The thing is there is no listing command in lizard. Would be beneficial to add it as it would help to make more popular amongst regular users not only compression fans.
Something like in gzip -l:
$ gzip -l lizard-1.0.help.gz
compressed uncompressed ratio uncompressed_name
961 1898 51.2% lizard-1.0.help
with uncompressed name in it so it didn't need additional, separate commands like sed/awk/whatever to create such listing.
I want to implement an lz5 encoder. Sadly there doesn't seem to be any format description beyond the source code. Is the lz5 format stable enough for you to document things already?
Compiler: Visual Studio 2017 64-Bit Community Edition
OS: Windows 10 64-bit
Binary size: my version lizard-MyBuild.exe (200+ KB) v.s. official version lizard.exe (500+KB)
File to compress: some Android firmware image (namely, "system-other.img")
As you can see from the results attached below, compression ratio of lizard-MyBuild.exe is always not as good as the lizard.exe given the same level (e.g., 25, 29, 45, 49). Any idea on this difference?
Compression ratio of lizard-MyBuild.exe:
Compression ratio of lizard.exe:
Example:
==341==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x41021543 in Lizard_compress_fastSmall obj-x86_64-linux-gnu/../contrib/lizard/lib/lizard_parser_fastsmall.h:83:45
#1 0x41021543 in Lizard_compress_generic obj-x86_64-linux-gnu/../contrib/lizard/lib/lizard_compress.c:506:19
#2 0x41021543 in Lizard_compress_extState obj-x86_64-linux-gnu/../contrib/lizard/lib/lizard_compress.c:592:12
#3 0x4102c480 in Lizard_compress obj-x86_64-linux-gnu/../contrib/lizard/lib/lizard_compress.c:602:13
As of 1.5 RC (9e8d558) levels are somewhat confusing.
Normally when app-level dev calls library, it is expected higher level gives better ratio at cost of slower compression speed. Builtin help of LZ5 also suggests it is a case:
-1...-18 : compression level; higher number == more compression but slower
However, it isn't like this in LZ5 v2:
$ ./lz5 -b9 lz5
9#lz5 : 192320 -> 102547 (1.875), 1.2 MB/s , 404.0 MB/s
$ ./lz5 -b10 lz5
10#lz5 : 192320 -> 113877 (1.689), 11.1 MB/s , 358.8 MB/s
So increasing level from 9 to 10 quite unxepectedly boosts compression speed and drops ratio.
I guess either levels may need to be fine-tuned to behave like expected or, maybe, devs should get explicit knowledge these are 2 different sub-algos, not sure which option is better.
It should be quite obvious for first time users to see some simplest APIs (to compress/uncompress buffer without dealing with any file io). Strangely, lizard has links to file format, but no clues to basic api use (is it in examples, which one is the "hello world"?. I've been using zlib for more than a decade, and I've never needed to know its file format, perhaps same for 90% of its users.
IMO, it should be on the main page to see how to use the library.
In commit c1f99fe it appears this variable was renamed to ML_RUN_BITS_LIZv1 which no longer exists.
lizard/lib/lizard_decompress_liz.h
Line 106 in 3aef793
In the branchless code, I am getting a MSAN error which I am trying to track down which is probably a non-issue.
Recently, someone asked about an LZ5 plugin for Squash, which pushed me to speak about something which has been bothering me for a while. LZ5 seems like a very interesting project, and AFAICT you've done an excellent job with it. However, copying part of my answer to that issue:
Ever since LZ5 was released I've been uncomfortable with it because of the name. "LZ5" implies that it is the "successor to LZ4" (i.e., LZ4 2.0), or at least an "upgraded" version, which isn't fair to LZ4. LZ5 simply makes different trade-offs.
To be clear, I'm completely fine with the idea of forking LZ4 to create a new codec; it's only the project name I'm uncomfortable with.
There are a lot of problems with implying such a link with LZ4, such as
People are likely to go to LZ4 for support, which isn't fair to the authors of LZ4.
My favorite example of the type of issue this could cause is a comment from SQLite:
The default prefix used to be "sqlite_". But then Mcafee started using SQLite in their anti-virus product and it started putting files with the "sqlite" name in the c:/temp folder. This annoyed many windows users. Those users would then do a Google search for "sqlite", find the telephone numbers of the developers and call to wake them up at night and complain. For this reason, the default name prefix is changed to be "sqlite" spelled backwards. So the temp files are still identified, but anybody smart enough to figure out the code is also likely smart enough to know that calling the developer will not help get rid of the file.
If people think there is a link between LZ4 and LZ5, any problems with LZ5 are likely to influence their perception of LZ4, which isn't fair.
LZ4 has created a very good image of itself by being really good at what it does. Other projects should be able to build their own reputations based on their own quality instead of using LZ4 for a head-start.
With all that said, I want to be very clear that, AFAICT (I haven't really looked at LZ5's code) everything seems to be pretty well-done. However, all the points above still stand, even if LZ5
I'm going to assume that LZ5 does, and continues to do, all those things. Every indication I've seen is that they do. No matter what else they do, though, the "LZ5" name still muddies the waters.
To be clear, these are my own opinions. I don't think we should involve Yann in this conversation (especially not publicly), as it would put him in a very difficult position.
I'd like to add that, because of the name, I've been hesitant to add an LZ5 plugin. It's quite possible other people will have the same concern, so it may actually be in LZ5's best interest to change their name.
In the end, I think that yes, I'd be willing to accept a patch for an LZ5 plugin, so long as the documentation made its relationship with LZ4 extremely clear. I'd feel a lot better about it if LZ5 had a different name, though.
The library is very impressive as it looks like a pareto frontier.
But I see there are no updates in about half a year.
E.g. trivial fix for UBSan did not get merged: #33
It makes me wondering about the perspectives of this project.
Hi, inikep, happy new 2016!
Your lz5 fills the gap between lz4 and Zstd, nice.
However, your next statement is valid mainly for mixed/binary data, yes? Or to be more precise, for compressors using up to 4MB dictionary, yes?
In my experiments there is no open-source bytewise compressor that gives better ratio than lz5hc.
For textual e.g. English texts it is not in my quick tests, my latest&last Nakamichi 'Goldenboy' yields better ratio while being faster in decompression (try Skylake):
Note: I used for all files below '>lz5 -18 infile' command line.
D:\_Deathship_textual_corpus\lz5>dir
Volume in drive D is S640_Vol5
Volume Serial Number is 5861-9E6C
Directory of D:\_Deathship_textual_corpus\lz5
01/06/2016 11:14 PM <DIR> .
01/06/2016 11:14 PM <DIR> ..
09/11/2015 05:25 AM 33,258,496 Agatha_Christie_89-ebooks_TXT.tar
01/06/2016 10:40 PM 11,894,354 Agatha_Christie_89-ebooks_TXT.tar.lz5
10/14/2015 02:29 PM 10,434,674 Agatha_Christie_89-ebooks_TXT.tar.Nakamichi
09/11/2015 05:25 AM 13,713,275 Complete_Works_of_Fyodor_Dostoyevsky.txt
01/06/2016 09:42 PM 4,981,395 Complete_Works_of_Fyodor_Dostoyevsky.txt.lz5
10/12/2015 09:00 AM 4,544,039 Complete_Works_of_Fyodor_Dostoyevsky.txt.Nakamichi
09/11/2015 05:25 AM 10,192,446 dickens
01/06/2016 09:35 PM 3,936,253 dickens.lz5
10/11/2015 04:14 PM 3,722,075 dickens.Nakamichi
01/06/2016 10:26 PM 544,256 lz5-1.3.3.tar
01/06/2016 10:27 PM 112,315 lz5-1.3.3.tar.lz5
01/06/2016 11:14 PM 145,822 lz5-1.3.3.tar.Nakamichi
01/05/2016 07:47 PM 217,556 lz5.exe
01/01/2002 04:41 AM 120,832 Nakamichi_Kintaro_Intel_15.0_64bit.exe
01/01/2002 05:50 AM 3,028,450 Nakamichi_Kintaro_source_booklet_executables_32bit_64bit_GCC510_Intel150.zip
11/02/2015 10:33 PM 132,728,832 New_Shorter_Oxford_English_Dictionary_fifth_edition.tar
01/06/2016 10:20 PM 24,712,948 New_Shorter_Oxford_English_Dictionary_fifth_edition.tar.lz5
12/01/2015 08:56 AM 25,592,405 New_Shorter_Oxford_English_Dictionary_fifth_edition.tar.Nakamichi
09/11/2015 05:25 AM 12,030,464 New_York_Times_Bestsellers_-_August_2015_-_20_ebooks.tar
01/06/2016 09:47 PM 4,519,211 New_York_Times_Bestsellers_-_August_2015_-_20_ebooks.tar.lz5
10/14/2015 05:18 PM 4,328,336 New_York_Times_Bestsellers_-_August_2015_-_20_ebooks.tar.Nakamichi
09/11/2015 05:25 AM 14,613,183 The_Book_of_The_Thousand_Nights_and_a_Night.txt
01/06/2016 10:26 PM 5,502,069 The_Book_of_The_Thousand_Nights_and_a_Night.txt.lz5
10/12/2015 03:26 AM 5,228,912 The_Book_of_The_Thousand_Nights_and_a_Night.txt.Nakamichi
01/06/2016 10:59 PM 92,096 The_Little_Prince_-_Antoine_de_Saint-Exupery.epub.txt
01/06/2016 10:59 PM 37,008 The_Little_Prince_-_Antoine_de_Saint-Exupery.epub.txt.lz5
10/11/2015 02:12 PM 43,944 The_Little_Prince_-_Antoine_de_Saint-Exupery.epub.txt.Nakamichi
01/06/2016 10:56 PM 4,445,260 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt
01/06/2016 10:22 PM 1,435,859 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt.lz5
10/11/2015 11:15 AM 1,420,630 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt.Nakamichi
30 File(s) 337,577,395 bytes
2 Dir(s) 83,927,027,712 bytes free
D:\_Deathship_textual_corpus\lz5>Nakamichi_Kintaro_Intel_15.0_64bit.exe The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt.Nakamichi /bench
Nakamichi 'Kintaro', written by Kaze, based on Nobuo Ito's LZSS source, babealicious suggestion by m^2 enforced, muffinesque suggestion by Jim Dempsey enforced.
Note: This compile can handle files up to 1711MB.
Current priority class is HIGH_PRIORITY_CLASS.
Decompressing 1420630 bytes ...
RAM-to-RAM performance: 704 MB/s.
Compression Ratio (bigger-the-better): 3.13:1
D:\_Deathship_textual_corpus\lz5>Nakamichi_Kintaro_Intel_15.0_64bit.exe dickens.Nakamichi /bench
Nakamichi 'Kintaro', written by Kaze, based on Nobuo Ito's LZSS source, babealicious suggestion by m^2 enforced, muffinesque suggestion by Jim Dempsey enforced.
Note: This compile can handle files up to 1711MB.
Current priority class is HIGH_PRIORITY_CLASS.
Decompressing 3722075 bytes ...
RAM-to-RAM performance: 512 MB/s.
Compression Ratio (bigger-the-better): 2.74:1
D:\_Deathship_textual_corpus\lz5>Nakamichi_Kintaro_Intel_15.0_64bit.exe The_Book_of_The_Thousand_Nights_and_a_Night.txt.Nakamichi /bench
Nakamichi 'Kintaro', written by Kaze, based on Nobuo Ito's LZSS source, babealicious suggestion by m^2 enforced, muffinesque suggestion by Jim Dempsey enforced.
Note: This compile can handle files up to 1711MB.
Current priority class is HIGH_PRIORITY_CLASS.
Decompressing 5228912 bytes ...
RAM-to-RAM performance: 448 MB/s.
Compression Ratio (bigger-the-better): 2.79:1
D:\_Deathship_textual_corpus\lz5>Nakamichi_Kintaro_Intel_15.0_64bit.exe Agatha_Christie_89-ebooks_TXT.tar.Nakamichi /bench
Nakamichi 'Kintaro', written by Kaze, based on Nobuo Ito's LZSS source, babealicious suggestion by m^2 enforced, muffinesque suggestion by Jim Dempsey enforced.
Note: This compile can handle files up to 1711MB.
Current priority class is HIGH_PRIORITY_CLASS.
Decompressing 10434674 bytes ...
RAM-to-RAM performance: 320 MB/s.
Compression Ratio (bigger-the-better): 3.19:1
D:\_Deathship_textual_corpus\lz5>lz5.exe -h
LZ5 command line interface 64-bit v1.3.3 by Y.Collet & P.Skibinski (Jan 5 2016)
Usage :
lz5.exe [arg] [input] [output]
input : a filename
with no FILE, or when FILE is - or stdin, read standard input
Arguments :
-0 : Fast compression (default)
-1...-18 : High compression; higher number == more compression but slower
-d : decompression (default for .lz5 extension)
-z : force compression
-f : overwrite output without prompting
-h/-H : display help/long help and exit
Advanced arguments :
-V : display Version number and exit
-v : verbose mode
-q : suppress warnings; specify twice to suppress errors too
-c : force write to standard output, even if it is the console
-t : test compressed file integrity
-m : multiple input files (implies automatic output filenames)
-l : compress using Legacy format (Linux kernel compression)
-B# : Block size [4-7](default : 7)
-BD : Block dependency (improve compression ratio)
--no-frame-crc : disable stream checksum (default:enabled)
--content-size : compressed frame includes original size (default:not present)
--[no-]sparse : sparse mode (default:enabled on file, disabled on stdout)
Benchmark arguments :
-b : benchmark file(s)
-i# : iteration loops [1-9](default : 3), benchmark mode only
D:\_Deathship_textual_corpus\lz5>\lz5 -18 -b --no-frame-crc The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt
The_Project_Gute : 4445260 -> 1435836 (32.30%), 0.1 MB/s , 455.6 MB/s
D:\_Deathship_textual_corpus\lz5>\lz5 -18 -b --no-frame-crc dickens
dickens : 10192446 -> 3936226 (38.62%), 0.1 MB/s , 390.3 MB/s
D:\_Deathship_textual_corpus\lz5>\lz5 -18 -b --no-frame-crc The_Book_of_The_Thousand_Nights_and_a_Night.txt
The_Book_of_The_ : 14613183 -> 5502038 (37.65%), 0.1 MB/s , 400.1 MB/s
D:\_Deathship_textual_corpus\lz5>\lz5 -18 -b --no-frame-crc Agatha_Christie_89-ebooks_TXT.tar
Agatha_Christie_ : 33258496 -> 11894307 (35.76%), 0.1 MB/s , 410.8 MB/s
D:\_Deathship_textual_corpus\lz5>
Note: The results were obtained on my laptop with Core 2 Q9550s, Win7 64bit.
As a final note, I wonder why Hamid didn't implement LzTurbo 19 with bigger window as well?! I fully expect his state-of-the-art parser to kick our asses right away, hee-hee!
Hi inikep,
some textual decompression feedback, again.
@github_staff
Shouldn't it be better to add 'Feedback' section, thus feedbacks won't go to 'Issues', inappropriate it is.
Anyway, seeing how v2.0 is way better than the one I played with before, wanted to see my new toy (nothing special) Nakamichi 'Zato' against LZSSE2, LZ5, and the rest superfast decompressors.
TurboBench from Sep 4 2016 was used.
Dind't have enough machine power and time to test more files, yet, the 19 ones serve well the overview.
The benchmark is downloadable at:
[https://1drv.ms/u/s!AmWWFXGMzDmEgmEpIiqenL43pg4N]
On a LZSSE2, two notes:
The benchmark package contains:
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>dir/on
10/02/2016 08:55 PM 9,025 _BENCH.BAT
10/02/2016 08:55 PM 10,687,517 Agatha_Christie_85-ebooks_(French)_TXT.tar.L17.LZSSE2
10/02/2016 08:55 PM 9,803,106 Agatha_Christie_85-ebooks_(French)_TXT.tar.Nakamichi
10/02/2016 08:55 PM 11,649,537 Agatha_Christie_89-ebooks_TXT.tar.L17.LZSSE2
10/02/2016 08:55 PM 11,441,644 Agatha_Christie_89-ebooks_TXT.tar.Nakamichi
10/02/2016 08:55 PM 5,524,854 Arabian_Nights_complete.html.L17.LZSSE2
10/02/2016 08:55 PM 5,660,839 Arabian_Nights_complete.html.Nakamichi
10/02/2016 08:55 PM 1,566,277 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.L17.LZSSE2
10/02/2016 08:55 PM 1,552,936 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.Nakamichi
10/02/2016 08:55 PM 4,840,938 Complete_Works_of_Fyodor_Dostoyevsky.txt.L17.LZSSE2
10/02/2016 08:55 PM 4,859,306 Complete_Works_of_Fyodor_Dostoyevsky.txt.Nakamichi
10/02/2016 08:55 PM 10,359,314 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.L17.LZSSE2
10/02/2016 08:55 PM 9,730,008 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.Nakamichi
10/02/2016 08:55 PM 668,060 Dictionary_of_Native_American_Mythology.tar.L17.LZSSE2
10/02/2016 08:55 PM 717,938 Dictionary_of_Native_American_Mythology.tar.Nakamichi
10/02/2016 08:55 PM 1,830,701 Duden_Synonymworterbuch_(De-De).dsl.L17.LZSSE2
10/02/2016 08:55 PM 1,610,899 Duden_Synonymworterbuch_(De-De).dsl.Nakamichi
10/02/2016 08:55 PM 4,573,102 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.L17.LZSSE2
10/02/2016 08:55 PM 4,564,126 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.Nakamichi
10/02/2016 08:55 PM 38,063,182 enwik8.L17.LZSSE2
10/02/2016 08:55 PM 38,261,116 enwik8.Nakamichi
10/02/2016 08:55 PM 1,768,505 Large_traffic_log_file_of_a_popular_website_fp.log.L17.LZSSE2
10/02/2016 08:55 PM 1,990,428 Large_traffic_log_file_of_a_popular_website_fp.log.Nakamichi
10/02/2016 08:55 PM 451,490 lz5-2.0_beta.exe
10/02/2016 08:55 PM 433,552 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.L17.LZSSE2
10/02/2016 08:55 PM 468,913 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.Nakamichi
10/02/2016 08:55 PM 1,632 MokujIN 224 prompt.lnk
10/02/2016 08:55 PM 17,363,811 Nakamichi_(Zato)_256MB-Sliding-Window_vs_LZSSE2.zip
10/02/2016 08:55 PM 146,944 Nakamichi_Zato_XMM_PREFETCH_4096_Intel_15.0_64bit_AVX.exe
10/02/2016 08:55 PM 154,624 Nakamichi_Zato_XMM_PREFETCH_4096_Intel_15.0_64bit_SSE41.exe
10/02/2016 08:55 PM 3,496,527 Slownik_jezyka_polskiego_(Pol-Pol).dsl.L17.LZSSE2
10/02/2016 08:55 PM 3,101,206 Slownik_jezyka_polskiego_(Pol-Pol).dsl.Nakamichi
10/02/2016 08:55 PM 505,572 Smyslov_(Vasily_Smyslov_2627_games).pgn.L17.LZSSE2
10/02/2016 08:55 PM 541,608 Smyslov_(Vasily_Smyslov_2627_games).pgn.Nakamichi
10/02/2016 08:55 PM 5,160,110 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.L17.LZSSE2
10/02/2016 08:55 PM 5,031,882 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.Nakamichi
10/02/2016 08:55 PM 2,556,494 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.L17.LZSSE2
10/02/2016 08:55 PM 2,540,805 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.Nakamichi
10/02/2016 08:55 PM 348 turbobench_singlefile_noNakamichi_2G.bat
10/02/2016 08:55 PM 9,129,173 turbobenchs.exe
10/02/2016 08:55 PM 1,142,536 University_of_Canterbury_The_Calgary_Corpus.tar.L17.LZSSE2
10/02/2016 08:55 PM 1,311,813 University_of_Canterbury_The_Calgary_Corpus.tar.Nakamichi
10/02/2016 08:55 PM 2,152,105 Word-list_584879_words_Russian_Spell-Check.txt.L17.LZSSE2
10/02/2016 08:55 PM 2,004,811 Word-list_584879_words_Russian_Spell-Check.txt.Nakamichi
10/02/2016 08:55 PM 1,297,189 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.L17.LZSSE2
10/02/2016 08:55 PM 1,456,466 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.Nakamichi
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>
And the actual console results on i5-2430M @3.00GHz, RAM @666MHz:
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Large_traffic_log_file_of_a_popular_website_fp.log -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 19:12:09 2016
C Size ratio% C MB/s D MB/s Name File
411108 2.0 0.36 0.36 zpaq 5 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
709242 3.4 1.00 403.91 lzma 9 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
723419 3.5 5.45 49.40 bzip2 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
742667 3.6 0.24 793.82 brotli 11 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
745809 3.6 0.76 2184.24 lzturbo 39 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
801029 3.9 1.21 1750.18 zstd 22 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
872740 4.2 1.39 735.04 lzham 4 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
970589 4.7 6.37 1603.19 xpack 9 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1010821 4.9 48.77 1718.66 zstd 12 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1064459 5.2 0.87 3725.53 lzturbo 29 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1084431 5.3 1.23 2447.71 lz5 15 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1085963 5.3 7.57 121.05 zpaq 2 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1143618 5.5 0.94 1043.22 crush 2 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1178879 3420.8 lz5-2.0_beta 18 ! Outside TurboBench !
1333117 6.5 41.81 637.39 zlib 9 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1408548 6.8 18.63 1386.39 libdeflate 9 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1554934 7.5 1.12 5195.83 lzturbo 19 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1561495 4302.4 lz5-2.0_beta 9 ! Outside TurboBench !
1567022 7.6 34.31 3805.99 lz4 16 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1724307 8.4 257.46 603.83 brieflz Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1768505 4416 LZSSE2 17 SSE4.1! Outside TurboBench !
1768505 4672 LZSSE2 17 AVX ! Outside TurboBench !
1768509 8.6 2.96 4414.79 lzsse2 17 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1939991 9.4 445.99 339.39 density 3 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
1990428 2560 Nakamichi-Zato ! Outside TurboBench !
2041871 9.9 11.78 3969.40 lzsse2 1 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
2458195 11.9 513.22 935.48 fastlz 2 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
2754356 13.4 971.13 1760.19 snappy_c Large_traffic_log_file_of_a_popular_website_fp.log.tbb
2806326 13.6 528.68 944.44 lzf Large_traffic_log_file_of_a_popular_website_fp.log.tbb
3016983 14.6 143.55 3652.92 yappy Large_traffic_log_file_of_a_popular_website_fp.log.tbb
11053474 53.6 1203.35 2929.39 chameleon 2 Large_traffic_log_file_of_a_popular_website_fp.log.tbb
20511136 99.5 143.57 1228.89 trle Large_traffic_log_file_of_a_popular_website_fp.log.tbb
20617075 100.0 6159.87 5509.64 memcpy Large_traffic_log_file_of_a_popular_website_fp.log.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Word-list_584879_words_Russian_Spell-Check.txt -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 19:17:03 2016
C Size ratio% C MB/s D MB/s Name File
395907 5.7 0.27 0.27 zpaq 5 Word-list_584879_words_Russian_Spell-Check.txt.tbb
961241 13.7 1.36 81.69 lzma 9 Word-list_584879_words_Russian_Spell-Check.txt.tbb
972129 13.9 0.30 282.44 brotli 11 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1038938 14.8 2.50 552.26 lzturbo 39 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1053174 15.0 0.93 462.60 zstd 22 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1068797 15.3 1.09 133.01 lzham 4 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1134768 16.2 12.76 498.47 libdeflate 9 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1331729 19.0 15.38 501.86 zstd 12 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1384453 19.8 2.75 492.78 xpack 9 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1410045 20.1 1.64 283.05 zlib 9 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1647224 23.5 13.32 34.23 bzip2 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1818611 26.0 0.12 254.46 crush 2 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1884914 26.9 5.18 64.44 zpaq 2 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1958090 28.0 139.17 211.29 brieflz Word-list_584879_words_Russian_Spell-Check.txt.tbb
1958925 28.0 2.67 951.28 lzturbo 29 Word-list_584879_words_Russian_Spell-Check.txt.tbb
1959563 28.0 0.83 828.85 lz5 15 Word-list_584879_words_Russian_Spell-Check.txt.tbb
2004811 832 Nakamichi-Zato ! Outside TurboBench !
2073997 1399.8 lz5-2.0_beta 18 ! Outside TurboBench !
2109594 30.1 70.40 1957.07 yappy Word-list_584879_words_Russian_Spell-Check.txt.tbb
2152105 1664 LZSSE2 17 SSE4.1! Outside TurboBench !
2152105 1664 LZSSE2 17 AVX ! Outside TurboBench !
2152109 30.7 8.05 1716.22 lzsse2 17 Word-list_584879_words_Russian_Spell-Check.txt.tbb
2236063 31.9 16.07 1530.82 lzsse2 1 Word-list_584879_words_Russian_Spell-Check.txt.tbb
2339944 33.4 3.23 1229.66 lzturbo 19 Word-list_584879_words_Russian_Spell-Check.txt.tbb
2346546 1547.4 lz5-2.0_beta 9 ! Outside TurboBench !
2363922 33.8 6.91 1595.00 lz4 16 Word-list_584879_words_Russian_Spell-Check.txt.tbb
2422149 34.6 281.36 480.54 lzf Word-list_584879_words_Russian_Spell-Check.txt.tbb
2445093 34.9 287.86 387.69 fastlz 2 Word-list_584879_words_Russian_Spell-Check.txt.tbb
2449587 35.0 373.98 915.45 snappy_c Word-list_584879_words_Russian_Spell-Check.txt.tbb
2536517 36.2 277.47 248.53 density 3 Word-list_584879_words_Russian_Spell-Check.txt.tbb
3888621 55.5 1169.47 2460.62 chameleon 2 Word-list_584879_words_Russian_Spell-Check.txt.tbb
7000457 100.0 137.35 5672.98 trle Word-list_584879_words_Russian_Spell-Check.txt.tbb
7000457 100.0 6346.74 5595.89 memcpy Word-list_584879_words_Russian_Spell-Check.txt.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Agatha_Christie_85-ebooks_(French)_TXT.tar -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 14:13:46 2016
C Size ratio% C MB/s D MB/s Name File
5639052 17.6 0.30 0.30 zpaq 5 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
6739693 21.1 0.95 81.88 lzma 9 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
6758120 21.1 0.79 166.72 lzham 4 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
6797247 21.2 0.92 389.67 lzturbo 39 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
6802328 21.3 1.10 301.01 zstd 22 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
6910154 21.6 0.37 245.03 brotli 11 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
7680438 24.0 9.49 20.61 bzip2 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
8367282 26.1 0.61 355.33 xpack 9 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
8897625 27.8 3.28 58.95 zpaq 2 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
8965799 28.0 4.68 499.64 zstd 12 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
8990951 28.1 0.93 610.22 lzturbo 29 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
9147309 28.6 0.12 263.86 crush 2 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
9332360 769.3 lz5-2.0_beta 18 ! Outside TurboBench !
9803106 960 Nakamichi-Zato ! Outside TurboBench !
10107487 31.6 1.30 396.87 lz5 15 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
10460666 32.7 6.97 454.65 libdeflate 9 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
10687517 2176 LZSSE2 17 SSE4.1! Outside TurboBench !
10687517 2304 LZSSE2 17 AVX ! Outside TurboBench !
10687521 33.4 2.07 2267.44 lzsse2 17 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
10973383 34.3 10.73 232.96 zlib 9 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
12107435 37.8 1.01 2658.62 lzturbo 19 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
12132016 1840.8 lz5-2.0_beta 9 ! Outside TurboBench !
12266943 38.3 16.35 1897.96 lz4 16 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
13238993 41.4 83.58 154.50 brieflz Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
14364428 44.9 283.27 251.05 density 3 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
15448533 48.3 12.31 1691.80 lzsse2 1 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
17262978 53.9 61.89 1265.41 yappy Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
17678955 55.2 178.84 361.43 lzf Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
17868118 55.8 182.69 313.48 fastlz 2 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
17915390 56.0 1049.93 2004.08 chameleon 2 Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
18694206 58.4 222.62 644.77 snappy_c Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
31952980 99.8 148.95 1323.10 trle Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
32007172 100.0 6225.86 5379.36 memcpy Agatha_Christie_85-ebooks_(French)_TXT.tar.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Agatha_Christie_89-ebooks_TXT.tar -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 14:31:39 2016
C Size ratio% C MB/s D MB/s Name File
6267713 18.8 0.31 0.31 zpaq 5 Agatha_Christie_89-ebooks_TXT.tar.tbb
7881393 23.7 0.94 72.93 lzma 9 Agatha_Christie_89-ebooks_TXT.tar.tbb
7917303 23.8 0.79 147.72 lzham 4 Agatha_Christie_89-ebooks_TXT.tar.tbb
7963999 23.9 1.04 260.59 zstd 22 Agatha_Christie_89-ebooks_TXT.tar.tbb
7973327 24.0 0.89 348.96 lzturbo 39 Agatha_Christie_89-ebooks_TXT.tar.tbb
8014306 24.1 0.35 218.91 brotli 11 Agatha_Christie_89-ebooks_TXT.tar.tbb
8416486 25.3 10.01 21.03 bzip2 Agatha_Christie_89-ebooks_TXT.tar.tbb
9595005 28.8 0.55 319.11 xpack 9 Agatha_Christie_89-ebooks_TXT.tar.tbb
9851022 29.6 4.71 471.32 zstd 12 Agatha_Christie_89-ebooks_TXT.tar.tbb
9926145 29.8 3.03 56.95 zpaq 2 Agatha_Christie_89-ebooks_TXT.tar.tbb
10076254 30.3 0.14 247.08 crush 2 Agatha_Christie_89-ebooks_TXT.tar.tbb
10334621 31.1 0.89 567.62 lzturbo 29 Agatha_Christie_89-ebooks_TXT.tar.tbb
11133495 774.0 lz5-2.0_beta 18 ! Outside TurboBench !
11145205 33.5 1.33 386.04 lz5 15 Agatha_Christie_89-ebooks_TXT.tar.tbb
11175291 33.6 7.26 437.57 libdeflate 9 Agatha_Christie_89-ebooks_TXT.tar.tbb
11441644 960 Nakamichi-Zato ! Outside TurboBench !
11649537 2112 LZSSE2 17 SSE4.1! Outside TurboBench !
11649537 2240 LZSSE2 17 AVX ! Outside TurboBench !
11649541 35.0 4.65 2172.48 lzsse2 17 Agatha_Christie_89-ebooks_TXT.tar.tbb
11716396 35.2 10.54 226.92 zlib 9 Agatha_Christie_89-ebooks_TXT.tar.tbb
13165738 39.6 1.04 2570.80 lzturbo 19 Agatha_Christie_89-ebooks_TXT.tar.tbb
13194355 1813.8 lz5-2.0_beta 9 ! Outside TurboBench !
13327732 40.1 16.41 1870.35 lz4 16 Agatha_Christie_89-ebooks_TXT.tar.tbb
14305207 43.0 77.90 148.64 brieflz Agatha_Christie_89-ebooks_TXT.tar.tbb
15146404 45.5 277.03 245.52 density 3 Agatha_Christie_89-ebooks_TXT.tar.tbb
15890698 47.8 12.23 1702.86 lzsse2 1 Agatha_Christie_89-ebooks_TXT.tar.tbb
18054497 54.3 61.85 1292.14 yappy Agatha_Christie_89-ebooks_TXT.tar.tbb
18191878 54.7 176.19 357.75 lzf Agatha_Christie_89-ebooks_TXT.tar.tbb
18478571 55.6 180.34 307.93 fastlz 2 Agatha_Christie_89-ebooks_TXT.tar.tbb
18773896 56.4 1103.36 1976.50 chameleon 2 Agatha_Christie_89-ebooks_TXT.tar.tbb
19377968 58.3 245.84 647.92 snappy_c Agatha_Christie_89-ebooks_TXT.tar.tbb
33156061 99.7 149.02 1321.56 trle Agatha_Christie_89-ebooks_TXT.tar.tbb
33258500 100.0 6312.11 5575.61 memcpy Agatha_Christie_89-ebooks_TXT.tar.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Arabian_Nights_complete.html -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 14:40:27 2016
C Size ratio% C MB/s D MB/s Name File
2975964 19.1 0.30 0.30 zpaq 5 Arabian_Nights_complete.html.tbb
3892786 25.0 1.05 68.74 lzma 9 Arabian_Nights_complete.html.tbb
3913051 25.1 0.34 212.05 brotli 11 Arabian_Nights_complete.html.tbb
3961082 25.4 0.89 145.14 lzham 4 Arabian_Nights_complete.html.tbb
3988669 25.6 1.05 299.58 zstd 22 Arabian_Nights_complete.html.tbb
3992884 25.6 1.04 385.67 lzturbo 39 Arabian_Nights_complete.html.tbb
4038902 25.9 10.07 21.41 bzip2 Arabian_Nights_complete.html.tbb
4589061 29.4 0.64 324.11 xpack 9 Arabian_Nights_complete.html.tbb
4620802 29.7 5.18 464.37 zstd 12 Arabian_Nights_complete.html.tbb
4724482 30.3 3.02 56.53 zpaq 2 Arabian_Nights_complete.html.tbb
4757444 30.5 0.16 243.38 crush 2 Arabian_Nights_complete.html.tbb
5035317 32.3 1.05 635.15 lzturbo 29 Arabian_Nights_complete.html.tbb
5223100 33.5 1.03 409.66 lz5 15 Arabian_Nights_complete.html.tbb
5259086 33.7 7.66 416.98 libdeflate 9 Arabian_Nights_complete.html.tbb
5478319 35.2 9.98 223.95 zlib 9 Arabian_Nights_complete.html.tbb
5509438 928.2 lz5-2.0_beta 18 ! Outside TurboBench !
5524854 2048 LZSSE2 17 SSE4.1! Outside TurboBench !
5524854 2240 LZSSE2 17 AVX ! Outside TurboBench !
5524858 35.5 5.26 2108.15 lzsse2 17 Arabian_Nights_complete.html.tbb
5660839 960 Nakamichi-Zato ! Outside TurboBench !
6183326 39.7 1.23 2527.32 lzturbo 19 Arabian_Nights_complete.html.tbb
6185910 1814.8 lz5-2.0_beta 9 ! Outside TurboBench !
6258303 40.2 15.74 1815.83 lz4 16 Arabian_Nights_complete.html.tbb
6740391 43.3 79.89 150.83 brieflz Arabian_Nights_complete.html.tbb
7101428 45.6 250.89 227.77 density 3 Arabian_Nights_complete.html.tbb
7294087 46.8 12.03 1700.32 lzsse2 1 Arabian_Nights_complete.html.tbb
8585442 55.1 168.81 356.18 lzf Arabian_Nights_complete.html.tbb
8613199 55.3 179.31 313.49 fastlz 2 Arabian_Nights_complete.html.tbb
8630338 55.4 62.13 1302.20 yappy Arabian_Nights_complete.html.tbb
9027159 57.9 247.54 632.03 snappy_c Arabian_Nights_complete.html.tbb
9049558 58.1 1093.34 1864.05 chameleon 2 Arabian_Nights_complete.html.tbb
15580578 100.0 144.65 1319.96 trle Arabian_Nights_complete.html.tbb
15583444 100.0 6495.81 5477.48 memcpy Arabian_Nights_complete.html.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Blacklegs_Card_Sharps_and_Confidence_Men.pdf -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 14:42:24 2016
C Size ratio% C MB/s D MB/s Name File
1433545 91.5 0.22 0.22 zpaq 5 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1437059 91.7 0.29 107.33 brotli 11 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1443674 92.2 3.83 2782.02 lzturbo 39 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1445273 92.3 4.97 92.01 lzham 4 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1447307 92.4 5.42 3361.11 zstd 22 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1448653 92.5 1.91 409.16 xpack 9 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1449223 92.5 4.37 6166.44 lzturbo 29 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1450837 92.6 34.31 4437.05 zstd 12 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1452139 92.7 23.39 286.44 libdeflate 9 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1452300 92.7 4.95 37.43 zpaq 2 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1454011 92.8 2.84 3975.32 lz5 15 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1454076 92.8 3.64 17.07 lzma 9 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1454251 7285.0 lz5-2.0_beta 18 ! Outside TurboBench !
1455523 92.9 4.85 7530.18 lzturbo 19 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1456105 93.0 25.75 234.33 zlib 9 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1458737 7950.6 lz5-2.0_beta 9 ! Outside TurboBench !
1458993 93.2 6.00 11.92 bzip2 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1459844 93.2 34.26 6190.82 lz4 16 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1474902 94.2 1857.98 4819.31 snappy_c Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1513157 96.6 333.89 317.19 density 3 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1513539 96.6 256.01 1596.61 fastlz 2 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1515421 96.8 165.74 1687.80 lzf Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1528008 97.6 58.24 5844.32 yappy Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1552936 2944 Nakamichi-Zato ! Outside TurboBench !
1559959 99.6 988.81 1220.79 chameleon 2 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1566281 100.0 165.01 7677.83 trle Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1566277 95552 LZSSE2 17 SSE4.1! Outside TurboBench !
1566277 95552 LZSSE2 17 AVX ! Outside TurboBench !
1566281 100.0 13.58 7677.83 lzsse2 17 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1566281 100.0 16.82 7640.38 lzsse2 1 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1566281 100.0 7458.46 4789.84 memcpy Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1617666 103.3 9.15 181.96 crush 2 Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
1634540 104.4 99.78 634.89 brieflz Blacklegs_Card_Sharps_and_Confidence_Men.pdf.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Complete_Works_of_Fyodor_Dostoyevsky.txt -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 14:50:16 2016
C Size ratio% C MB/s D MB/s Name File
2650827 19.3 0.31 0.31 zpaq 5 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
3470856 25.3 10.08 20.71 bzip2 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
3474265 25.3 1.08 70.01 lzma 9 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
3488656 25.4 0.36 227.95 brotli 11 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
3490560 25.5 0.92 147.68 lzham 4 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
3502860 25.5 1.25 314.05 zstd 22 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
3512645 25.6 1.03 396.60 lzturbo 39 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
4097920 29.9 4.59 469.52 zstd 12 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
4104258 29.9 0.62 342.84 xpack 9 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
4203354 30.7 0.13 246.45 crush 2 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
4211285 30.7 3.03 56.49 zpaq 2 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
4555584 33.2 1.07 615.66 lzturbo 29 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
4618517 33.7 7.25 437.03 libdeflate 9 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
4697463 34.3 1.38 420.74 lz5 15 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
4840938 2112 LZSSE2 17 SSE4.1! Outside TurboBench !
4840938 2176 LZSSE2 17 AVX ! Outside TurboBench !
4840942 35.3 4.71 2161.27 lzsse2 17 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
4852707 35.4 10.44 227.07 zlib 9 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
4859306 960 Nakamichi-Zato ! Outside TurboBench !
4885062 867.2 lz5-2.0_beta 18 ! Outside TurboBench !
5488915 40.0 1.26 2532.93 lzturbo 19 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
5491134 1805 lz5-2.0_beta 9 ! Outside TurboBench !
5554582 40.5 15.92 1845.41 lz4 16 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
5944346 43.3 74.60 147.71 brieflz Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
6378991 46.5 273.52 243.70 density 3 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
6664446 48.6 12.22 1679.31 lzsse2 1 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
7434625 54.2 62.38 1279.70 yappy Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
7557433 55.1 168.58 358.05 lzf Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
7662064 55.9 176.15 307.64 fastlz 2 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
7741282 56.5 1025.52 1966.62 chameleon 2 Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
8028636 58.5 246.64 629.16 snappy_c Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
13704843 99.9 144.97 1312.28 trle Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
13713279 100.0 6624.77 6086.67 memcpy Complete_Works_of_Fyodor_Dostoyevsky.txt.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 15:16:32 2016
C Size ratio% C MB/s D MB/s Name File
5881208 14.6 0.34 0.34 zpaq 5 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
6942121 17.2 0.92 96.16 lzma 9 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
6996681 17.4 0.77 198.87 lzham 4 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
7028270 17.4 0.80 437.34 lzturbo 39 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
7055102 17.5 1.08 355.74 zstd 22 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
7110795 17.6 0.33 297.52 brotli 11 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
7125187 17.7 9.43 23.06 bzip2 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
8984648 22.3 3.62 67.43 zpaq 2 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
9221434 22.9 1.14 510.86 xpack 9 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
9253662 874.7 lz5-2.0_beta 18 ! Outside TurboBench !
9263242 23.0 0.06 318.59 crush 2 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
9298174 23.1 7.08 588.24 zstd 12 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
9491604 23.6 0.80 710.56 lzturbo 29 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
9730008 896 Nakamichi-Zato ! Outside TurboBench !
10048812 24.9 1.35 555.35 lz5 15 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
10125158 25.1 5.37 605.01 libdeflate 9 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
10359314 2752 LZSSE2 17 SSE4.1! Outside TurboBench !
10359314 2880 LZSSE2 17 AVX ! Outside TurboBench !
10359318 25.7 4.28 2803.69 lzsse2 17 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
10771358 26.7 4.40 275.13 zlib 9 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
11806303 29.3 0.95 2915.87 lzturbo 19 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
11817690 1983.0 lz5-2.0_beta 9 ! Outside TurboBench !
12065538 29.9 8.37 1983.71 lz4 16 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
15196140 37.7 95.09 159.80 brieflz Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
15278814 37.9 57.32 1675.18 yappy Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
17802562 44.2 341.20 281.89 density 3 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
18298845 45.4 12.66 1793.96 lzsse2 1 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
19974577 49.6 280.56 718.80 snappy_c Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
20525594 50.9 201.42 387.03 lzf Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
20596670 51.1 209.58 340.90 fastlz 2 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
21693398 53.8 1168.54 2562.83 chameleon 2 Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
40278350 99.9 150.33 1323.24 trle Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
40303108 100.0 6211.95 5505.14 memcpy Complete_works_of_Fyodor_Dostoyevsky_in_15_volumes_(Russian).tar.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Dictionary_of_Native_American_Mythology.tar -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 15:19:11 2016
C Size ratio% C MB/s D MB/s Name File
304124 10.4 0.33 0.33 zpaq 5 Dictionary_of_Native_American_Mythology.tar.tbb
419209 14.3 8.96 34.37 bzip2 Dictionary_of_Native_American_Mythology.tar.tbb
422997 14.4 0.46 399.98 brotli 11 Dictionary_of_Native_American_Mythology.tar.tbb
427665 14.6 1.34 111.56 lzma 9 Dictionary_of_Native_American_Mythology.tar.tbb
447293 15.3 1.60 828.00 zstd 22 Dictionary_of_Native_American_Mythology.tar.tbb
448088 15.3 1.60 1176.63 lzturbo 39 Dictionary_of_Native_American_Mythology.tar.tbb
451937 15.4 1.41 243.61 lzham 4 Dictionary_of_Native_American_Mythology.tar.tbb
489933 16.7 14.65 821.73 zstd 12 Dictionary_of_Native_American_Mythology.tar.tbb
492053 16.8 3.63 846.92 xpack 9 Dictionary_of_Native_American_Mythology.tar.tbb
513025 17.5 0.85 404.51 crush 2 Dictionary_of_Native_American_Mythology.tar.tbb
533603 18.2 6.45 82.20 zpaq 2 Dictionary_of_Native_American_Mythology.tar.tbb
563084 19.2 0.62 1046.32 lz5 15 Dictionary_of_Native_American_Mythology.tar.tbb
563974 19.3 1.81 1571.16 lzturbo 29 Dictionary_of_Native_American_Mythology.tar.tbb
617741 21.1 12.25 607.35 libdeflate 9 Dictionary_of_Native_American_Mythology.tar.tbb
617824 2122.2 lz5-2.0_beta 18 ! Outside TurboBench !
629521 21.5 23.63 319.20 zlib 9 Dictionary_of_Native_American_Mythology.tar.tbb
668060 2816 LZSSE2 17 SSE4.1! Outside TurboBench !
668060 3712 LZSSE2 17 AVX ! Outside TurboBench !
668064 22.8 1.82 2557.76 lzsse2 17 Dictionary_of_Native_American_Mythology.tar.tbb
705547 2655.2 lz5-2.0_beta 9 ! Outside TurboBench !
706652 24.1 2.12 3358.53 lzturbo 19 Dictionary_of_Native_American_Mythology.tar.tbb
709584 24.2 22.47 2418.36 lz4 16 Dictionary_of_Native_American_Mythology.tar.tbb
717938 1600 Nakamichi-Zato ! Outside TurboBench !
738627 25.2 111.96 246.60 brieflz Dictionary_of_Native_American_Mythology.tar.tbb
799115 27.3 13.67 2412.39 lzsse2 1 Dictionary_of_Native_American_Mythology.tar.tbb
885362 30.2 224.09 221.16 density 3 Dictionary_of_Native_American_Mythology.tar.tbb
977236 33.4 257.58 476.82 fastlz 2 Dictionary_of_Native_American_Mythology.tar.tbb
1055618 36.0 245.24 498.49 lzf Dictionary_of_Native_American_Mythology.tar.tbb
1064801 36.4 390.07 959.58 snappy_c Dictionary_of_Native_American_Mythology.tar.tbb
1364314 46.6 77.02 1847.72 yappy Dictionary_of_Native_American_Mythology.tar.tbb
1695276 57.9 1098.51 1914.14 chameleon 2 Dictionary_of_Native_American_Mythology.tar.tbb
2563763 87.5 167.14 1430.70 trle Dictionary_of_Native_American_Mythology.tar.tbb
2928644 100.0 7143.02 7666.60 memcpy Dictionary_of_Native_American_Mythology.tar.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Duden_Synonymworterbuch_(De-De).dsl -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 15:25:19 2016
C Size ratio% C MB/s D MB/s Name File
811958 7.2 0.37 0.38 zpaq 5 Duden_Synonymworterbuch_(De-De).dsl.tbb
1004261 8.9 1.18 166.17 lzma 9 Duden_Synonymworterbuch_(De-De).dsl.tbb
1068742 9.4 1.19 919.77 lzturbo 39 Duden_Synonymworterbuch_(De-De).dsl.tbb
1070058 9.4 1.00 353.88 lzham 4 Duden_Synonymworterbuch_(De-De).dsl.tbb
1081825 9.5 0.41 456.20 brotli 11 Duden_Synonymworterbuch_(De-De).dsl.tbb
1088298 9.6 1.25 716.11 zstd 22 Duden_Synonymworterbuch_(De-De).dsl.tbb
1147399 10.1 7.13 33.48 bzip2 Duden_Synonymworterbuch_(De-De).dsl.tbb
1335007 11.8 4.51 93.77 zpaq 2 Duden_Synonymworterbuch_(De-De).dsl.tbb
1420344 12.5 1.25 1430.67 lzturbo 29 Duden_Synonymworterbuch_(De-De).dsl.tbb
1424372 12.6 3.16 733.06 xpack 9 Duden_Synonymworterbuch_(De-De).dsl.tbb
1466311 12.9 15.76 823.66 zstd 12 Duden_Synonymworterbuch_(De-De).dsl.tbb
1485383 13.1 0.13 482.12 crush 2 Duden_Synonymworterbuch_(De-De).dsl.tbb
1489744 13.1 1.00 1011.44 lz5 15 Duden_Synonymworterbuch_(De-De).dsl.tbb
1493790 1467.7 lz5-2.0_beta 18 ! Outside TurboBench !
1610899 1472 Nakamichi-Zato ! Outside TurboBench !
1740676 15.4 8.44 822.53 libdeflate 9 Duden_Synonymworterbuch_(De-De).dsl.tbb
1744697 15.4 4.34 393.79 zlib 9 Duden_Synonymworterbuch_(De-De).dsl.tbb
1830701 3648 LZSSE2 17 SSE4.1! Outside TurboBench !
1830701 3648 LZSSE2 17 AVX ! Outside TurboBench !
1830705 16.2 3.28 3288.43 lzsse2 17 Duden_Synonymworterbuch_(De-De).dsl.tbb
2037204 18.0 1.59 2809.23 lzturbo 19 Duden_Synonymworterbuch_(De-De).dsl.tbb
2042405 2175.3 lz5-2.0_beta 9 ! Outside TurboBench !
2090628 18.4 11.50 1909.57 lz4 16 Duden_Synonymworterbuch_(De-De).dsl.tbb
2479859 21.9 92.42 2067.72 yappy Duden_Synonymworterbuch_(De-De).dsl.tbb
2588673 22.8 10.74 2962.68 lzsse2 1 Duden_Synonymworterbuch_(De-De).dsl.tbb
2605920 23.0 156.21 246.93 brieflz Duden_Synonymworterbuch_(De-De).dsl.tbb
3015462 26.6 317.64 271.72 density 3 Duden_Synonymworterbuch_(De-De).dsl.tbb
3040630 26.8 371.51 620.77 lzf Duden_Synonymworterbuch_(De-De).dsl.tbb
3049421 26.9 355.83 532.30 fastlz 2 Duden_Synonymworterbuch_(De-De).dsl.tbb
3233776 28.5 441.40 993.10 snappy_c Duden_Synonymworterbuch_(De-De).dsl.tbb
6029314 53.2 1216.62 3272.30 chameleon 2 Duden_Synonymworterbuch_(De-De).dsl.tbb
11335236 100.0 139.52 5470.67 trle Duden_Synonymworterbuch_(De-De).dsl.tbb
11335236 100.0 6339.62 5452.25 memcpy Duden_Synonymworterbuch_(De-De).dsl.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 15:31:51 2016
C Size ratio% C MB/s D MB/s Name File
2375112 18.4 0.30 0.30 zpaq 5 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
3087799 23.9 0.36 234.26 brotli 11 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
3102796 24.0 1.23 70.16 lzma 9 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
3149792 24.4 1.00 152.52 lzham 4 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
3163361 24.5 1.37 345.08 zstd 22 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
3176170 24.6 1.07 437.88 lzturbo 39 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
3423837 26.5 10.31 21.69 bzip2 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
3610798 27.9 0.74 353.93 xpack 9 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
3702916 28.6 6.20 479.15 zstd 12 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
3726436 28.8 3.32 59.29 zpaq 2 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
3795056 29.3 0.25 244.54 crush 2 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
3999478 30.9 1.12 677.81 lzturbo 29 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
4188499 32.4 1.51 436.06 lz5 15 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
4327921 946.6 lz5-2.0_beta 18 ! Outside TurboBench !
4354767 33.7 8.04 417.97 libdeflate 9 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
4521811 35.0 14.10 222.49 zlib 9 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
4564126 896 Nakamichi-Zato ! Outside TurboBench !
4573102 2048 LZSSE2 17 SSE4.1! Outside TurboBench !
4573102 2240 LZSSE2 17 AVX ! Outside TurboBench !
4573106 35.4 5.89 2139.98 lzsse2 17 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
5095655 39.4 1.32 2609.78 lzturbo 19 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
5097610 1864.2 lz5-2.0_beta 9 ! Outside TurboBench !
5138931 39.7 21.52 1869.08 lz4 16 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
5399037 41.7 80.69 153.48 brieflz Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
5762189 44.6 11.93 1790.43 lzsse2 1 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
5929465 45.8 230.92 211.71 density 3 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
6858738 53.0 64.45 1444.34 yappy Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
6989183 54.0 168.78 357.56 lzf Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
7078866 54.7 172.08 309.69 fastlz 2 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
7285714 56.3 249.62 673.72 snappy_c Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
7713222 59.6 1018.75 1764.06 chameleon 2 Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
12928445 100.0 148.79 1311.90 trle Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
12934059 100.0 6387.19 5613.74 memcpy Encyclopedia_of_Pain_(3_Vol_Set).pdf.txt.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe enwik8 -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 16:16:52 2016
C Size ratio% C MB/s D MB/s Name File
20596996 20.6 0.30 0.29 zpaq 5 enwik8.tbb
24763998 24.8 0.85 64.99 lzma 9 enwik8.tbb
25021508 25.0 0.71 127.88 lzham 4 enwik8.tbb
25390181 25.4 0.95 220.17 zstd 22 enwik8.tbb
25403540 25.4 0.79 301.23 lzturbo 39 enwik8.tbb
25743413 25.7 0.31 195.25 brotli 11 enwik8.tbb
29008762 29.0 10.20 21.06 bzip2 enwik8.tbb
30520879 30.5 0.44 245.40 xpack 9 enwik8.tbb
31056443 31.1 5.94 427.89 zstd 12 enwik8.tbb
31373273 31.4 0.81 549.03 lzturbo 29 enwik8.tbb
31699553 31.7 0.26 222.10 crush 2 enwik8.tbb
32073935 32.1 2.89 54.35 zpaq 2 enwik8.tbb
34427078 34.4 1.51 382.56 lz5 15 enwik8.tbb
35197151 35.2 8.78 375.05 libdeflate 9 enwik8.tbb
36475796 36.5 14.57 212.84 zlib 9 enwik8.tbb
36576568 822.0 lz5-2.0_beta 18 ! Outside TurboBench !
38063182 1856 LZSSE2 17 SSE4.1! Outside TurboBench !
38063182 1984 LZSSE2 17 AVX ! Outside TurboBench !
38063186 38.1 6.05 1981.30 lzsse2 17 enwik8.tbb
38261116 768 Nakamichi-Zato ! Outside TurboBench !
41924195 41.9 0.94 2443.32 lzturbo 19 enwik8.tbb
41956938 1809.8 lz5-2.0_beta 9 ! Outside TurboBench !
42196877 42.2 23.96 1802.61 lz4 16 enwik8.tbb
43196169 43.2 76.49 152.52 brieflz enwik8.tbb
45854489 45.9 11.65 1712.27 lzsse2 1 enwik8.tbb
47817696 47.8 218.21 205.05 density 3 enwik8.tbb
53844730 53.8 65.94 1450.22 yappy enwik8.tbb
53915996 53.9 177.09 358.29 lzf enwik8.tbb
54163017 54.2 173.58 311.66 fastlz 2 enwik8.tbb
56539849 56.5 249.67 631.71 snappy_c enwik8.tbb
61528422 61.5 1096.56 1661.82 chameleon 2 enwik8.tbb
99138398 99.1 147.45 1277.33 trle enwik8.tbb
100000004 100.0 6125.95 5728.36 memcpy enwik8.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 16:17:59 2016
C Size ratio% C MB/s D MB/s Name File
263042 22.0 0.32 0.31 zpaq 5 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
312058 26.1 10.06 20.84 bzip2 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
337771 28.3 0.43 272.73 brotli 11 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
343393 28.7 1.91 61.27 lzma 9 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
346407 29.0 2.33 489.11 zstd 22 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
347791 29.1 1.75 704.00 lzturbo 39 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
354786 29.7 1.54 136.34 lzham 4 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
376650 31.5 6.94 491.73 zstd 12 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
380752 31.9 2.43 521.55 xpack 9 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
396876 33.2 0.56 229.75 crush 2 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
410524 34.3 7.00 425.26 libdeflate 9 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
414525 34.7 4.81 59.35 zpaq 2 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
432292 36.2 9.60 224.40 zlib 9 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
433552 2176 LZSSE2 17 SSE4.1! Outside TurboBench !
433552 2176 LZSSE2 17 AVX ! Outside TurboBench !
433556 36.3 5.36 2127.04 lzsse2 17 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
451898 37.8 2.11 643.72 lz5 15 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
454747 38.0 1.80 862.48 lzturbo 29 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
468913 1152 Nakamichi-Zato ! Outside TurboBench !
475502 1419.7 lz5-2.0_beta 18 ! Outside TurboBench !
489322 40.9 2.17 2565.23 lzturbo 19 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
489595 1853.3 lz5-2.0_beta 9 ! Outside TurboBench !
495240 41.4 15.24 1864.89 lz4 16 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
529655 44.3 75.61 147.96 brieflz Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
594040 49.7 12.24 1655.67 lzsse2 1 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
616213 51.5 189.90 190.68 density 3 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
650156 54.4 61.36 1319.42 yappy Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
666147 55.7 175.41 357.05 lzf Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
677269 56.7 174.23 301.94 fastlz 2 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
707035 59.1 1084.75 1747.66 chameleon 2 Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
708040 59.2 241.06 634.16 snappy_c Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
1195311 100.0 147.47 1262.30 trle Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
1195401 100.0 7565.80 7762.32 memcpy Mike_Tyson_-_Undisputed_Truth_-_My_Autobiography_-_2013.epub.txt.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Slownik_jezyka_polskiego_(Pol-Pol).dsl -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 16:28:53 2016
C Size ratio% C MB/s D MB/s Name File
1851513 10.8 0.37 0.37 zpaq 5 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
2141578 12.4 1.11 124.47 lzma 9 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
2179075 12.7 0.90 263.14 lzham 4 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
2260440 13.1 0.99 683.08 lzturbo 39 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
2276496 13.2 1.30 503.93 zstd 22 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
2293510 13.3 0.38 374.40 brotli 11 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
2394859 13.9 9.09 27.47 bzip2 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
2840567 16.5 3.64 80.16 zpaq 2 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
2997101 17.4 2.47 589.93 xpack 9 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
3011302 17.5 10.79 676.74 zstd 12 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
3060604 1107.2 lz5-2.0_beta 18 ! Outside TurboBench !
3060960 17.8 0.07 380.21 crush 2 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
3091927 18.0 1.00 1037.46 lzturbo 29 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
3101206 1216 Nakamichi-Zato ! Outside TurboBench !
3231730 18.8 1.44 720.80 lz5 15 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
3353571 19.5 5.80 736.70 libdeflate 9 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
3496527 3200 LZSSE2 17 SSE4.1! Outside TurboBench !
3496527 3328 LZSSE2 17 AVX ! Outside TurboBench !
3496531 20.3 4.07 3243.61 lzsse2 17 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
3572553 20.8 3.10 319.69 zlib 9 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
4006357 23.3 1.22 2934.36 lzturbo 19 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
4012459 1970.0 lz5-2.0_beta 9 ! Outside TurboBench !
4138755 24.1 6.95 1812.87 lz4 16 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
4901067 28.5 70.29 2109.64 yappy Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
5500763 32.0 126.29 194.35 brieflz Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
5730218 33.3 12.13 2397.12 lzsse2 1 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
6101970 35.5 341.66 289.44 density 3 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
6356114 36.9 309.19 541.25 lzf Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
6652854 38.7 294.85 447.56 fastlz 2 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
6936171 40.3 330.45 868.41 snappy_c Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
9150049 53.2 1256.05 3298.34 chameleon 2 Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
17204132 100.0 6325.05 5638.85 memcpy Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
17204132 100.0 144.01 5562.28 trle Slownik_jezyka_polskiego_(Pol-Pol).dsl.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe Smyslov_(Vasily_Smyslov_2627_games).pgn -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 16:30:22 2016
C Size ratio% C MB/s D MB/s Name File
257330 15.5 0.34 0.33 zpaq 5 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
314975 19.0 8.51 25.07 bzip2 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
363995 21.9 1.53 69.25 lzma 9 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
364389 22.0 0.40 277.83 brotli 11 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
389023 23.4 1.89 525.16 zstd 22 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
390007 23.5 1.49 787.49 lzturbo 39 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
394133 23.7 1.53 154.98 lzham 4 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
418780 25.2 6.92 560.26 zstd 12 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
420151 25.3 1.95 599.08 xpack 9 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
456748 27.5 0.56 287.75 crush 2 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
466362 28.1 5.36 67.65 zpaq 2 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
474789 28.6 8.79 490.12 libdeflate 9 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
490598 29.6 12.43 269.27 zlib 9 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
495673 29.9 1.90 834.19 lz5 15 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
500488 30.1 1.64 1158.44 lzturbo 29 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
505572 3008 LZSSE2 17 SSE4.1! Outside TurboBench !
505572 3008 LZSSE2 17 AVX ! Outside TurboBench !
505576 30.5 4.45 2299.22 lzsse2 17 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
520201 1576.5 lz5-2.0_beta 18 ! Outside TurboBench !
541608 1280 Nakamichi-Zato ! Outside TurboBench !
557400 33.6 2.01 2585.73 lzturbo 19 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
557733 1964.5 lz5-2.0_beta 9 ! Outside TurboBench !
566194 34.1 23.06 1928.04 lz4 16 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
623758 37.6 96.30 184.37 brieflz Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
688581 41.5 220.43 233.22 density 3 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
710927 42.8 10.47 1790.76 lzsse2 1 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
768969 46.3 235.83 442.09 lzf Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
770887 46.4 242.16 418.78 fastlz 2 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
782311 47.1 75.40 1277.94 yappy Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
850739 51.2 295.33 676.19 snappy_c Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
937681 56.5 1097.18 2004.88 chameleon 2 Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
1659995 100.0 148.80 1300.97 trle Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
1660043 100.0 7410.89 7377.95 memcpy Smyslov_(Vasily_Smyslov_2627_games).pgn.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 16:38:19 2016
C Size ratio% C MB/s D MB/s Name File
2767461 20.2 0.31 0.30 zpaq 5 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
3554802 26.0 1.06 69.16 lzma 9 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
3562585 26.0 0.35 206.99 brotli 11 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
3567992 26.1 0.89 142.08 lzham 4 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
3585035 26.2 1.19 284.31 zstd 22 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
3591509 26.2 1.05 366.58 lzturbo 39 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
3703637 27.0 9.99 19.22 bzip2 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
4197653 30.7 0.58 323.83 xpack 9 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
4256970 31.1 4.27 452.97 zstd 12 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
4294809 31.4 2.97 55.37 zpaq 2 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
4371236 31.9 0.13 241.10 crush 2 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
4632254 33.8 1.07 624.71 lzturbo 29 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
4855421 35.5 1.30 372.19 lz5 15 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
4924148 36.0 7.39 408.26 libdeflate 9 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
5031882 960 Nakamichi-Zato ! Outside TurboBench !
5032797 826.5 lz5-2.0_beta 18 ! Outside TurboBench !
5158479 37.7 9.68 217.04 zlib 9 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
5160110 2048 LZSSE2 17 SSE4.1! Outside TurboBench !
5160110 2112 LZSSE2 17 AVX ! Outside TurboBench !
5160114 37.7 5.00 2034.31 lzsse2 17 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
5828546 42.6 1.26 2456.94 lzturbo 19 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
5838641 1758.9 lz5-2.0_beta 9 ! Outside TurboBench !
5892721 43.0 14.96 1811.50 lz4 16 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
6211745 45.4 77.62 145.52 brieflz The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
6526796 47.7 266.83 239.25 density 3 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
7029733 51.3 12.33 1592.99 lzsse2 1 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
7814850 57.1 1084.32 1902.87 chameleon 2 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
7913296 57.8 174.46 352.14 lzf The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
7978444 58.3 61.33 1223.09 yappy The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
8050844 58.8 170.32 306.06 fastlz 2 The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
8508270 62.1 235.40 611.22 snappy_c The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
13674844 99.9 142.59 1323.70 trle The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
13694980 100.0 6396.53 5440.99 memcpy The_complete_Aubrey_Maturin_by_Patrick_O'Brian_(21-TXTs).tar.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 16:43:20 2016
C Size ratio% C MB/s D MB/s Name File
1399417 19.6 0.31 0.31 zpaq 5 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
1835483 25.7 1.23 70.09 lzma 9 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
1839607 25.8 0.37 248.10 brotli 11 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
1858314 26.0 10.04 20.92 bzip2 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
1858842 26.0 1.04 151.87 lzham 4 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
1860631 26.1 1.45 380.19 zstd 22 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
1865477 26.1 1.22 477.92 lzturbo 39 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
2133573 29.9 4.78 482.44 zstd 12 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
2135754 29.9 0.74 413.11 xpack 9 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
2184827 30.6 0.14 249.13 crush 2 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
2223226 31.1 3.47 58.98 zpaq 2 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
2407621 33.7 1.24 692.67 lzturbo 29 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
2421732 33.9 7.13 432.41 libdeflate 9 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
2445769 34.3 1.43 447.98 lz5 15 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
2539892 35.6 8.70 228.14 zlib 9 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
2540805 1088 Nakamichi-Zato ! Outside TurboBench !
2556494 2112 LZSSE2 17 SSE4.1! Outside TurboBench !
2556494 2304 LZSSE2 17 AVX ! Outside TurboBench !
2556498 35.8 5.22 2141.40 lzsse2 17 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
2595499 1056.1 lz5-2.0_beta 18 ! Outside TurboBench !
2891200 40.5 1.44 2364.12 lzturbo 19 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
2896240 1809.7 lz5-2.0_beta 9 ! Outside TurboBench !
2929699 41.0 13.19 1850.47 lz4 16 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
3047250 42.7 79.41 151.27 brieflz The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
3405772 47.7 261.79 237.63 density 3 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
3505515 49.1 12.34 1662.93 lzsse2 1 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
3793663 53.2 61.37 1295.80 yappy The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
3917217 54.9 171.14 360.47 lzf The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
3984251 55.8 179.46 307.76 fastlz 2 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
4054300 56.8 1026.06 1958.10 chameleon 2 The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
4167383 58.4 248.95 629.89 snappy_c The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
7108504 99.6 148.08 1317.81 trle The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
7137284 100.0 6553.98 5765.17 memcpy The_Project_Gutenberg_12_Fairy_Books_by_Andrew_Lang.tar.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe University_of_Canterbury_The_Calgary_Corpus.tar -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 16:46:20 2016
C Size ratio% C MB/s D MB/s Name File
688059 21.1 0.31 0.31 zpaq 5 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
851327 26.1 1.75 60.80 lzma 9 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
863141 26.4 0.38 233.77 brotli 11 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
890859 27.3 11.64 25.20 bzip2 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
899273 27.5 2.23 493.73 zstd 22 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
902432 27.6 1.59 686.90 lzturbo 39 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
913218 28.0 1.61 137.29 lzham 4 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
958042 29.3 1.89 493.21 xpack 9 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
968043 29.6 9.54 532.63 zstd 12 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1021136 31.3 9.31 422.94 libdeflate 9 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1022374 31.3 0.70 235.13 crush 2 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1060310 32.5 9.04 235.18 zlib 9 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1077695 33.0 4.90 60.46 zpaq 2 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1114797 34.1 1.77 966.13 lzturbo 29 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1125796 34.5 1.83 688.06 lz5 15 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1142536 2112 LZSSE2 17 SSE4.1! Outside TurboBench !
1142536 2496 LZSSE2 17 AVX ! Outside TurboBench !
1142540 35.0 0.44 2042.24 lzsse2 17 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1192352 1509.7 lz5-2.0_beta 18 ! Outside TurboBench !
1231161 37.7 2.07 2625.03 lzturbo 19 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1237244 2014.5 lz5-2.0_beta 9 ! Outside TurboBench !
1240210 38.0 4.17 1942.62 lz4 16 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1298840 39.8 89.11 180.28 brieflz University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1311813 960 Nakamichi-Zato ! Outside TurboBench !
1405647 43.0 12.14 1749.08 lzsse2 1 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1596322 48.9 199.22 394.06 lzf University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1607581 49.2 194.25 360.83 fastlz 2 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1610706 49.3 206.42 194.15 density 3 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1624766 49.8 60.18 1554.28 yappy University_of_Canterbury_The_Calgary_Corpus.tar.tbb
1682025 51.5 289.47 734.82 snappy_c University_of_Canterbury_The_Calgary_Corpus.tar.tbb
2084324 63.8 1077.73 1576.79 chameleon 2 University_of_Canterbury_The_Calgary_Corpus.tar.tbb
2810417 86.1 191.46 5224.86 trle University_of_Canterbury_The_Calgary_Corpus.tar.tbb
3265540 100.0 6903.88 6610.40 memcpy University_of_Canterbury_The_Calgary_Corpus.tar.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>turbobenchs.exe www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt -ezlib,9/libdeflate,9/xpack,9/fastlz,2/lzturbo,19,29,39/bzip2/chameleon,2/snappy_c/zstd,12,22/density,3/lz4,16/lz5,15/lzham,4/brieflz/brotli,11/crush,2/lzma,9/zpaq,2,5/lzf/yappy/trle/memcpy/lzsse2,1,17 -g -k0 -J13 -B2G
TurboBench: - Sun Oct 02 16:52:02 2016
C Size ratio% C MB/s D MB/s Name File
973794 21.7 0.26 0.26 zpaq 5 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1098241 24.5 0.28 111.41 brotli 11 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1107450 24.7 1.02 78.87 lzma 9 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1115796 24.9 1.26 446.60 zstd 22 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1124307 25.1 0.89 152.31 lzham 4 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1156565 25.8 10.30 19.93 bzip2 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1163771 25.9 4.55 542.94 libdeflate 9 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1205730 26.9 1.08 294.70 zlib 9 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1229904 27.4 2.81 553.32 xpack 9 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1231733 27.4 7.63 478.05 zstd 12 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1297189 2432 LZSSE2 17 SSE4.1! Outside TurboBench !
1297189 2880 LZSSE2 17 AVX ! Outside TurboBench !
1297193 28.9 1.21 2429.58 lzsse2 17 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1366183 30.4 0.04 282.19 crush 2 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1396907 31.1 4.25 61.98 zpaq 2 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1437041 32.0 1.23 727.18 lz5 15 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1441772 32.1 1.47 867.14 lzturbo 29 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1456466 1152 Nakamichi-Zato ! Outside TurboBench !
1503145 1516.5 lz5-2.0_beta 18 ! Outside TurboBench !
1509154 33.6 1.76 2556.94 lzturbo 19 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1510839 2000.6 lz5-2.0_beta 9 ! Outside TurboBench !
1556225 34.7 50.22 2329.92 yappy www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1596981 35.6 1.79 2074.63 lz4 16 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1598279 35.6 145.80 224.33 brieflz www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
1995712 44.5 362.36 731.09 snappy_c www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
2161251 48.2 437.41 367.88 density 3 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
2385361 53.2 1224.07 3230.69 chameleon 2 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
2403235 53.6 240.33 441.11 lzf www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
2408470 53.7 219.97 406.62 fastlz 2 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
2559866 57.0 15.08 1358.59 lzsse2 1 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
3677928 82.0 159.99 856.38 trle www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
3849226 85.8 1.38 3717.84 lzturbo 39 www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
4487437 100.0 6758.18 6198.11 memcpy www.ncbi.nlm.nih.gov_(Ladona_fulva_-_whole_genome_shotgun_sequence)_sequence.fasta.txt.tbb
D:\LZ5_v2beta_VS_LZSSE2_VS_Zato>
Hope that someone will give the above picture on some serious machine, not like my crappy i5 ruining the beautiful ... exploitation of registers.
Still haven't given up on making the richest textual roster with 400+ files...
https://1drv.ms/i/s!AmWWFXGMzDmEgmI7l-33IsTzHnMy
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.