Comments (4)
Sanmayce
commented on April 28, 2024
The above quick run was done on my Core 2 laptop, on Haswell the 3x may jump up to 5x hands down, hate that I don't have Haswell or alike to share the actual stats.
I am not alone, one overclocker helped me a lot to benchmark your Brotli against the superb Zstd and my Shifune:
- 1st column: Intel Core 2 Q9550s @2833MHz laptop Asus;
- 2nd column: Intel i7 5960x 'Haswell E/EP' core/cache @4700MHz/@4200MHz RAM @2666MHz; MOBO Asus Rampage V Extreme; http://cdn.overclock.net/8/8b/8b2c8f32_decompshowdown.jpeg
- 3rd column: Intel Core i5 6600K 'Skylake' core/cache @4700MHz/4500MHz RAM @3200MHz CL15; MOBO Asus Maximus VIII Extreme; http://cdn.overclock.net/6/6b/6b1770b9_decompshowdown.jpeg
D:\Showdown_Brotli_vs_Zstd_vs_GZIP_vs_Shifune>dir
09/26/2015 10:08 PM 13,713,275 Complete_Works_of_Fyodor_Dostoyevsky.txt
09/26/2015 10:35 PM 3,717,191 Complete_Works_of_Fyodor_Dostoyevsky.txt.4MB.lzturbo12-39.lzt
09/26/2015 10:16 PM 3,717,583 Complete_Works_of_Fyodor_Dostoyevsky.txt.brotli ! 153.228 MB/s; 364.29 MB/s; 397.508 MB/s !
09/08/2015 02:33 AM 4,582,363 Complete_Works_of_Fyodor_Dostoyevsky.txt.Nakamichi ! 448 MB/s; 2112 MB/s; 1728 MB/s!
09/26/2015 10:19 PM 4,617,360 Complete_Works_of_Fyodor_Dostoyevsky.txt.zip
09/26/2015 10:11 PM 5,209,670 Complete_Works_of_Fyodor_Dostoyevsky.txt.zst ! 302.5 MB/s; 619.8 MB/s; 628.3 MB/s !
09/26/2015 10:08 PM 10,192,446 dickens
09/26/2015 10:35 PM 2,976,910 dickens.4MB.lzturbo12-39.lzt
09/26/2015 10:17 PM 2,962,118 dickens.brotli ! 146.61 MB/s; 336.342 MB/s; 374.577 MB/s !
09/08/2015 02:33 AM 3,740,418 dickens.Nakamichi ! 448 MB/s; 1984 MB/s; 1664 MB/s !
09/26/2015 10:19 PM 3,681,828 dickens.zip
09/26/2015 10:11 PM 4,134,924 dickens.zst ! 298.9 MB/s; 619.7 MB/s; 623.8 MB/s !
09/26/2015 10:09 PM 100,000,000 enwik8
09/26/2015 10:37 PM 29,148,393 enwik8.4MB.lzturbo12-39.lzt
09/26/2015 10:24 PM 27,722,164 enwik8.brotli ! 134.094 MB/s; 285.19 MB/s; 305.225 MB/s !
09/08/2015 02:33 AM 34,218,460 enwik8.Nakamichi ! 256 MB/s; 1024 MB/s; 1152 MB/s !
09/26/2015 10:21 PM 35,102,891 enwik8.zip
09/26/2015 10:11 PM 40,024,854 enwik8.zst ! 325.0 MB/s; 651.6 MB/s; 653.8 MB/s !
09/26/2015 10:09 PM 14,613,183 The_Book_of_The_Thousand_Nights_and_a_Night.txt
09/26/2015 10:37 PM 4,241,855 The_Book_of_The_Thousand_Nights_and_a_Night.txt.4MB.lzturbo12-39.lzt
09/26/2015 10:19 PM 4,163,630 The_Book_of_The_Thousand_Nights_and_a_Night.txt.brotli ! 144.867 MB/s; 336.624 MB/s; 372.129 MB/s !
09/08/2015 02:33 AM 5,293,102 The_Book_of_The_Thousand_Nights_and_a_Night.txt.Nakamichi ! 384 MB/s; 1984 MB/s; 1600 MB/s !
09/26/2015 10:22 PM 5,198,949 The_Book_of_The_Thousand_Nights_and_a_Night.txt.zip
09/26/2015 10:11 PM 5,932,453 The_Book_of_The_Thousand_Nights_and_a_Night.txt.zst ! 305.8 MB/s; 625.4 MB/s; 631.3 MB/s !
09/26/2015 10:09 PM 4,445,260 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt
09/26/2015 10:37 PM 1,089,279 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt.4MB.lzturbo12-39.lzt
09/26/2015 10:19 PM 1,087,439 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt.brotli ! 160.277 MB/s; 339.146 MB/s; 370.247 MB/s !
09/08/2015 02:33 AM 1,441,679 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt.Nakamichi ! 704 MB/s; 2432 MB/s; 2368 MB/s !
09/26/2015 10:22 PM 1,320,100 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt.zip
09/26/2015 10:11 PM 1,537,047 The_Project_Gutenberg_EBook_of_The_King_James_Bible_kjv10.txt.zst ! 320.7 MB/s; 656.1 MB/s; 668.6 MB/s !
09/26/2015 10:09 PM 3,265,536 University_of_Canterbury_The_Calgary_Corpus.tar
09/26/2015 10:37 PM 921,958 University_of_Canterbury_The_Calgary_Corpus.tar.4MB.lzturbo12-39.lzt
09/26/2015 10:19 PM 867,503 University_of_Canterbury_The_Calgary_Corpus.tar.brotli ! 144.849 MB/s; 266.176 MB/s; 283.114 MB/s !
09/08/2015 02:33 AM 1,319,701 University_of_Canterbury_The_Calgary_Corpus.tar.Nakamichi ! 576 MB/s; 1792 MB/s; 1792 MB/s !
09/26/2015 10:22 PM 1,017,658 University_of_Canterbury_The_Calgary_Corpus.tar.zip
09/26/2015 10:11 PM 1,174,349 University_of_Canterbury_The_Calgary_Corpus.tar.zst ! 367.3 MB/s; 740.6 MB/s; 747.2 MB/s !
D:\Showdown_Brotli_vs_Zstd_vs_GZIP_vs_Shifune>
Note: Skylake is good, however being newer than Broadwell I expected more.
from brotli.
Sanmayce
commented on April 28, 2024
I was wrong about 5x hands down, in fact it is 4x-6x.
from brotli.
Sanmayce
commented on April 28, 2024
Just a note on the unfairness about comparing LzTurbo, I deliberately hurt the compression ratio of LzTurbo by choosing 4MB block, my intention was to silent the empty-talkers who always complain, using 4MB sliding window gives significantly better compression since the inheritance in chunks/blocks is lost, this is in its turn unfair to LzTurbo!
Nowadays, with constantly growing #cores & #caches & RAM size the bigger is the better, so all these resources have to be utilized not like now - UNDERUTILIZED.
However, running with small resources footprint is so cool - BUT ONLY IN HEAVY MULTI-THREADING, yes?!
from brotli.
eustas
commented on April 28, 2024
I've added support for icc recently.
Though it looks that icc and clang produce about 10% slower binary than gcc 5.2.0
from brotli.
Related Issues (20)
- OSS-Fuzz issue 60394 HOT 1
- Ability to clone/serialize/deserialize state in Compressor and Decompressor HOT 4
- Exception while brotliDecode in decode.ts HOT 5
- EMPTY VERSION FIELDS IN PKGCONFIG FILES (LINUX) HOT 4
- Support for user-supplied dictionaries in Python binding
- Release v1.1 HOT 14
- Publish JNI artefacts (including platform-dependent) HOT 7
- Stale comment referring to nonexisting WriteMetadata() function HOT 6
- v1.1.0rc does not build on macOS 10.12.6 HOT 8
- Strange compression ratio on large CSV file... HOT 1
- Brotli v1.1.0 tests fail with pypy3 HOT 10
- Brotli 1.1.0 breaks Python 2 compatibility HOT 3
- Create a static source tarball for releases
- CMake build broken, tries to install man files in /man not in CMAKE_INSTALL_PREFIX
- bazel error no such package '@org_brotli// HOT 4
- Brotli 1.1.0 NginX HOT 3
- Brotli 1.1.0 changed return values? HOT 5
- [Feature Request] adding a option to control whether install the executable HOT 4
- makefile broken HOT 7
- undefined symbol BrotliEncoderDestroyPreparedDictionary HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from brotli.