thekvs / cpp-serializers Goto Github PK
View Code? Open in Web Editor NEWBenchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++
Benchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++
For Google's protobuf, when a significant portion of negative numbers is expected, use sint64 instead of int64 in the proto definition will make it run much faster and give a much smaller size.
Currently, benchmark.cpp has everything lumped together in one file. It would be helpful if there were a separate program for each library being tested. That way the sizes of the programs could be compared.
zlib is not included.
"This project does not have any external library dependencies. All (boost, thrift etc.) needed libraries are downloaded and built automatically, but you need enough free disk space to build all components"
Fails to build on Windows using CMake 3.7.2, MSVC 2015 x64
I searched for a simple cmake command such as DISABLE_THRIFT
or something similar to compile and benchmark a subset of serializers, but couldn't find any. Is there any simple way to avoid building all of these serializers?
Pretty nice benchmark
Did you see this library https://github.com/HBPVIS/ZeroBuf ?
From glancing at your graphs, for say capnproto vs flatbuffers, one might think that capnproto takes half the time.
Ditto for the other graphs. You should ensure that the bottom of the y axis is zero.
Would it be possible to add the ORC file format to the benchmarks here?
See:
Also:
Here is a list
function ** avro_serialization_test** is not at all efficient. As can be seen, for every encode / decode new resources are initialised, not an optimal technique.
Also the the default avro::InputStream and avro::OutputStream implementations are very rudimentary. Best to use a custom implementation.
for (size_t i = 0; i < iterations; i++) {
auto out = avro::memoryOutputStream();
auto encoder = avro::binaryEncoder();
encoder->init(*out);
avro::encode(*encoder, r1);
auto in = avro::memoryInputStream(*out);
auto decoder = avro::binaryDecoder();
decoder->init(*in);
avro::decode(*decoder, r2);
}
I found avro to be as performant as protobuf after making necessary changes
Quick comment that your images make it appear that e.g. protobuf'd objects are 10x smaller than cap'n proto ones, which is untrue. This is because the y-axis doesn't start at 'zero' (which is considered bad practice for this reason).
For anyone thinking about these formats for client-server communication, they will want to know what the size of them is after gzipping or applying other forms of compression. It can be very surprising - JSON can beat binary formats.
$ cmake -DCMAKE_BUILD_TYPE=Release ..
-- The CXX compiler identification is GNU 12.2.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Performing Test CXX14
-- Performing Test CXX14 - Success
-- Looking for include file inttypes.h
-- Looking for include file inttypes.h - found
-- Looking for include file netinet/in.h
-- Looking for include file netinet/in.h - found
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.13")
-- Found autoreconf: /usr/bin/autoreconf
-- Found libtool: /usr/bin/libtoolize
CMake Warning (dev) at /usr/share/cmake-3.25/Modules/ExternalProject.cmake:3075 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
/usr/share/cmake-3.25/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
CMakeLists.txt:55 (ExternalProject_Add)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at /usr/share/cmake-3.25/Modules/ExternalProject.cmake:3075 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
/usr/share/cmake-3.25/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
CMakeLists.txt:67 (ExternalProject_Add)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Using Boost from /home/marek/git/cpp-serializers/build/external/boost/include
CMake Warning (dev) at /usr/share/cmake-3.25/Modules/ExternalProject.cmake:3075 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
/usr/share/cmake-3.25/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
CMakeLists.txt:86 (ExternalProject_Add)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at /usr/share/cmake-3.25/Modules/ExternalProject.cmake:3075 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
/usr/share/cmake-3.25/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
CMakeLists.txt:104 (ExternalProject_Add)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at /usr/share/cmake-3.25/Modules/ExternalProject.cmake:3075 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
/usr/share/cmake-3.25/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
CMakeLists.txt:118 (ExternalProject_Add)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at /usr/share/cmake-3.25/Modules/ExternalProject.cmake:3075 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
/usr/share/cmake-3.25/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
CMakeLists.txt:135 (ExternalProject_Add)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at /usr/share/cmake-3.25/Modules/ExternalProject.cmake:3075 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
/usr/share/cmake-3.25/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
CMakeLists.txt:153 (ExternalProject_Add)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at /usr/share/cmake-3.25/Modules/ExternalProject.cmake:3075 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
/usr/share/cmake-3.25/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
CMakeLists.txt:164 (ExternalProject_Add)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at /usr/share/cmake-3.25/Modules/ExternalProject.cmake:3075 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
/usr/share/cmake-3.25/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
CMakeLists.txt:183 (ExternalProject_Add)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at /usr/share/cmake-3.25/Modules/ExternalProject.cmake:3075 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
/usr/share/cmake-3.25/Modules/ExternalProject.cmake:4185 (_ep_add_download_command)
CMakeLists.txt:197 (ExternalProject_Add)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/marek/git/cpp-serializers/build
$ cmake --build .
[ 1%] Creating directories for 'cxxopts'
[ 2%] Performing download step (download, verify and extract) for 'cxxopts'
-- Downloading...
dst='/home/marek/git/cpp-serializers/build/external/cxxopts/src/v2.1.2.tar.gz'
timeout='none'
inactivity timeout='none'
-- Using src='https://github.com/jarro2783/cxxopts/archive/v2.1.2.tar.gz'
-- verifying file...
file='/home/marek/git/cpp-serializers/build/external/cxxopts/src/v2.1.2.tar.gz'
-- Downloading... done
-- extracting...
src='/home/marek/git/cpp-serializers/build/external/cxxopts/src/v2.1.2.tar.gz'
dst='/home/marek/git/cpp-serializers/build/external/cxxopts/src/cxxopts'
-- extracting... [tar xfz]
-- extracting... [analysis]
-- extracting... [rename]
-- extracting... [clean up]
-- extracting... done
[ 3%] No update step for 'cxxopts'
[ 4%] No patch step for 'cxxopts'
[ 5%] Performing configure step for 'cxxopts'
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/marek/git/cpp-serializers/build/external/cxxopts/src/cxxopts-build
[ 6%] Performing build step for 'cxxopts'
CMake Error at /home/marek/git/cpp-serializers/build/external/cxxopts/src/cxxopts-stamp/cxxopts-build-Release.cmake:49 (message):
Command failed: 2
'/usr/bin/gmake'
See also
/home/marek/git/cpp-serializers/build/external/cxxopts/src/cxxopts-stamp/cxxopts-build-*.log
gmake[2]: *** [CMakeFiles/cxxopts.dir/build.make:86: external/cxxopts/src/cxxopts-stamp/cxxopts-build] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:103: CMakeFiles/cxxopts.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2
i am having problem compiling this project on Mac. is this tested working on Mac yet?
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... configure: error: in /Users/xxx/Library/Caches/CLion2016.2/cmake/generated/cpp-serializers-master-d78257/d78257/Debug/external/protobuf/src/protobuf': configure: error: cannot run C compiled programs. If you meant to cross compile, use
--host'.
See `config.log' for more details
make[2]: *** [external/protobuf/src/protobuf-stamp/protobuf-configure] Error 1
make[1]: *** [CMakeFiles/protobuf.dir/all] Error 2
make: *** [all] Error 2
Could you include https://github.com/felixguendling/cista ?
I would be very interested in how it performs against the others...
When compiling this project as shown in the README, it gives
[ 17%] Completed 'capnproto'
[ 17%] Built target capnproto
Scanning dependencies of target flatbuffers
[ 18%] Creating directories for 'flatbuffers'
[ 19%] Performing download step (download, verify and extract) for 'flatbuffers'
-- Downloading...
dst='/home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/v1.10.0.tar.gz'
timeout='none'
-- Using src='https://github.com/google/flatbuffers/archive/v1.10.0.tar.gz'
-- [download 100% complete]
-- verifying file...
file='/home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/v1.10.0.tar.gz'
-- Downloading... done
-- extracting...
src='/home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/v1.10.0.tar.gz'
dst='/home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/flatbuffers'
-- extracting... [tar xfz]
-- extracting... [analysis]
-- extracting... [rename]
-- extracting... [clean up]
-- extracting... done
[ 20%] No patch step for 'flatbuffers'
[ 21%] No update step for 'flatbuffers'
[ 22%] Performing configure step for 'flatbuffers'
-- flatbuffers configure command succeeded. See also /home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/flatbuffers-stamp/flatbuffers-configure-*.log
[ 23%] Performing build step for 'flatbuffers'
CMake Error at /home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/flatbuffers-stamp/flatbuffers-build-Release.cmake:49 (message):
Command failed: 2
'/usr/bin/make'
See also
/home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/flatbuffers-stamp/flatbuffers-build-*.log
make[2]: *** [CMakeFiles/flatbuffers.dir/build.make:115: external/flatbuffers/src/flatbuffers-stamp/flatbuffers-build] Error 1
make[1]: *** [CMakeFiles/Makefile2:240: CMakeFiles/flatbuffers.dir/all] Error 2
make: *** [Makefile:84: all] Error 2
with the further inner log being
$ cat /home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/flatbuffers-stamp/flatbuffers-build-*.log
In file included from /home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/flatbuffers/tests/test.cpp:32:
/home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/flatbuffers/tests/monster_test_generated.h: In constructor ‘MyGame::Example::Vec3::Vec3()’:
/home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/flatbuffers/tests/monster_test_generated.h:315:33: error: ‘void* memset(void*, int, size_t)’ clearing an object of non-trivial type ‘struct MyGame::Example::Vec3’; use assignment or value-initialization instead [-Werror=class-memaccess]
315 | memset(this, 0, sizeof(Vec3));
| ^
/home/max/md_compile_test/cpp-serializers/build/external/flatbuffers/src/flatbuffers/tests/monster_test_generated.h:301:41: note: ‘struct MyGame::Example::Vec3’ declared here
301 | FLATBUFFERS_MANUALLY_ALIGNED_STRUCT(16) Vec3 FLATBUFFERS_FINAL_CLASS {
| ^~~~
cc1plus: all warnings being treated as errors
make[5]: *** [CMakeFiles/flattests.dir/build.make:145: CMakeFiles/flattests.dir/tests/test.cpp.o] Error 1
make[4]: *** [CMakeFiles/Makefile2:113: CMakeFiles/flattests.dir/all] Error 2
make[3]: *** [Makefile:141: all] Error 2
Machine is running Ubuntu Budgie 20.04, x64.
You provide nice benchmarks on your readme, however i'm curious what the original size of the object is before serialization.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.