Compare various data serialization libraries for C++.

• Thrift
• Protobuf
• Boost.Serialization
• Msgpack
• Cereal
• Avro
• Capnproto
• Flatbuffers
• YAS

This project does not have any external serialization libraries dependencies. All (boost, thrift etc.) needed libraries are downloaded and built automatically, but you need enough free disk space (approx. 2.3G) to build all components. To build this project you need a compiler that supports C++14 features. Project was tested with Clang and GCC compilers.

1. git clone https://github.com/thekvs/cpp-serializers.git
2. cd cpp-serializers
3. mkdir build
4. cd build
5. cmake -DCMAKE_BUILD_TYPE=Release ..
6. cmake --build .

$ ./benchmark -h
Benchmark various C++ serializers
Usage:
  benchmark [OPTION...]

  -h, --help             show this help and exit
  -l, --list             show list of supported serializers
  -c, --csv              output in CSV format
  -i, --iterations arg   number of serialize/deserialize iterations
  -s, --serializers arg  comma separated list of serializers to benchmark

• Benchmark all serializers, run each serializer 100000 times:

$ ./benchmark -i 100000

• Benchmark only protobuf serializer, run it 100000 times:

$ ./benchmark -i 100000 -s protobuf

• Benchmark protobuf and cereal serializers only, run each of them 100000 times:

$ ./benchmark -i 100000 -s protobuf,cereal

Following results were obtained by running 100000 serialize-deserialize operations 3 times and then averaging results on a server, which equits with Intel Xeon E5-2650 processor and 2400MHz, 32G DDR4 memory. And the operating system is ubuntu 16.04 server Exact versions of libraries used are:

• thrift 0.12.0
• protobuf 3.7.0
• boost 1.69.0
• msgpack 3.1.1
• cereal 1.2.2
• avro 1.8.2
• capnproto 0.7.0
• flatbuffers 1.10.0
• YAS 7.0.2

For capnproto and flatbuffers since they already store data in a "serialized" form and serialization basically means getting pointer to the internal storage, we measure full build/serialize/deserialize cycle. In the case of other libraries we measure serialize/deserialize cycle of the already built data structure.

In order to see the time of serialization and deserialization respectively, we modify benchmark.cpp to serialize-deserilize iteration same objects, and record serialization & deserialization time.

However, we need to do some preprocess to warm-up , in order to avoid the effect of memory allocation and memory copy when using some classes which would allocate memory dynamically during running, like std::string, std::ostringstream etc.

Specially, we modify the constructor of class sbuffer in src/msgpack/sbuffer, and next() of MemoryOutputStream in lang/c++/impl/Stream.cc (please see the following picture)

The following results were obtained by running serialize-deserialize 100000 same objects 3 times and averaging results. The environment is listed above.