eishay / jvm-serializers Goto Github PK

The latest version of the wiki shows total times that seem to be an order of magnitude higher than previously reported. The results from running on my macbook pro are also more in line with previous results.

https://blogs.oracle.com/javamagazine/post/simpler-object-and-data-serialization-using-java-records

Oracle is claiming ObjectOutputStream/ObjectInputStream have optimizations for Java Records. Can we implement a benchmark to see if Java serialization is any faster with records rather than plain classes?

Unfortunately it doesn't compile under java 8.
First problem was the Scala version (2.8) is incompatible. I upgraded the support compiler to 2.11 but that doesn't work at all, as some required annotations have been removed (@serialization)
2.10 seems to get further ( The annotations are present, but deprecated) but I then blows up at another error

error: bad symbolic reference. A signature in Generic.class refers to type Collection
in package scala.package which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling Generic.class.

Include protobuf-json as well in the benchmarks https://github.com/dpp-name/protobuf-json?

http://msgpack.org/

Hi there,
first of all thank you for the great benchmark. It is a very nice resource.
I wanted to point out that format which support random value access like Cap'nProto and FlatBuffers are measured a bit unfair. The deserialise methods of the test make a copy of every value encoded in the buffer even though it is not necessary. The buffers are serialised in a way that make them accessible on demand, without the need of actual unmarshaling.
In this gist you can find a rewritten test for FlatBuffers:
https://gist.github.com/mzaks/a0dd3d68f8958da72068022749469531
Which passes and returns following result for decoding:
average:5.92301542549922ms deviation:0.6135154254992203ms
Formats which invest in supporting random value access do it at cost of size and encoding speed.
This is why I find it unfair to show the shortcomings of encoding and go around the benefits of the decoding.

http://msgpack.org/ is a robust serialization library. Would be nice to see its performance compared.

Hi,

the create time is visible in the performance breakdown but not the deallocation time. For applications which huge number of object creation the object deletion might be just as important for the performance.

Looks like there's finally a prototype for ubjson (https://github.com/thebuzzmedia/universal-binary-json-java), so it would be nice to see how it fares.

Since we seem to have lingering disagreements on whether to reuse ByteArrayOutputStream used for output, it would make sense to add one more command-line switch to enable/disable such reuse.

I know this might seem trivial to the authors, but not easy for outsiders like me.

Right now, most of the protocol versions in this master branch are out of date and I saw @vbehar made effort in trying to update the versions in 2013: #36.

I forked this awesome repo and tried to update on my own, so far, only protobuf works out (generated a new MediaContentHolder.java class using protobuf 3.1.0, replace the protobuf jar file under lib/ folder and re-run ./run-bench.sh protobuf ).

I tried to do the same thing for the rest of the protocols, e.g. avro, thrift, flatbuffers, etc. But none of them worked out, where I'm blocked is, take avro as an example:

I replaced all three jar files for avro with the latest one: avro-tools-1.8.1.jar, avro-1.8.1.jar and avro-compiler-1.8.1.jar, then try to run make under tpc/ directory, it's always throwing

Compile: pregen/media.proto
pregen/media.proto/serializers/protobuf/media/MediaContentHolder.java:11: error: Image is not abstract and does not override abstract method newBuilderForType(BuilderParent) in GeneratedMessage
	public static final class Image extends
	                    ^
pregen/media.proto/serializers/protobuf/media/MediaContentHolder.java:274: error: serializers.protobuf.media.MediaContentHolder.Image.Builder is not abstract and does not override abstract method internalGetFieldAccessorTable() in com.google.protobuf.GeneratedMessage.Builder
		public static final class Builder extends
		                    ^
pregen/media.proto/serializers/protobuf/media/MediaContentHolder.java:535: error: Media is not abstract and does not override abstract method newBuilderForType(BuilderParent) in GeneratedMessage
	public static final class Media extends
	                    ^
pregen/media.proto/serializers/protobuf/media/MediaContentHolder.java:892: error: serializers.protobuf.media.MediaContentHolder.Media.Builder is not abstract and does not override abstract method internalGetFieldAccessorTable() in com.google.protobuf.GeneratedMessage.Builder
		public static final class Builder extends
		                    ^
pregen/media.proto/serializers/protobuf/media/MediaContentHolder.java:1338: error: MediaContent is not abstract and does not override abstract method newBuilderForType(BuilderParent) in GeneratedMessage
	public static final class MediaContent extends
	                    ^
pregen/media.proto/serializers/protobuf/media/MediaContentHolder.java:1502: error: serializers.protobuf.media.MediaContentHolder.MediaContent.Builder is not abstract and does not override abstract method internalGetFieldAccessorTable() in com.google.protobuf.GeneratedMessage.Builder
		public static final class Builder extends

The above stacktrace comes from: I tried latest version of avro with old version (version in master branch currently) of protobuf, then I thought it's complaining about the protobuf, then I used the newly generated protobuf java class: MediaContentHolder.java with 3.1.0 protobuf jar, it's not compiling either and with the following stacktrace:

# Compile: pregen/media.proto
pregen/media.proto/serializers/protobuf/media/MediaContentHolder.java:828: error: cannot find symbol
				com.google.protobuf.Descriptors.OneofDescriptor oneof) {
				                               ^
  symbol:   class OneofDescriptor
  location: class Descriptors
pregen/media.proto/serializers/protobuf/media/MediaContentHolder.java:2669: error: cannot find symbol
				com.google.protobuf.Descriptors.OneofDescriptor oneof) {
				                               ^
  symbol:   class OneofDescriptor
  location: class Descriptors
pregen/media.proto/serializers/protobuf/media/MediaContentHolder.java:4131: error: cannot find symbol
				com.google.protobuf.Descriptors.OneofDescriptor oneof) {
				                               ^
  symbol:   class OneofDescriptor
  location: class Descriptors
3 errors

Looks like very obvious error, but I'm not able to figure it out. Any steps I'm missing? Any idea would be greatly appreciated!

The same applies to my other tries to benchmark other protocols of their latest version.

So, any help or instructions on how to update the protocols to run with their latest versions would be greatly appreciated!
I think it will also be helpful for other people to use this project.
So far, this is the best project I found on the Internet to benchmark Java serialization performance, if all authors/contributors are busy at the moment, I'd be more than happy to volunteer to maintain the wiki and keep the code up to date.

Cheers!

Thanks!

On running make, several classes are missing.

Compile: src [javac]
src/serializers/jackson/JacksonAvroDatabind.java:12: package com.fasterxml.jackson.dataformat.avro does not exist
import com.fasterxml.jackson.dataformat.avro.AvroFactory;
                                            ^
src/serializers/jackson/JacksonAvroDatabind.java:13: package com.fasterxml.jackson.dataformat.avro does not exist
import com.fasterxml.jackson.dataformat.avro.AvroSchema;
                                            ^
src/serializers/jackson/JacksonAvroDatabind.java:21: cannot find symbol
symbol  : class AvroFactory
location: class serializers.jackson.JacksonAvroDatabind
        ObjectMapper mapper = new ObjectMapper(new AvroFactory());
                                                   ^
src/serializers/jackson/JacksonAvroDatabind.java:24: cannot find symbol
symbol  : class AvroSchema
location: class serializers.jackson.JacksonAvroDatabind
        AvroSchema schema = new AvroSchema(Avro.Media.sMediaContent);
        ^
src/serializers/jackson/JacksonAvroDatabind.java:24: cannot find symbol
symbol  : class AvroSchema
location: class serializers.jackson.JacksonAvroDatabind
        AvroSchema schema = new AvroSchema(Avro.Media.sMediaContent);

To fix it, the following jar file needs to be put into the lib directory:

cd jvm-serializers/tcp/lib
wget http://search.maven.org/remotecontent?filepath=com/fasterxml/jackson/dataformat/jackson-dataformat-avro/2.1.2/jackson-dataformat-avro-2.1.2.jar

As the title implies, add benchmark for fastjson2

fastjson2 is successor of fastjson

Can you add Moshi from Square https://github.com/square/moshi as one of the serializer and compare it with the others?

https://github.com/scala/pickling

I'd love to see where capnproto falls in here...
https://capnproto.org

It would be nice to see Amazon Ion added to the benchmarks.

The serializers in the serializers.avro directory all have media imports that do not exist in the library.
The media sub-directory under the serializers.avro directory does not exist.
Here are the non-existent imports:

import serializers.avro.media.Image;
import serializers.avro.media.Media;
import serializers.avro.media.MediaContent;

What am I missing?

make details
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0

error when i do make in tpc folder

build/bytecode/main/serializers/stephenerialization/MediaStephenerializer.java:41: warning: [unchecked] unchecked cast object.setPersons((java.util.List<java.lang.String>) stream.readObject()); ^ required: List<String> found: Object build/bytecode/main/serializers/stephenerialization/MediaContentStephenerializer.java:23: warning: [unchecked] unchecked cast object.setImages((java.util.List<serializers.stephenerialization.Image>) stream.readObject()); ^ required: List<Image> found: Object warning: Implicitly compiled files were not subject to annotation processing. Use -proc:none to disable annotation processing or -implicit to specify a policy for implicit compilation. 4 warnings

Can someone with a recent high-end CPU [Xeon or equivalent] run the benchmarks and update the wiki content?

Not a huge deal, just wanted to bring this to your attention.

src/serializers/jackson/BaseJacksonDataBind.java:24: warning: [deprecation] <W>writerWithType(JavaType) in ObjectMapper has been deprecated
        writer = mapper.writerWithType(type);
                       ^
  where W is a type-variable:
    W extends ObjectWriter declared in method <W>writerWithType(JavaType)
src/serializers/jackson/JacksonAvroDatabind.java:25: warning: [deprecation] <W>writerWithType(JavaType) in ObjectMapper has been deprecated
        ObjectWriter writer = mapper.writerWithType(type).withSchema(schema);
                                    ^
  where W is a type-variable:
    W extends ObjectWriter declared in method <W>writerWithType(JavaType)
src/serializers/jackson/JacksonAvroDatabind.java:25: warning: [deprecation] withSchema(FormatSchema) in ObjectWriter has been deprecated
        ObjectWriter writer = mapper.writerWithType(type).withSchema(schema);
                                                         ^
src/serializers/jackson/JacksonBsonDatabind.java:57: warning: [deprecation] <W>writerWithType(JavaType) in ObjectMapper has been deprecated
        objectWriter = mapper.writerWithType(type);
                             ^
  where W is a type-variable:
    W extends ObjectWriter declared in method <W>writerWithType(JavaType)

This might be a dumb question, I'm not able to run this project with non-ASCII test case, but as the wiki says, it should be.

The wiki says: "Other test data will yield different results (e.g. adding a non ascii char to every string :-) ). "

And I used one non-ascii word in the test, but got "invalid character in string literal: #c2a4", here's how I'm using it:

Images = [
{
Uri = "http://javaone.com/keynote_large.jpg"
Title = Set: "14K 스와로브스키 시그니티 1캐럿 목걸이_NC110"
Width = 1024
Height = 768
Size = Large
}
{
Uri = "http://javaone.com/keynote_large.jpg"
Title = Set: "보쉬 두루마리 100P 파우치세트 드릴비트 100PCS 보쉬 100PCS GSB10RE "
Width = 1024
Height = 800
Size = Large
}
]

It's complaining at this line: Title = Set: "14K 스와로브스키 시그니티 1캐럿 목걸이_NC110"

The Title field is String.

How can I enable non-ASCII for testing?

Thanks a lot!