ISerializer'1.Read seems to be the core. used often and recursively

facts

• the code is a lot faster for smaller chunksizes but also far away from linear scaling
• we see lowest cpu usage
• we don't see significant memory usage

assumtions

• more concurrency insider the de-serializer does not help
• locks and lockcontetion not involved (lot of tests)

questions

• also if the reader is slow, why parallelization is not helping ?
• the reader gets slower and slower with the growing of the bulk-size
• which resource/artifact is not scaling withe the number of nodes in one bucket ?
• can be the .net memory and instantiating system be the root cause ?
  • the number of nodes is always the same. with smaller buckets of nodes we are faster
• what is making slow the deserializer for bigger nodes numbers
  • this must be something that has shared user over multiple threads but it must be isolated via processes!

Some Benchmark results (before optimizing)

Issue and Symptoms

Doing a higher number of protobuf deserializations, I'm not able to improve performance using more tasks/threads!

I have written a smallest sample that should help to isolate the problem.
It is a console program, so it can be executed and also some profiling tool can be used.
The output on my machine looks like the following:

We see on the green rows: adding more Tasks/Threads reduces the deserialization rate inside all threads. At the end there is basically no win.

In numbers: 4.9 seconds with 1 thread, 3.3 seconds with 8 threads.
My expectation for 8 threads in this scenario: worst case less then 2 seconds, best case less then one seconds (not talking about milliseconds at this time).

Also the Taskmanager shows that the CPU Power is not used.

What I'm doing wrong ?

About the example

A proto file and generated c# types are part of this repo (MIT licensed).
All of them are coming from the OsmSharp OpenStreetMap file format (MIT licensed)

Any idea, any help is welcome ;-)

BR
Werner