Caching performance should be better about mountpoint-s3 HOT 4 OPEN

After sprinkling in a bunch of instrument attributes, spans, and a flamegraph layer; I can confirm that all of our time is spent in deserializing at mountpoint_s3::data_cache::disk_data_cache.

from mountpoint-s3.

Based on looking at metrics in the debug logs it looks like the cache is "working", blocks are pulled and read from cache. The performance is not close to what I would expect for reading from disk. The same 10G file copied to my /mnt/raid0 disk comes off as such...

# time cat /mnt/raid0/file_1 > /dev/null

real	0m1.150s
user	0m0.000s
sys	0m1.150s

Next steps I'm going to try adding some flamegraphs to the tracing and see where this peaking.

from mountpoint-s3.

Thanks for reporting this. In [1], we are introducing cache benchmarks. With this in place, we want to explore improvements by experimenting with changing the serialization library, for example.

[1] #783

from mountpoint-s3.

I should mention that the tracing changes I made are in my fork here https://github.com/raykrueger/mountpoint-s3/tree/flamegraphs

I had no intention of sending a PR for the changes as they were pretty heavy handed.

from mountpoint-s3.