SEARCH_V2 buffer is too big, slows compsize down about compsize HOT 8 OPEN

The first part, reducing buffer size, works even far more impressively on my test filesystem. That's pure awesomesity!

On the other hand, using max_object beyond the st_ino, makes it slower by ~11%, on the same filesystem.

from compsize.

Branch with both changes applied: https://github.com/kilobyte/compsize/tree/loop_retry

from compsize.

On the other hand, using max_object beyond the st_ino, makes it slower by ~11%, on the same filesystem.

That's interesting. Any idea why?

Hmmm...maybe the other max_ fields should be set differently. It will be fetching two inodes' complete extent maps. Maybe set max_type to BTRFS_INODE_ITEM, so it stops when it reaches the next inode, but doesn't dump out its extent list?

from compsize.

That would be:

min = (st_ino, BTRFS_EXTENT_DATA_KEY, 0),
max = (st_ino + 1, BTRFS_INODE_ITEM_KEY, 0)

That should make the search stop as soon as it finds the next inode, without including the inode's extents (or directory entries, if the next inode is a directory). At the end of the subvol it will have to do another 0-item loop, but usually there are a lot more inodes than subvols.

from compsize.

This helped, but is still slower than continuing if previous run had >512.

from compsize.

Guys this is super cool! compsize run on one sample directory on my system is just 0.15s with loop_retry branch while it takes 5.5s with version from package manager. loop_retry branch allowed me to calculate which directories are actually take space!

Why is this not merged already? I suspect you'd like to get it perfect before merging. Is it possible to make release with main improvement and ponder about fine tuning afterwards?

from compsize.

@Self-Perfection: what version are you comparing with? 1,5 should have the buffer size speedup.

If the alternate retry condition makes such a large difference that's good to hear — but, I suspect your distro release has a version older than 1.5. Care to check?

from compsize.

@kilobyte you are right. My distro has older compsize version. Thank you for pointing this out.

from compsize.