When benchmarking x2gd.xlarge ssd disk with nominal r

randwrite benchmark on the same instance: <div class="snippet-clipboard-content no

Allow asynchronous disk I/O processing in each shard thread about dragonfly HOT 2 OPEN

romange commented on June 1, 2024

Allow asynchronous disk I/O processing in each shard thread

from dragonfly.

Comments (2)

romange commented on June 1, 2024

randwrite benchmark on the same instance:

> fio --name=fiotest --ioengine=io_uring --iodepth=4 --rw=randwrite --bs=4k --direct=1 --size=4G --numjobs=4 --runtime=20 --group_reporting`

write: IOPS=24.3k, BW=95.0MiB/s (99.6MB/s)(1900MiB/20001msec); 0 zone resets
lat (usec): min=29, max=32451, avg=656.80, stdev=577.20
lat (usec)   : 250=0.44%, 500=5.96%, 750=84.52%, 1000=0.41%

writes are at least twice slower on this instance family.
with iodepth=2 we get better results:

lat (usec): min=25, max=13598, avg=327.89, stdev=103.96 
lat (usec)   : 100=1.64%, 250=0.08%, 500=91.19%, 750=0.01%, 1000=0.01%

need to read what it actually means iodepth

from dragonfly.

romange commented on June 1, 2024

iodepth means how many requests are issued by the app to disk at the same time. with iodepth=1, writes are:

write: IOPS=23.8k, BW=93.0MiB/s (97.5MB/s)(1861MiB/20001msec); 0 zone resets
lat (usec): min=23, max=26881, avg=166.58, stdev=324.45
lat (usec)   : 100=0.27%, 250=92.06%, 500=0.22%, 750=0.01%, 1000=0.01%

which reaches the nominal capacity of the instance of 22.5K
but reads are:

read: IOPS=37.9k, BW=148MiB/s (155MB/s)(2957MiB/20001msec)
lat (usec): min=65, max=1868, avg=104.58, stdev=35.45
lat (usec)   : 20=0.01%, 100=70.14%, 250=28.97%, 500=0.87%, 750=0.01%

and they do not reach the nominal capacity of the instance.
Another important observation - read and write budgets are independently enforced by cloud provider. So our total budget for this instance is 23K write IOPS + 53K read IOPS.

Verified by the following fio file:

[global]                                                                                                                                                                                                                                    
ioengine=io_uring                                                                                                                                                                                                                           
group_reporting
direct=1                                                                                                                                                                                                                                    
size=4G                                                                                                                                                                                                                                     
runtime=20s                                                                                                                                                                                                                                 
directory=/var/lib/dragonfly/fio                                                                                                                                                                                                            
iodepth=1                                                                                                                                                                                                                                   
numjobs=4                                                                                                                                                                                                                                   
bs=4k                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                            
[randread]                                                                                                                                                                                                                                  
rw=randread                                                                                                                                                                                                                                 
                                                                                                                                                                                                                                            
[randwrite]                                                                                                                                                                                                                                 
rw=randwrite

from dragonfly.

Recommend Projects