high latency when saving snapshots about dragonfly HOT 16 OPEN

high CPU by itself is not a problem - it's a design choice. high latency is.
in any case we do not have enough data to comment on the issue, we would need to see continuous monitoring metrics from our latest version v1.13 and that's not something we can resolve effectively via github.

from dragonfly.

@liusf1993 are you able to share an RDB file with which this is reproducible? Or maybe reproduce this issue with a script or another mean?

from dragonfly.

I kicked off our weekly build now. Check this out:
https://github.com/dragonflydb/dragonfly/actions/runs/7472965174
should update https://github.com/dragonflydb/dragonfly/pkgs/container/dragonfly-weekly

from dragonfly.

Please ignore Roman's comment above, it was supposed to go to another issue :)

from dragonfly.

I need some time to complete the data desensitization, and I will provide the rdb file later

from dragonfly.

that would be MUCH appreciated, thank you!

from dragonfly.

@liusf1993 are you able to share an RDB file with which this is reproducible? Or maybe reproduce this issue with a script or another mean?
Dump file is here, the file server is in china, may be a litter slow.
https://snapshot2-tmp.oss-cn-shenzhen.aliyuncs.com/01-11/dragonfly_dfs.tgz?Expires=1704999873&OSSAccessKeyId=TMP.3Kjy1qQBa9rCRfN5yTeYecfWeKPS418arJkCiLWMvYDXDz8ySJHXvvuLALNRiBg2Z3W7adJfdxxhqznrHqVZfH6UBU1Bko&Signature=H6rmKerfFt1coVhRUE4n%2BqQhtVk%3D

from dragonfly.

thanks!
and how can I inspect whether the traffic is slower?
can you please reach out on Discord (I'm shahar5696) so we can discuss this faster? thanks!

from dragonfly.

I used a clumsy means to watch the result.
In shell,I executed for i in {1..100}; do time redis-cli -a [PASSWORD] dbsize ; sleep 2; done to see the execution time.
And result is like this
While bgsave is in progress

(integer) 51169005
real	0m0.039s
user	0m0.003s
sys	0m0.002s

After gbsave finished

(integer) 51169005
real	0m0.021s
user	0m0.002s
sys	0m0.002s

from dragonfly.

Discord

I don't have a Discord account, let me see how to register an account.

from dragonfly.

I used a clumsy means to watch the result. In shell,I executed for i in {1..100}; do time redis-cli -a [PASSWORD] dbsize ; sleep 2; done to see the execution time. And result is like this While bgsave is in progress

(integer) 51169005
real	0m0.039s
user	0m0.003s
sys	0m0.002s

After gbsave finished

(integer) 51169005
real	0m0.021s
user	0m0.002s
sys	0m0.002s

To add, the slow query was, many query within 1ms was changed to 4ms to 15ms while bgsave was in progress. The result was observed using a performance monitoring tool named skywalking.

from dragonfly.

@liusf1993 sorry I did not get it in time. is it possible to share it again?

from dragonfly.

@liusf1993 sorry I did not get it in time. is it possible to share it again?

My VPN is not very stable, and currently I am unable to connect to Discord, so can we communicate via email? My email is [email protected].

from dragonfly.

Use this permanent link https://snapshot2-tmp.oss-cn-shenzhen.aliyuncs.com/01-11/dragonfly_dfs.tgz, remind me to cancel sharing after completing the download

from dragonfly.

Coming from Redis v6.0, 200GB snapshot with default compression enabled takes around 20+ minutes with single cpu at 100%. Dragonfly on same db also with default compression enabled takes only 1.1 minute with high/100% cpu usage.

from dragonfly.

@liusf1993 thank you, I copied it 👍🏼

from dragonfly.