Refactor the data migration code about kvrocks HOT 6 OPEN

Hi, i'm interested in this issue, could you please assign this to me?

from kvrocks.

@wsehjk Assigned. Go ahead :D

Based on my experience, you can share your opinion on this issue and how you'd like to implement it. If it's trivial, directly sending a patch is good.

Barely asking for an assignment without any thoughts or progress attempts often ends with giving up.

from kvrocks.

@caipengbo Would you mind giving some input about what you plan to improve?

from kvrocks.

@git-hulk This is my personal opinion for reference only:

1. decouple the iteration logic from the send logic:
   • use DBIterator and WALIterator for data iteration
   • use a Sender helper class for sending data and rate limit
2. Make the two migration types more uniform
3. Refactor Golang unit tests about data migration to make them more concise and faster to run
4. Other minor change:
   • MigrationJob stores what is really needed
   • ...

As @tisonkun said, you can also share some of your plans and we can discuss them together @wsehjk .

from kvrocks.

@caipengbo Thanks for your input.

from kvrocks.

@git-hulk This is my personal opinion for reference only:

1. decouple the iteration logic from the send logic:

   • use DBIterator and WALIterator for data iteration
   • use a Sender helper class for sending data and rate limit

2. Make the two migration types more uniform

3. Refactor Golang unit tests about data migration to make them more concise and faster to run

4. Other minor change:

   • MigrationJob stores what is really needed
   • ...

As @tisonkun said, you can also share some of your plans and we can discuss them together @wsehjk .

Thank you for the info, really helpful, and i will walk through the code and share my thought before developing

from kvrocks.