Comments (10)
hi @smaznet
I create a commit that add option -g
to rdt
command
now you can convert db as following step
step 1 rdt -b redis://127.0.0.1:6379 -o ./dump-2.rdb -d 31 -g 2
step 2 rmt -s ./dump-2.rdb -m redis://127.0.0.1:6380 -r
but this feature not release yet. you can build a local release base on doc by yourself.
from redis-rdb-cli.
Also i tried to set db as connection url but it doesnt work
redis://127.0.0.1:6379/5
and
redis://127.0.0.1:6379/?db=5
from redis-rdb-cli.
We can't use url like redis://127.0.0.1:6379/?db=5
or redis://127.0.0.1:6379/5
.
rmt
now can filter souce db using rmt -s redis://127.0.0.1:6379 -m redis://127.0.0.1:6380 -d 0 1 2 -r
if we add above way to set dest db. there are several scenarios to consider.
let set dest db use -x
option.
rmt -s redis://127.0.0.1:6379 -m redis://127.0.0.1:6380 -x 3
means migrate all 6379
data to 6380
db 3
-
if every
6379
db has a same keysame key
withstring
type, but value different likevalue1
,value2
,value3
the question is after migration what is thesame key
's value in dest6380
db 3 -
if that key is not
string
type butlist
type. then in this scenario what is the value in6380
-
if
rmt
does't specify--replace
option. and dest db 3 also has a keysame key
. what is the final value in dest db 3
we can implement this feature after we covered above scenarios.
we can add an option in rdt
command instead of rmt
command
rdt -b redis://127.0.0.1:6379 -o dump-3.rdb -x 3
means backup data from 6379
and save the file dump-3.rdb
to make sure saved data's db is 3.
after that user can use rmt -s dump-3.rdb -m redis://127.0.0.1:6380 -r
to migrate data to dest db 3
from redis-rdb-cli.
let me explain why i need that feature
i created many projects on 1 server and i increased max db count from 16 (which is default in redis)
now i want to move some projects to other server in i dont want modify redis settings
when i tried
rmt -s ./dump.rdb -m redis://127.0.0.1:6380 -d 31
and
rmt -s ./dump.rdb -m redis://127.0.0.1:6380 -d 21
all keys moved to db 0 (because i dont changed redis.conf to increase db counts and i dont want do that)
now i want move from db 21 to db 1 and from 31 to db 2
from redis-rdb-cli.
We can't use url like
redis://127.0.0.1:6379/?db=5
orredis://127.0.0.1:6379/5
.
rmt
now can filter souce db usingrmt -s redis://127.0.0.1:6379 -m redis://127.0.0.1:6380 -d 0 1 2 -r
if we add above way to set dest db. there are several scenarios to consider.
let set dest db use-x
option.
rmt -s redis://127.0.0.1:6379 -m redis://127.0.0.1:6380 -x 3
means migrate all6379
data to6380
db 3
- if every
6379
db has a same keysame key
withstring
type, but value different likevalue1
,value2
,value3
the question is after migration what is thesame key
's value in dest6380
db 3- if that key is not
string type but
listtype. then in this scenario what is the value in
6380`- if
rmt
does't specify--replace
option. and dest db 3 also has a keysame key
. what is the final value in dest db 3we can implement this feature after we covered above scenarios.
we can add an option in
rdt
command instead ofrmt
command
rdt -b redis://127.0.0.1:6379 -o dump-3.rdb -x 3
means backup data from6379
and save the filedump-3.rdb
to make sure saved data's db is 3.
after that user can usermt -s dump-3.rdb -m redis://127.0.0.1:6380 -r
to migrate data to dest db 3
Yes also adding this option to rdt is good idea
from redis-rdb-cli.
following are examples about above scenarios we discussed before.
scenario 1
127.0.0.1:6379>select 0
127.0.0.1:6379>set string_key value1
127.0.0.1:6379>select 1
127.0.0.1:6379>set string_key value2
127.0.0.1:6379>select 3
127.0.0.1:6379>set string_key value3
$ rdt -b redis://127.0.0.1:6379 -o ./dump.rdb -g 5
# string override by db3
$ rmt -s ./dump.rdb -m redis://127.0.0.1:6380 -r
127.0.0.1:6380>select 5
127.0.0.1:6380>get string_key
"value3"
scenario 2
127.0.0.1:6379>select 0
127.0.0.1:6379>lpush list_key value1
127.0.0.1:6379>select 1
127.0.0.1:6379>lpush list_key value2
127.0.0.1:6379>select 3
127.0.0.1:6379>lpush list_key value3
$ rdt -b redis://127.0.0.1:6379 -o ./dump.rdb -g 5
# list override by db3
$ rmt -s ./dump.rdb -m redis://127.0.0.1:6380 -r
127.0.0.1:6380>select 5
127.0.0.1:6380>lrange list_key 0 -1
1) "value3"
scenario 3
127.0.0.1:6379>select 0
127.0.0.1:6379>set string_key value1
127.0.0.1:6379>select 1
127.0.0.1:6379>set string_key value2
127.0.0.1:6379>select 3
127.0.0.1:6379>set string_key value3
127.0.0.1:6380>select 5
127.0.0.1:6380>set string_key value5
$ rdt -b redis://127.0.0.1:6379 -o ./dump.rdb -g 5
# without -r option
$ rmt -s ./dump.rdb -m redis://127.0.0.1:6380
127.0.0.1:6380>select 5
127.0.0.1:6380>get string_key
"value5"
# with -r option
$ rmt -s ./dump.rdb -m redis://127.0.0.1:6380 -r
127.0.0.1:6380>select 5
127.0.0.1:6380>get string_key
"value3"
from redis-rdb-cli.
Yes you right
but what about mapping 🤔
for example
specify -d like this
-d 31:0 -d 20:1 -d 17:2 -d 3
which will move
all db 31 keys to db 0
all db 20 keys to db 1
all db 17 keys to db 2
all db 3 keys to db 3
from redis-rdb-cli.
@smaznet
db mapping is a good idea. but not only rdt
has -d
option, but also rct
, rmt
, rst
has -d
option.
we want to provide a consistent command options to users. and db mapping could broke this consistence.
I suggest write a shell to handle db mapping and migration. pseudo code may following
# backup remote redis rdb only once
rdt -b redis://${source_host}:${source_port} -o ./dump.rdb
for (mapping) {
# filter
rdt -b ./dump.rdb -o ./dump-{mapping.target}.rdb -d ${mapping.source} -g ${mapping.target}
# migration
rmt -s ./dump-${mapping.target}.rdb -m redis://${target_host}:${target_port} -r
}
from redis-rdb-cli.
It's also useful for rmt and rct and rct
from redis-rdb-cli.
I'm not sure for now how other command can add db mapping feature. especially rst
is hard to add this feature.
so for now we only add -g
option in rdt
command. If I surely clear how implement this feature in all command. then I'll consider to add db mapping in that time.
from redis-rdb-cli.
Related Issues (20)
- Invalid parsing result after upgrade to redis 6.0.9 HOT 6
- 在prometheus 可以新增监控bigkey ? HOT 3
- merge two rdb files with repeated key HOT 2
- Problem installing HOT 1
- password can't have "%" HOT 1
- NOAUTH despite authPassword and/or ACL style on Redis 6.2.6 HOT 4
- Missing required option: '--source <source>' HOT 3
- Redis remote backup import failing HOT 19
- Feature Request: rmt sync from cluster to single redis HOT 1
- java.lang.RuntimeException: java.lang.NumberFormatException: null HOT 6
- expiry is not right HOT 2
- More information about applications HOT 4
- Sync data between KeyDB v6 - Redis v7 HOT 26
- Terminating due to java.lang.OutOfMemoryError: Java heap space HOT 7
- Check migrated data consistency HOT 5
- Bloom keys not synced to target redis instance HOT 1
- rst's password maybe can't contain # HOT 2
- Adding any -k parameter returns an empty list HOT 4
- Publish to maven central HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from redis-rdb-cli.