Redis-shake 数据迁移工具

Redis-shake 数据迁移工具

redis-shake 是阿里云开源的用于 Redis 数据迁移和过滤的工具。

redis-shake 支持特性：

• ⚡高性能
• ✅在 Redis 5.0、Redis 6.0 和 Redis 7.0 上测试
• 🤗支持使用lua自定义过滤规则
• 💪支持大实例迁移
• 💖支持restore模式和sync模式
• ☁️支持阿里云 Redis 和 ElastiCache
  

项目地址：https://github.com/alibaba/RedisShake

Redis-shake 快速体验

1、下载安装redis-shake

wget https://github.com/alibaba/RedisShake/releases/download/v3.0.0/redis-shake-v3.0.0.tar.gz
mv redis-shake-linux-amd64 /usr/local/bin/redis-shake
1
2

2、使用容器部署两个Redis测试实例

docker run -d --name redis-4.0.14 -p 6379:6379 redis:4.0.14
docker run -d --name redis-6.0.16 -p 6380:6379 redis:6.0.16
1
2

3、源端创建测试数据

[root@redis ~]# docker exec -it redis-4.0.14 redis-cli -h localhost SET key1 value1
OK
1
2

4、创建迁移配置文件

完整示例配置文件参考：https://github.com/alibaba/RedisShake/blob/v3/redis-shake.toml

cat >redis-shake.toml<<EOF
[source]
type = "sync"
address = "localhos:6379"

[target]
type = "standalone"
address = "localhos:6380"
EOF
1
2
3
4
5
6
7
8
9

带有用户名密码认证信息的示例配置文件：

[source]
type = "sync"
address = "r-aaaaa.redis.zhangbei.rds.aliyuncs.com:6379"
password = "r-aaaaa:xxxxx"

[target]
type = "standalone"
address = "r-bbbbb.redis.zhangbei.rds.aliyuncs.com:6379"
password = "r-bbbbb:xxxxx"
1
2
3
4
5
6
7
8
9

目标端是redis-cluster的示例配置文件

[source]
type = "sync"
address = "r-aaaaa.redis.zhangbei.rds.aliyuncs.com:6379"
password = "r-aaaaa:xxxxx"

[target]
type = "cluster"
address = "192.168.0.1:6379" # 这里写集群中的任意一个节点的地址即可
password = "r-ccccc:xxxxx"
1
2
3
4
5
6
7
8
9

5、执行redis数据库迁移

[root@redis ~]# redis-shake ./redis-shake.toml
2022-08-26 11:20:28 INF GOOS: linux, GOARCH: amd64
2022-08-26 11:20:28 INF Ncpu: 3, GOMAXPROCS: 3
2022-08-26 11:20:28 INF pid: 21504
2022-08-26 11:20:28 INF pprof_port: 0
2022-08-26 11:20:28 INF No lua file specified, will not filter any cmd.
2022-08-26 11:20:28 INF no password. address=[127.0.0.1:6380]
2022-08-26 11:20:28 INF redisWriter connected to redis successful. address=[127.0.0.1:6380]
2022-08-26 11:20:28 INF no password. address=[127.0.0.1:6379]
2022-08-26 11:20:28 INF psyncReader connected to redis successful. address=[127.0.0.1:6379]
2022-08-26 11:20:28 WRN remove file. filename=[4200.aof]
2022-08-26 11:20:28 WRN remove file. filename=[dump.rdb]
2022-08-26 11:20:28 INF start save RDB. address=[127.0.0.1:6379]
2022-08-26 11:20:28 INF send [replconf listening-port 10007]
2022-08-26 11:20:28 INF send [PSYNC ? -1]
2022-08-26 11:20:28 INF receive [FULLRESYNC 1db7c7618b6d0af25ffafb1645d4fba573624d02 0]
2022-08-26 11:20:28 INF source db is doing bgsave. address=[127.0.0.1:6379]
2022-08-26 11:20:28 INF source db bgsave finished. timeUsed=[0.09]s, address=[127.0.0.1:6379]
2022-08-26 11:20:28 INF received rdb length. length=[194]
2022-08-26 11:20:28 INF create dump.rdb file. filename_path=[dump.rdb]
2022-08-26 11:20:28 INF save RDB finished. address=[127.0.0.1:6379], total_bytes=[194]
2022-08-26 11:20:28 INF start send RDB. address=[127.0.0.1:6379]
2022-08-26 11:20:28 INF RDB version: 8
2022-08-26 11:20:28 INF RDB AUX fields. key=[redis-ver], value=[4.0.14]
2022-08-26 11:20:28 INF RDB AUX fields. key=[redis-bits], value=[64]
2022-08-26 11:20:28 INF RDB AUX fields. key=[ctime], value=[1661484028]
2022-08-26 11:20:28 INF RDB AUX fields. key=[used-mem], value=[1897096]
2022-08-26 11:20:28 INF RDB repl-stream-db: 0
2022-08-26 11:20:28 INF RDB AUX fields. key=[repl-id], value=[1db7c7618b6d0af25ffafb1645d4fba573624d02]
2022-08-26 11:20:28 INF RDB AUX fields. key=[repl-offset], value=[0]
2022-08-26 11:20:28 INF RDB AUX fields. key=[aof-preamble], value=[0]
2022-08-26 11:20:28 INF RDB resize db. db_size=[1], expire_size=[0]
2022-08-26 11:20:28 INF send RDB finished. address=[127.0.0.1:6379], repl-stream-db=[0]
2022-08-26 11:20:28 INF start save AOF. address=[127.0.0.1:6379]
2022-08-26 11:20:28 INF AOFWriter open file. filename=[0.aof]
2022-08-26 11:20:29 INF AOFReader open file. aof_filename=[0.aof]
2022-08-26 11:20:33 INF syncing aof. allowOps=[0.20], disallowOps=[0.00], entryId=[0], unansweredBytesCount=[0]bytes, diff=[0], aofReceivedOffset=[0], aofAppliedOffset=[0]
2022-08-26 11:20:38 INF syncing aof. allowOps=[0.20], disallowOps=[0.00], entryId=[1], unansweredBytesCount=[0]bytes, diff=[0], aofReceivedOffset=[14], aofAppliedOffset=[14]
2022-08-26 11:20:43 INF syncing aof. allowOps=[0.00], disallowOps=[0.00], entryId=[1], unansweredBytesCount=[0]bytes, diff=[0], aofReceivedOffset=[14], aofAppliedOffset=[14]
2022-08-26 11:20:48 INF syncing aof. allowOps=[0.20], disallowOps=[0.00], entryId=[2], unansweredBytesCount=[0]bytes, diff=[0], aofReceivedOffset=[28], aofAppliedOffset=[28]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

6、验证迁移结果

[root@redis ~]# docker exec -it redis-6.0.16 redis-cli -h localhost GET key1
"value1"
1
2