《Linux运维总结：使用新版redis |
您所在的位置：网站首页 › node使用redis › 《Linux运维总结：使用新版redis |
《Linux运维总结：使用新版redis
|
一、redis-shake简介
redis-shake是阿里云开源的用于Redis数据迁移和过滤的工具。 Github参考：redis-shake简介 1.1、迁移工具对比redis-migrate-tool redis-migrate-tool是唯品会开源的一款Redis异构集群之间的数据实时迁移工具，不过已经有两年没有更新了，我个人觉得这是一款比较完善的工具，特别是数据校验。 redis-shake RedisShake是阿里云基于豌豆荚开源的redis-port进行二次开发的一个支持Redis异构集群实时同步的工具，它和redis-migrate-tool相比较，我觉得它的优点在于支持前缀key的同步，支持多DB同步，而redis-migrate-tool 只能全量同步，并且如果源做了分库，同步到目标Redis的时候都同步到了db0一个库里面了，这对于做了分库场景的业务是不可行的。 redis-port redis-port是豌豆荚当年为了让大家方便从redis迁移到Codis开源的一个Redis数据迁移工具，现在也已经很久没更新了。 1.2、redis-shake版本对比 迁移工具版本类型版本说明支持redis版本支持模式redis-shakeredis-shake 3.x新版Redis 5.0, Redis 6.0 and Redis 7.0Supports Standalone, Cluster and some proxies type like Codis, twemproxy, Aliyun Cluster Proxy, Tencent Cloud Proxyredis-shakeredis-shake 2.x旧版Redis version from 2.x to 6.xstandalone sentinel cluster proxy 二、单机redis数据迁移《Linux运维总结：Centos7.6源码安装单实例redis6.2.8》 2.1、环境信息如下迁移环境信息如下： 主机IP操作系统Redis版本CPU架构端口角色192.168.1.191Centos7.66.2.8x86_647001源主机192.168.1.192Centos7.66.2.8x86_647001目标主机 2.2、安装redis-shake1、源端创建测试数据 [root@localhost ~]# for line in {1..10000};do /opt/redis6/bin/redis-cli -h 192.168.1.191 -p 7001 -c -a 1UEJjjGfYZU7dCWy set ops_${line} ${line}; done2、下载解压redis-shake文件 [root@localhost ~]# mkdir redis-shake && cd redis-shake [root@localhost redis-shake]# wget https://github.com/alibaba/RedisShake/releases/download/v3.1.7/redis-shake-linux-amd64.tar.gz [root@localhost redis-shake]# tar axf redis-shake-linux-amd64.tar.gz3、修改配置文件 [root@localhost redis-shake]# vim sync.toml type = "sync" [source] version = 6.2 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ... address = "192.168.1.191:7001" username = "" # keep empty if not using ACL password = "1UEJjjGfYZU7dCWy" # keep empty if no authentication is required tls = false elasticache_psync = "" # using when source is ElastiCache. ref: https://github.com/alibaba/RedisShake/issues/373 [target] type = "standalone" # "standalone" or "cluster" version = 6.2 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ... # When the target is a cluster, write the address of one of the nodes. # redis-shake will obtain other nodes through the `cluster nodes` command. address = "192.168.1.192:7001" username = "" # keep empty if not using ACL password = "1UEJjjGfYZU7dCWy" # keep empty if no authentication is required tls = false 2.3、redis-shake执行数据迁移1、执行下述命令启动redis-shake，redis-shake将开始执行数据迁移 [root@localhost redis-shake]# ./redis-shake sync.toml说明：当打印的日志出现send RDB finished，表示完成全量数据迁移，接下来进入增量数据迁移阶段，如下图所示：
2、暂停向源库写入数据，等待返回日志中allowOps对应值连续多次为0时，使用Ctrl+C组合键停止运行redis-shake，此时目标库的数据与源库完全一致，您可以将业务的数据库服务由自建Redis数据库切换至Tair或Redis实例。 2.4、校验迁移后的数据参考官网：校验迁移后的数据 说明：redis-full-check is used to compare whether two redis have the same data. redis-full-check用于比较2个redis数据是否一致，支持单节点、主从、集群版、以及多种proxy，支持同构以及异构对比，redis的版本支持2.x-5.x。当前redis版本为6.x，也是可以使用，因为测试的不是很全面，也有可能会在某些场景下存在问题。 [root@localhost ~]# wget https://github.com/alibaba/RedisFullCheck/releases/download/release-v1.4.8-20200212/redis-full-check-1.4.8.tar.gz [root@localhost ~]# tar axf redis-full-check-1.4.8.tar.gz && cd redis-full-check-1.4.8 [root@localhost redis-full-check-1.4.8]# ./redis-full-check -s "192.168.1.191:7001" -p 1UEJjjGfYZU7dCWy -t 192.168.1.192:7001 -a 1UEJjjGfYZU7dCWy --sourcedbtype=0 --targetdbtype=0 说明：执行完成后命令行输出校对结，如果此处为0，则两端数据一致 《Linux运维总结：Centos7.6部署redis6.2.8哨兵集群》 3.1、环境信息如下源端环境信息如下： 主机IP操作系统Redis版本CPU架构端口角色192.168.1.191Centos7.66.2.8x86_647001master192.168.1.191Centos7.66.2.8x86_647002slave192.168.1.191Centos7.66.2.8x86_647003slave192.168.1.191Centos7.66.2.8x86_647004sentinel192.168.1.191Centos7.66.2.8x86_647005sentinel192.168.1.191Centos7.66.2.8x86_647006sentinel目标端环境信息如下： 主机IP操作系统Redis版本CPU架构端口角色192.168.1.192Centos7.66.2.8x86_647001master192.168.1.192Centos7.66.2.8x86_647002slave192.168.1.192Centos7.66.2.8x86_647003slave192.168.1.192Centos7.66.2.8x86_647004sentinel192.168.1.192Centos7.66.2.8x86_647005sentinel192.168.1.192Centos7.66.2.8x86_647006sentinel 3.2、安装redis-shake1、源端创建测试数据 [root@localhost ~]# for line in {1..10};do /data/pkgs/redis/master/bin/redis-cli -h 192.168.1.191 -p 7001 -c -a 1UEJjjGfYZU7dCWy set ops_${line} ${line}; done2、下载解压redis-shake文件 [root@localhost ~]# mkdir redis-shake && cd redis-shake [root@localhost redis-shake]# wget https://github.com/alibaba/RedisShake/releases/download/v3.1.7/redis-shake-linux-amd64.tar.gz [root@localhost redis-shake]# tar axf redis-shake-linux-amd64.tar.gz3、修改配置文件 说明：standalone：标准（主从）架构、集群架构代理模式（Proxy）、读写分离架构。 [root@localhost redis-shake]# vim sync.toml type = "sync" [source] version = 6.2 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ... address = "192.168.1.191:7001" username = "" # keep empty if not using ACL password = "1UEJjjGfYZU7dCWy" # keep empty if no authentication is required tls = false elasticache_psync = "" # using when source is ElastiCache. ref: https://github.com/alibaba/RedisShake/issues/373 [target] type = "standalone" # "standalone" or "cluster" version = 6.2 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ... # When the target is a cluster, write the address of one of the nodes. # redis-shake will obtain other nodes through the `cluster nodes` command. address = "192.168.1.192:7001" username = "" # keep empty if not using ACL password = "1UEJjjGfYZU7dCWy" # keep empty if no authentication is required tls = false 3.3、redis-shake执行数据迁移1、在主节点上，执行下述命令启动redis-shake，redis-shake将开始执行数据迁移 [root@localhost redis-shake]# ./redis-shake sync.toml2、暂停向源库写入数据，等待返回日志中allowOps对应值连续多次为0时，使用Ctrl+C组合键停止运行redis-shake，此时目标库的数据与源库完全一致，您可以将业务的数据库服务由自建Redis数据库切换至Tair或Redis实例。 3.4、校验迁移后的数据参考官网：校验迁移后的数据 说明：redis-full-check is used to compare whether two redis have the same data. redis-full-check用于比较2个redis数据是否一致，支持单节点、主从、集群版、以及多种proxy，支持同构以及异构对比，redis的版本支持2.x-5.x。当前redis版本为6.x，也是可以使用，因为测试的不是很全面，也有可能会在某些场景下存在问题。 [root@localhost ~]# wget https://github.com/alibaba/RedisFullCheck/releases/download/release-v1.4.8-20200212/redis-full-check-1.4.8.tar.gz [root@localhost ~]# tar axf redis-full-check-1.4.8.tar.gz && cd redis-full-check-1.4.8 [root@localhost redis-full-check-1.4.8]# ./redis-full-check -s "192.168.1.191:7001" -p 1UEJjjGfYZU7dCWy -t "192.168.1.192:7001" -a 1UEJjjGfYZU7dCWy --sourcedbtype=0 --targetdbtype=0说明：redis-full-check参数如下所示：
说明：执行完成后命令行输出校对结，如果此处为0，则两端数据一致
《Linux运维总结：Centos7.6部署redis6.2.8 cluster集群》 4.1、环境信息如下源端环境信息如下： 主机IP操作系统Redis版本CPU架构端口角色192.168.1.191Centos7.66.2.8x86_647001master192.168.1.191Centos7.66.2.8x86_647002master192.168.1.191Centos7.66.2.8x86_647003master192.168.1.191Centos7.66.2.8x86_647004slave192.168.1.191Centos7.66.2.8x86_647005slave192.168.1.191Centos7.66.2.8x86_647006slave目标端环境信息如下： 主机IP操作系统Redis版本CPU架构端口角色192.168.1.192Centos7.66.2.8x86_647001master192.168.1.192Centos7.66.2.8x86_647002master192.168.1.192Centos7.66.2.8x86_647003master192.168.1.192Centos7.66.2.8x86_647004slave192.168.1.192Centos7.66.2.8x86_647005slave192.168.1.192Centos7.66.2.8x86_647006slave 4.2、安装redis-shake1、源端创建测试数据 [root@localhost ~]# for line in {1..1000};do /data/pkgs/redis/7001/bin/redis-cli -h 192.168.1.191 -p 7001 -c -a 1UEJjjGfYZU7dCWy set ops_${line} ${line}; done2、分别在三个主节点上下载解压redis-shake文件 [root@localhost ~]# mkdir redis-shake1 && cd redis-shake1 [root@localhost redis-shake1]# wget https://github.com/alibaba/RedisShake/releases/download/v3.1.7/redis-shake-linux-amd64.tar.gz [root@localhost redis-shake1]# tar axf redis-shake-linux-amd64.tar.gz [root@localhost ~]# mkdir redis-shake2 && cd redis-shake2 [root@localhost redis-shake2]# wget https://github.com/alibaba/RedisShake/releases/download/v3.1.7/redis-shake-linux-amd64.tar.gz [root@localhost redis-shake2]# tar axf redis-shake-linux-amd64.tar.gz [root@localhost ~]# mkdir redis-shake1 && cd redis-shake3 [root@localhost redis-shake3]# wget https://github.com/alibaba/RedisShake/releases/download/v3.1.7/redis-shake-linux-amd64.tar.gz [root@localhost redis-shake3]# tar axf redis-shake-linux-amd64.tar.gz3、修改配置文件 说明：如果源库为集群架构，需要为N个节点启动N个redis-shake工具，源端分别为各节点的连接地址，目标端为相同目标实例地址。源库集群架构为三主三从，所以分别在三个主节点上分别启动redis-shake工具即可，目标端为三个主节点地址即可。 主节点一（192.168.1.191:7001） [root@localhost redis-shake1]# vim sync.toml type = "sync" [source] version = 6.2 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ... address = "192.168.1.191:7001" username = "" # keep empty if not using ACL password = "1UEJjjGfYZU7dCWy" # keep empty if no authentication is required tls = false elasticache_psync = "" # using when source is ElastiCache. ref: https://github.com/alibaba/RedisShake/issues/373 [target] type = "cluster" # "standalone" or "cluster" version = 6.2 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ... # When the target is a cluster, write the address of one of the nodes. # redis-shake will obtain other nodes through the `cluster nodes` command. address = "192.168.1.192:7001" username = "" # keep empty if not using ACL password = "1UEJjjGfYZU7dCWy" # keep empty if no authentication is required tls = false主节点二（192.168.1.191:7002） [root@localhost redis-shake2]# vim sync.toml type = "sync" [source] version = 6.2 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ... address = "192.168.1.191:7002" username = "" # keep empty if not using ACL password = "1UEJjjGfYZU7dCWy" # keep empty if no authentication is required tls = false elasticache_psync = "" # using when source is ElastiCache. ref: https://github.com/alibaba/RedisShake/issues/373 [target] type = "cluster" # "standalone" or "cluster" version = 6.2 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ... # When the target is a cluster, write the address of one of the nodes. # redis-shake will obtain other nodes through the `cluster nodes` command. address = "192.168.1.192:7002" username = "" # keep empty if not using ACL password = "1UEJjjGfYZU7dCWy" # keep empty if no authentication is required tls = false主节点三（192.168.1.191:7003） [root@localhost redis-shake3]# vim sync.toml type = "sync" [source] version = 6.2 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ... address = "192.168.1.191:7003" username = "" # keep empty if not using ACL password = "1UEJjjGfYZU7dCWy" # keep empty if no authentication is required tls = false elasticache_psync = "" # using when source is ElastiCache. ref: https://github.com/alibaba/RedisShake/issues/373 [target] type = "cluster" # "standalone" or "cluster" version = 6.2 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ... # When the target is a cluster, write the address of one of the nodes. # redis-shake will obtain other nodes through the `cluster nodes` command. address = "192.168.1.192:7003" username = "" # keep empty if not using ACL password = "1UEJjjGfYZU7dCWy" # keep empty if no authentication is required tls = false 4.3、redis-shake执行数据迁移1、分别在三个主节点上，执行下述命令启动redis-shake，redis-shake将开始执行数据迁移 [root@localhost redis-shake1]# ./redis-shake sync.toml [root@localhost redis-shake2]# ./redis-shake sync.toml [root@localhost redis-shake3]# ./redis-shake sync.toml2、暂停向源库写入数据，等待返回日志中allowOps对应值连续多次为0时，使用Ctrl+C组合键停止运行redis-shake，此时目标库的数据与源库完全一致，您可以将业务的数据库服务由自建Redis数据库切换至Tair或Redis实例。 4.4、校验迁移后的数据参考官网：校验迁移后的数据 说明：redis-full-check is used to compare whether two redis have the same data. redis-full-check用于比较2个redis数据是否一致，支持单节点、主从、集群版、以及多种proxy，支持同构以及异构对比，redis的版本支持2.x-5.x。当前redis版本为6.x，也是可以使用，因为测试的不是很全面，也有可能会在某些场景下存在问题。 [root@localhost ~]# wget https://github.com/alibaba/RedisFullCheck/releases/download/release-v1.4.8-20200212/redis-full-check-1.4.8.tar.gz [root@localhost ~]# tar axf redis-full-check-1.4.8.tar.gz && cd redis-full-check-1.4.8 [root@localhost redis-full-check-1.4.8]# ./redis-full-check -s "192.168.1.191:7001;192.168.1.191:7002;192.168.1.191:7003" -p 1UEJjjGfYZU7dCWy \ -t "192.168.1.192:7001;192.168.1.192:7002;192.168.1.192:7003" -a 1UEJjjGfYZU7dCWy \ --sourcedbtype=1 --targetdbtype=1说明：-s表示源端Redis的连接地址和端口，如果源Redis为集群版，集群中每个地址间需要以半角分号（;）分割不同的连接地址，-s后必须接三个主节点地址或者三个从节点地址，不能把三个主节点和三个从节点全部写上。-t表示目的端Redis的连接地址和端口，如果目的Redis为集群版，集群中每个地址间需要以半角分号（;）分割不同的连接地址，-t后必须接三个主节点地址或者三个从节点地址，不能把三个主节点和三个从节点全部写上。 说明：redis-full-check参数如下所示：
说明：执行完成后命令行输出校对结，如果此处为0，则两端数据一致
更多详细内容请参考：《Linux运维篇：Linux系统运维指南》 |
【本文地址】
今日新闻 |
推荐新闻 |