Redis Sentinel 搭建指南：从入门到精通

记录 Redis Sentinel 的搭建过程

现有架构的问题

  1. master 挂掉之后, 需要手动切换, 运维复杂

Redis Sentinel 高可用

20241229154732_8Zk7xKho.webp

下面以 1 个主节点、2 个从节点、3 个 Sentinel 节点组成的 Redis Sentinel 为例子

故障转移处理逻辑:

  1. 主节点出现故障, 此时两个从节点与主节点时区连接, 主从复制失败;

    20241229154732_HwvXcbfZ.webp

  2. 每个 Sentinel 节点通过定期监控发现主节点出现故障;

    20241229154732_aQBkbupn.webp

  3. 多个 Sentinel 节点对主节点的故障达成一致, 选举出 sentinel-3 节点作为领导者负责故障转移;

    20241229154732_s7YZQex5.webp

    1. 原来的从节点 slave-1 称为新的主节点后, 更新应用方的主节点信息, 重新启动应用方;
    2. 客户端命令另一个从节点 slave-2 去复制性的主节点;
    3. 待原来的主节点恢复后, 让它去复制新的主节点;

    20241229154732_FUe3W94T.webp

  4. 故障转移后的结构图

    20241229154732_znOFqeq4.webp

    Redis Sentinel 功能

    1. 监控: Sentinel 节点会定期检测 Redis 数据节点和其余 Sentinel 节点是否可达;
    2. 通知: Sentinel 节点会将故障转移的结果通知给应用方;
    3. 主节点故障转移: 实现从节点晋升为主节点并维护后续正确的主从关系;
    4. 配置提供者: 客户端在初始化时, 连接 Sentinel 节点集群, 从中获取主节点信息;

Redis Sentinel 安装与部署

下面将以 3 个 Sentinel 节点、1 个主节点、2 个从节点组成一个 Redis Sentinel 进行说明

20241229154732_2SRjp4lI.webp

具体的物理部署:

角色 ip port 别名
master 127.0.0.1 6379 主节点
slave-1 127.0.0.1 6380 slave-1
slave-2 127.0.0.1 6381 slave-2
sentinel-1 127.0.0.1 26379 sentinel-1
sentinel-2 127.0.0.1 26380 sentinel-2
sentinel-3 127.0.0.1 26381 sentinel-3

1. master 配置

1
2
3
4
5
6
daemonize yes
dbfilename "6379.db"
dir "/Users/codeai/Develop/logs/redis/db/"
logfile "/Users/codeai/Develop/logs/redis/log/6379.log"
port 6379
requirepass 1234

2. slave-1 配置

1
2
3
4
5
6
7
8
9
10
daemonize yes
dbfilename "6380.db"
dir "/Users/codeai/Develop/logs/redis/db/"
logfile "/Users/codeai/Develop/logs/redis/log/6380.log"
port 6380
slaveof 127.0.0.1 6379
# 设置 master 验证密码
masterauth 1234
# 设置 slave 密码
requirepass 1234

3. slave-2 配置

1
2
3
4
5
6
7
8
9
10
daemonize yes
dbfilename "6381.db"
dir "/Users/codeai/Develop/logs/redis/db/"
logfile "/Users/codeai/Develop/logs/redis/log/6381.log"
port 6381
slaveof 127.0.0.1 6379
# 设置 master 验证密码
masterauth 1234
# 设置 slave 密码
requirepass 1234

4. 启动 redis 服务

1
redis-server redis-6379.conf; redis-server redis-6380.conf; redis-server redis-6381.conf

20241229154732_gq307pmS.webp

5. 确认主从关系

主节点视角

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
redis-cli -h 127.0.0.1 -p 6379 info replication
# Replication
# 主节点
role:master
# 有 2 个 从节点
connected_slaves:2
# 从节点 1 信息
slave0:ip=127.0.0.1,port=6380,state=online,offset=112,lag=0
# 从节点 2 信息
slave1:ip=127.0.0.1,port=6381,state=online,offset=112,lag=0
master_replid:8f43dc48cca779f46fa1516a38e24fb8c5423d94
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:112
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:112

从节点视角

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
redis-cli -h 127.0.0.1 -p 6380 info replication
# Replication
# 从节点
role:slave
# 主节点信息
master_host:127.0.0.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:2
master_sync_in_progress:0
slave_repl_offset:238
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:8f43dc48cca779f46fa1516a38e24fb8c5423d94
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:238
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:238

6. 部署 Sentinel 节点

1
2
3
4
5
6
7
8
9
10
11
12
13
14
port 26379
daemonize yes
logfile "/Users/codeai/Develop/logs/redis/log/26379.log"
dir "/Users/codeai/Develop/logs/redis/db/"
# 监控 127.0.0.1:6379 主节点; 2 表示判断主节点失败至少需要 2 个 sentinel 节点同意
sentinel monitor mymaster 127.0.0.1 6379 2
# 设置监听的 master 密码
sentinel auth-pass mymaster 1234
# 30 秒内 ping 失败, sentinel 则认为 master 不可用
sentinel down-after-milliseconds mymaster 30000
# 在发生 failover 主备切换时，这个选项指定了最多可以有多少个 slave 同时对新的 master 进行同步
sentinel parallel-syncs mymaster 1
# 如果在该时间（ms）内未能完成 failover 操作，则认为该 failover 失败
sentinel failover-timeout mymaster 180000

其他节点只是端口不同

7. 启动 sentinel 节点

1
2
3
4
# 方式一
redis-sentinel redis-sentinel-26379.conf; redis-sentinel redis-sentinel-26380.conf; redis-sentinel redis-sentinel-26381.conf
# 方式二
redis-server redis-sentinel-26379.conf --sentinel; redis-server redis-sentinel-26380.conf --sentinel; redis-server redis-sentinel-26381.conf --sentinel;

20241229154732_5ay2F1RC.webp

8. 确认关系

1
2
3
4
5
6
7
8
redis-cli -h 127.0.0.1 -p 26379 info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=127.0.0.1:6379,slaves=2,sentinels=3

通过 Jedis 操作 Redis

https://github.com/dong4j/redis-toolkit

包含 redis 配置文件, 搭建方式与单元测试

Jedis 操作 Redis 的 三种方式

单机模式

直接通过 JedisPool 操作 Redis

1
2
3
4
5
6
7
8
9
10
11
12
13
14
Jedis jedis = null;
try {
    // 从连接池获取一个 Jedis 实例
    jedis = jedisPool.getResource();
    jedis.set(key, value);
    log.info(jedis.get(key));
} catch (Exception e) {
    log.error("set error", e);
} finally {
    if (null != jedis) {
        // 释放资源还给连接池
        jedis.close();
    }
}
1
2
3
4
5
6
try(Jedis jedis = jedisPool.getResource()) {
    jedis.set(key, value);
    log.info(jedis.get(key));
} catch (Exception e) {
    log.error("set error", e);
}

分片模式（ShardedJedis）

使用一致性哈希算法, 将 key 存储在对应实例中

1
2
3
4
5
6
try(ShardedJedis jedis = shardedJedisPool.getResource();) {
    jedis.set(key, value);
    log.info(jedis.get(key));
} catch (Exception e) {
    log.error("set error", e);
}

集群模式（BinaryJedisCluster）

需要 Redis 3.0 以上才自带集群功能

集成 Jedis 的两种方式

  1. 使用 spring-data-redis 集成 redis, 使用 RedisTemplate 或者 JedisConnectionFactory 获取 jedis 操作 Redis
  2. 直接使用 JedisPool , ShardedJedisPool, ShardedJedisPool, ShardedJedisSentinelPool 获取 jedis 操作 Redis

spring-data-redis

原生 jedis

JedisPool
ShardedJedisPool

高可用的 Redis

JedisSentinelPool
ShardedJedisSentinelPool

代码详见: https://github.com/dong4j/redis-toolkit

问题

1
All sentinels down, cannot determine where is mymaster master is running...

sentinel 安全模式默认是打开的, 又因为没有绑定可以访问的 ip 和设置访问密码，就不允许从外部访问;
redis 内部把 127 的地址转换成了 192.168.2.101 了，也就是你的本机的 ip 地址。所以访问 192 的地址就相当于从外部访问哨兵;

1
2
3
4
5
6
Cannot get master address from sentinel running @ 192.168.2.101:26379.
Reason: redis.clients.jedis.exceptions.JedisDataException: DENIED Redis is running in protected mode because protected mode is enabled, no bind address was specified, no authentication password is requested to clients. In this mode connections are only accepted from the loopback interface. If you want to connect from external computers to Redis you may adopt one of the following solutions:
1) Just disable protected mode sending the command 'CONFIG SET protected-mode no' from the loopback interface by connecting to Redis from the same host the server is running, however MAKE SURE Redis is not publicly accessible from internet if you do so. Use CONFIG REWRITE to make this change permanent.
2) Alternatively you can just disable the protected mode by editing the Redis configuration file, and setting the protected mode option to 'no', and then restarting the server.
3) If you started the server manually just for testing, restart it with the '--protected-mode no' option.
4) Setup a bind address or an authentication password. NOTE: You only need to do one of the above things in order for the server to start accepting connections from the outside.. Trying next one