redis集群及redis集群监控redis-stat搭建

发表于   |   分类于   |  

redis集群搭建笔记

节点规划

准备了3台机子，每台机子设置两个redis几点，规划如下
192.168.0.131:7001(master)
192.168.0.131:7002(slave)
192.168.0.132:7001(master)
192.168.0.132:7002(slave)
192.168.0.133:7001(master)
192.168.0.133:7002(slave)

安装步骤

安装ruby

1、下载与上传

下载ruby安装包https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz
使用scp命令把下载好的文件上传到3个主机中（安装包我下载到/Users/demondevil/mydatas/softs/ruby/路径中）

1
2
3
scp /Users/demondevil/mydatas/softs/ruby/ruby-2.3.1.tar.gz root@192.168.0.131:/home/softs
scp /Users/demondevil/mydatas/softs/ruby/ruby-2.3.1.tar.gz root@192.168.0.132:/home/softs
scp /Users/demondevil/mydatas/softs/ruby/ruby-2.3.1.tar.gz root@192.168.0.133:/home/softs

2、安装

登陆主机
ssh root@192.168.0.131
解压

1
2
cd /home/softs/
tar -zxvf ruby-2.3.1.tar.gz

配置

1
2
cd ruby-2.3.1
./configure --prefix=/usr/local/ruby

编译与安装

1
2
make
make install

设置环境变量

1
vim /etc/profile

在最后面添加一行

1
export PATH=/usr/local/ruby/bin:$PATH

使配置起作用

1
source /etc/profile

查看是否配置成功

1
echo $PATH

安装redis-3.0.7.gem

方法一

下载安装redis-3.0.7.gem,安装目的是使得ruby可以与redis集群通信,这样子就可以使用 redis-trib.rb工具管理集群了
下载

1
2
cd /home/softs/
wget --no-check-certificate https://rubygems.global.ssl.fastly.net/gems/redis-3.0.7.gem

安装

1
gem install -l redis-3.0.7.gem

报错

cannot load such file – zlib

下载安装zlib

1
2
3
4
5
6
wget http://down1.chinaunix.net/distfiles/zlib-1.2.7.tar.gz
tar -zxvf zlib-1.2.7.tar.gz
cd zlib-1.2.7
./configure
make
make install

再次运行gem install -l redis-3.0.7.gem还是报相同的错误，解决方法
进入ruby源码文件夹

1
cd /home/softs/ruby-2.3.1

安装ruby自身提供的zlib包

1
2
3
4
cd ext/zlib
ruby ./extconf.rb
make
make install

回到redis-3.0.7.gem当前目录
再次运行gem install -l redis-3.0.7.gem成功

方法二

1
gem install redis

执行报错

Unable to require openssl, install OpenSSL and rebuild ruby (preferred) or use non-HTTPS sources

解决方法

1
2
yum -y install openssl-devel
yum -y install openssl

进入ruby 源码目录下的ext/openssl 目录

1
ruby extconf.rb

将ruby 源码目录下的include 目录软链接到 / 目录下：

1
2
3
ln -s /home/softs/ruby-2.3.1/include/ /
make
make install

再次执行gem install redis成功

如果有报源的错误则进行gem源更换，方法如下
gem使用的是https://rubygems.org/，境外源，国内无法访问
解决办法：
更换gem源为https://ruby.taobao.org/，淘宝源
查看当前使用源

1
2
3
4
gem source –l
结果显示
 *** CURRENT SOURCES ***
https://rubygems.org/

1
2
gem source -a https://ruby.taobao.org/ #添加淘宝源
gem source -r https://rubygems.org/	#删除境外源

192.168.0.131和192.168.0.133使用的是方法一
192.168.0.132使用的是方法二

安装redis

安装

下载安装包
http://download.redis.io/releases/redis-3.0.7.tar.gz
上传至主机

1
2
3
scp /Users/demondevil/mydatas/softs/redis/redis-3.0.7.tar.gz root@192.168.0.131:/home/softs
scp /Users/demondevil/mydatas/softs/redis/redis-3.0.7.tar.gz root@192.168.0.132:/home/softs
scp /Users/demondevil/mydatas/softs/redis/redis-3.0.7.tar.gz root@192.168.0.133:/home/softs

编译与安装

1
2
3
4
5
cd /home/softs
tar -zxvf redis-3.0.7.tar.gz
cd redis-3.0.7
make
make install

配置

对192.168.0.131主机进行如下操作

1
2
3
4
5
6
7
8
9
cd /home/softs/redis-3.0.7/src
cp redis-cli /usr/local/bin/
cp redis-server /usr/local/bin/
cp redis-trib.rb /usr/local/bin/
cd  /home/softs/
mkdir redis-cluster
cd redis-cluster
mkdir data
mkdir log

创建并编辑7001端口的redis命令脚本，用于启动停止等

1
vim redis_7001

redis_7001文件内容如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
#!/bin/sh
#Configurations injected by install_server below....

EXEC=/usr/local/bin/redis-server
CLIEXEC=/usr/local/bin/redis-cli
PIDFILE=/home/softs/redis-cluster/data/redis-7001.pid
CONF="/home/softs/redis-cluster/redis_7001.conf"
REDISPORT="7001"
###############
# SysV Init Information
# chkconfig: - 58 74
# description: redis_6379 is the redis daemon.
### BEGIN INIT INFO
# Provides: redis_7001
# Required-Start: $network $local_fs $remote_fs
# Required-Stop: $network $local_fs $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Should-Start: $syslog $named
# Should-Stop: $syslog $named
# Short-Description: start and stop redis_7001
# Description: Redis daemon
### END INIT INFO


case "$1" in
    start)
        if [ -f $PIDFILE ]
        then
            echo "$PIDFILE exists, process is already running or crashed"
        else
            echo "Starting Redis server..."
            $EXEC $CONF
        fi
        ;;
    stop)
        if [ ! -f $PIDFILE ]
        then
            echo "$PIDFILE does not exist, process is not running"
        else
            PID=$(cat $PIDFILE)
            echo "Stopping ..."
            $CLIEXEC -p $REDISPORT shutdown
            while [ -x /proc/${PID} ]
            do
                echo "Waiting for Redis to shutdown ..."
                sleep 1
            done
            echo "Redis stopped"
        fi
        ;;
    status)
        PID=$(cat $PIDFILE)
        if [ ! -x /proc/${PID} ]
        then
            echo 'Redis is not running'
        else
            echo "Redis is running ($PID)"
        fi
        ;;
    restart)
        $0 stop
        $0 start
        ;;
    *)
        echo "Please use start, stop, restart or status as first argument"
        ;;
esac

创建并编辑7001端口的redis配置文件，用于集群

1
vim redis_7001.conf

redis_7001.conf配置文件内容如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
################################ GENERAL  #####################################

daemonize yes
pidfile /home/softs/redis-cluster/data/redis-7001.pid
port 7001

# TCP listen() backlog.
#
# In high requests-per-second environments you need an high backlog in order
# to avoid slow clients connections issues. Note that the Linux kernel
# will silently truncate it to the value of /proc/sys/net/core/somaxconn so
# make sure to raise both the value of somaxconn and tcp_max_syn_backlog
# in order to get the desired effect.
#需要修改somaxconn and tcp_max_syn_backlog
tcp-backlog 1500


#客户端空闲多长时间，关闭链接。0表示不关闭；此处表示空闲20s后关闭连接
timeout 0

#心跳包,60s未收到心跳认为其死亡
tcp-keepalive 60

loglevel notice

logfile "/home/softs/redis-cluster/log/redis-7001.log"

databases 16

################################ SNAPSHOTTING  ################################

dir /home/softs/redis-cluster/data

################################# REPLICATION #################################
# 当从库同主机失去连接或者复制正在进行，从机库有两种运行方式：
#yes(默认设置)，从库会继续相应客户端的请求
#no除了INFO和SLAVOF命令之外的任何请求都会返回一个错误"SYNC with master in progress"
slave-serve-stale-data yes


# Since Redis 2.6 by default slaves are read-only.
slave-read-only yes


#slave会每隔repl-ping-slave-period(默认10秒)ping一次master，如果超过repl-timeout(默认 60秒)都没有收到响应，就会认为Master挂了。如果Master明明没挂但被阻塞住了也会报MASTER time out: no data nor PING received…这个错。
#slave每隔10s向主发送一次心跳
repl-ping-slave-period 10

#slave认为复制超时时间
repl-timeout 120

#在slave和master同步后（发送psync/sync），后续的同步是否设置成TCP_NODELAY
#假如设置成yes，则redis会合并小的TCP包从而节省带宽，但会增加同步延迟（40ms），造成master与slave数据不一致
#假如设置成no，则redis master会立即发送同步数据，没有延迟
#前者关注性能，后者关注一致性
repl-disable-tcp-nodelay yes

#多slave竞争转换为master的权值
slave-priority 100


################################## SECURITY ###################################



################################### LIMITS ####################################

# Set the max number of connected clients at the same time. By default
# this limit is set to 10000 clients, however if the Redis server is not
# able to configure the process file limit to allow for the specified limit
# the max number of allowed clients is set to the current file limit
# minus 32 (as Redis reserves a few file descriptors for internal uses).
#
# Once the limit is reached Redis will close all the new connections sending
# an error 'max number of clients reached'.
#
maxclients 20000

# Don't use more memory than the specified amount of bytes.
#外网40G的阀值，达到阀值开始清理数据
#maxmemory 40G
maxmemory 1G

maxmemory-policy allkeys-lru

############################## APPEND ONLY MODE ###############################
#不开启aof功能，该项目下配置的所有参数都不会生效
appendonly no

# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly.aof"
appendfsync everysec
#重写aof的期间新的写操作不要fsync,暂时存在内存中，等rewrite完成后再写入
no-appendfsync-on-rewrite yes

#达到上一份文件的80%或者文件至少达到1g时候才开始rewrite aof
auto-aof-rewrite-percentage 80-100
auto-aof-rewrite-min-size 1G

#aof rewrite过程中,是否采取增量文件同步策略,默认为“yes
aof-rewrite-incremental-fsync yes
aof-load-truncated yes

################################ LUA SCRIPTING  ###############################

lua-time-limit 5000

################################ REDIS CLUSTER  ###############################


cluster-enabled yes


cluster-config-file nodes-7001.conf

#节点超时时间20s认为超时 
cluster-node-timeout 20000


cluster-migration-barrier 1


################################## SLOW LOG ###################################
#记录查询超过200毫秒的所有查询
slowlog-log-slower-than 200000

# There is no limit to this length. Just be aware that it will consume memory.
# You can reclaim memory used by the slow log with SLOWLOG RESET.
#日志超过128后旧的记录被移除，以回收内存
slowlog-max-len 128

################################ LATENCY MONITOR ##############################

hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
activerehashing yes

client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 512mb 300mb 60
client-output-buffer-limit pubsub 32mb 8mb 60

hz 10

把命令脚本文件拷贝到/usr/local/bin/目录便于使用redis_7001命令

1
cp redis_7001 /usr/local/bin/

接着配置7002端口

1
2
3
4
5
cp redis_7001 redis_7002
vim redis_7002(把里面的7001改为7002)
cp redis_7002 /usr/local/bin/
cp redis_7001.conf redis_7002.conf
vim redis_7002.conf(把里面的7001改为7002)

添加执行权限

1
chmod u+x /usr/local/bin/redis_*

启动redis

1
2
redis_7001 start
redis_7002 start

停止redis

1
2
redis_7001 stop
redis_7002 stop

启动之后查看进程

1
2
3
4
ps -ef |grep redis
root     24381     1  0 08:09 ?        00:00:00 /usr/local/bin/redis-server *:7001 [cluster]                        
root     24418     1  4 08:17 ?        00:00:01 /usr/local/bin/redis-server *:7002 [cluster]                        
root     24425 24165  6 08:18 pts/0    00:00:00 grep redis

对192.168.0.133进行和192.168.0.132相同的操作

启动集群

通过redis-trib创建cluster

–replicas 则指定了为Redis Cluster中的每个Master节点配备几个Slave节点
要求至少6个节点、3个master节点

1
redis-trib.rb create --replicas 1 192.168.0.131:7001 192.168.0.132:7001 192.168.0.133:7001 192.168.0.131:7002 192.168.0.132:7002 192.168.0.133:7002

输出结果

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.0.131:7001
192.168.0.132:7001
192.168.0.133:7001
Adding replica 192.168.0.132:7002 to 192.168.0.131:7001
Adding replica 192.168.0.131:7002 to 192.168.0.132:7001
Adding replica 192.168.0.133:7002 to 192.168.0.133:7001
M: 51211a7220b37b53d826bc139e7780a06c39c6b0 192.168.0.131:7001
   slots:0-5460 (5461 slots) master
M: fda3e4e217f67f72c2082a9d417c7001eff4cfc8 192.168.0.132:7001
   slots:5461-10922 (5462 slots) master
M: 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5 192.168.0.133:7001
   slots:10923-16383 (5461 slots) master
S: 73c18b72a7543881feede89f0a249e83ac63b337 192.168.0.131:7002
   replicates fda3e4e217f67f72c2082a9d417c7001eff4cfc8
S: 31a7a5c5b5dbaf0ab7f0b73eb58c08d13a6d02d2 192.168.0.132:7002
   replicates 51211a7220b37b53d826bc139e7780a06c39c6b0
S: e7823da58afdf968948e39259ca80032bee28d6e 192.168.0.133:7002
   replicates 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join......
>>> Performing Cluster Check (using node 192.168.0.131:7001)
M: 51211a7220b37b53d826bc139e7780a06c39c6b0 192.168.0.131:7001
   slots:0-5460 (5461 slots) master
M: fda3e4e217f67f72c2082a9d417c7001eff4cfc8 192.168.0.132:7001
   slots:5461-10922 (5462 slots) master
M: 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5 192.168.0.133:7001
   slots:10923-16383 (5461 slots) master
M: 73c18b72a7543881feede89f0a249e83ac63b337 192.168.0.131:7002
   slots: (0 slots) master
   replicates fda3e4e217f67f72c2082a9d417c7001eff4cfc8
M: 31a7a5c5b5dbaf0ab7f0b73eb58c08d13a6d02d2 192.168.0.132:7002
   slots: (0 slots) master
   replicates 51211a7220b37b53d826bc139e7780a06c39c6b0
M: e7823da58afdf968948e39259ca80032bee28d6e 192.168.0.133:7002
   slots: (0 slots) master
   replicates 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

根据信息可知master、slave对应节点
192.168.0.131:7001(master)—>192.168.0.132:7002(slave)
192.168.0.132:7001(master)—>192.168.0.131:7002(slave)
192.168.0.133:7001(master)—>192.168.0.133:7002(slave)

检查集群信息
redis-trib.rb check anyExistNodeIp:port

1
redis-trib.rb check 192.168.0.131:7001

即可查看集群的相关信息

集群模式登录redis客户端

1
redis-cli -c -p 7001

在线添加/删除节点

先在192.168.0.133上添加两个redis服务

1
2
3
4
5
cd /home/softs/redis-cluster/
cp redis_7001 redis_7003
cp redis_7001 redis_7004
cp redis_7001.conf redis_7003.conf
cp redis_7001.conf redis_7004.conf

修改这4个文件，把7001改为对应的端口

1
2
3
cp redis_7003 /usr/local/bin/
cp redis_7004 /usr/local/bin/
chmod u+x /usr/local/bin/redis_700*

启动redis

1
2
redis_7003 start
redis_7004 start

添加master节点

执行redis-trib.rbadd-nodenewNodeIp:portanyExistNodeIp:port将新节点加入到 集群中

1
redis-trib.rb add-node 192.168.0.133:7003 192.168.0.133:7001

输出下面信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
>>> Adding node 192.168.0.133:7003 to cluster 192.168.0.133:7001
>>> Performing Cluster Check (using node 192.168.0.133:7001)
M: 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5 192.168.0.133:7001
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: e7823da58afdf968948e39259ca80032bee28d6e 192.168.0.133:7002
   slots: (0 slots) slave
   replicates 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5
S: 73c18b72a7543881feede89f0a249e83ac63b337 192.168.0.131:7002
   slots: (0 slots) slave
   replicates fda3e4e217f67f72c2082a9d417c7001eff4cfc8
M: fda3e4e217f67f72c2082a9d417c7001eff4cfc8 192.168.0.132:7001
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: 31a7a5c5b5dbaf0ab7f0b73eb58c08d13a6d02d2 192.168.0.132:7002
   slots: (0 slots) slave
   replicates 51211a7220b37b53d826bc139e7780a06c39c6b0
M: 51211a7220b37b53d826bc139e7780a06c39c6b0 192.168.0.131:7001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.0.133:7003 to make it join the cluster.
[OK] New node added correctly.

检查集群信息

1
redis-trib.rb check 192.168.0.131:7002

输出信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
>>> Performing Cluster Check (using node 192.168.0.131:7002)
S: 73c18b72a7543881feede89f0a249e83ac63b337 192.168.0.131:7002
   slots: (0 slots) slave
   replicates fda3e4e217f67f72c2082a9d417c7001eff4cfc8
M: b04d24e465d199e9f1a4fe52298dcaa9b48c187d 192.168.0.133:7003
   slots: (0 slots) master
   0 additional replica(s)
M: 51211a7220b37b53d826bc139e7780a06c39c6b0 192.168.0.131:7001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 31a7a5c5b5dbaf0ab7f0b73eb58c08d13a6d02d2 192.168.0.132:7002
   slots: (0 slots) slave
   replicates 51211a7220b37b53d826bc139e7780a06c39c6b0
M: fda3e4e217f67f72c2082a9d417c7001eff4cfc8 192.168.0.132:7001
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: e7823da58afdf968948e39259ca80032bee28d6e 192.168.0.133:7002
   slots: (0 slots) slave
   replicates 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5
M: 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5 192.168.0.133:7001
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

分配slot

数据分片redis-trib.rb reshard anyExistNodeIp:port

1
redis-trib.rb reshard 192.168.0.133:7001

输出信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
>>> Performing Cluster Check (using node 192.168.0.133:7003)
M: b04d24e465d199e9f1a4fe52298dcaa9b48c187d 192.168.0.133:7003
   slots: (0 slots) master
   0 additional replica(s)
S: 73c18b72a7543881feede89f0a249e83ac63b337 192.168.0.131:7002
   slots: (0 slots) slave
   replicates fda3e4e217f67f72c2082a9d417c7001eff4cfc8
S: e7823da58afdf968948e39259ca80032bee28d6e 192.168.0.133:7002
   slots: (0 slots) slave
   replicates 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5
S: 31a7a5c5b5dbaf0ab7f0b73eb58c08d13a6d02d2 192.168.0.132:7002
   slots: (0 slots) slave
   replicates 51211a7220b37b53d826bc139e7780a06c39c6b0
M: 51211a7220b37b53d826bc139e7780a06c39c6b0 192.168.0.131:7001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: fda3e4e217f67f72c2082a9d417c7001eff4cfc8 192.168.0.132:7001
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5 192.168.0.133:7001
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
--- 想要移动的slots的个数 
--- 想要移动的slots个数,如果是想移除某个节点,则该处的slots个数应该是此将要移除的节点上的slots个数
How many slots do you want to move (from 1 to 16384)? 2500
--- 哪个节点将会接收这些移动的slots，这里就是新添加的节点的nodeID，即b04d24e465d199e9f1a4fe52298dcaa9b48c187d
What is the receiving node ID? b04d24e465d199e9f1a4fe52298dcaa9b48c187d
--- 选择要移动的slots数据来源
Please enter all the source node IDs.
--- all表示均衡的从所有节点中获取slot
  Type 'all' to use all the nodes as source nodes for the hash slots.
--- 输入某个或者几个节点上面的slot并且以done结束
  Type 'done' once you entered all the source nodes IDs.
Source node #1:4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5
Source node #2:done
Do you want to proceed with the proposed reshard plan (yes/no)? yes

— 后面的中文是对应的解释，自己加上去。

添加对应的slave节点

1
redis-trib.rb add-node 192.168.0.133:7004 192.168.0.133:7001

连接到要添加的slave数据库中,然后执行replicate操作,id为该slave node应对的master nodeid即对应新添加的192.168.0.133:7003节点的nodeid

1
2
redis-cli -h 192.168.0.133 -p 7004
cluster replicate b04d24e465d199e9f1a4fe52298dcaa9b48c187d

再检查集群信息

1
redis-trib.rb check 192.168.0.131:7002

输出信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
>>> Performing Cluster Check (using node 192.168.0.131:7002)
S: 73c18b72a7543881feede89f0a249e83ac63b337 192.168.0.131:7002
   slots: (0 slots) slave
   replicates fda3e4e217f67f72c2082a9d417c7001eff4cfc8
M: b04d24e465d199e9f1a4fe52298dcaa9b48c187d 192.168.0.133:7003
   slots:0-188,5461-7580,10923-11112 (2499 slots) master
   1 additional replica(s)
M: 51211a7220b37b53d826bc139e7780a06c39c6b0 192.168.0.131:7001
   slots:189-5460 (5272 slots) master
   1 additional replica(s)
S: 31a7a5c5b5dbaf0ab7f0b73eb58c08d13a6d02d2 192.168.0.132:7002
   slots: (0 slots) slave
   replicates 51211a7220b37b53d826bc139e7780a06c39c6b0
M: fda3e4e217f67f72c2082a9d417c7001eff4cfc8 192.168.0.132:7001
   slots:7581-10922 (3342 slots) master
   1 additional replica(s)
S: eb5e0e1d3736f34137a6c60e50c3315add8c2355 192.168.0.133:7004
   slots: (0 slots) slave
   replicates b04d24e465d199e9f1a4fe52298dcaa9b48c187d
S: e7823da58afdf968948e39259ca80032bee28d6e 192.168.0.133:7002
   slots: (0 slots) slave
   replicates 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5
M: 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5 192.168.0.133:7001
   slots:11113-16383 (5271 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

删除节点

删除192.168.0.133:7003(master),192.168.0.133:7004(slave)两个节点
使用redis-trib.rb del-node willRemoveNodeIp:port willRemoveNodeID命令依次删除从节 点和主节点
——— 先删除slave节点

1
redis-trib.rb del-node 192.168.0.133:7004 eb5e0e1d3736f34137a6c60e50c3315add8c2355

输出信息

1
2
3
>>> Removing node eb5e0e1d3736f34137a6c60e50c3315add8c2355 from cluster 192.168.0.133:7004
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

——— 删除master节点
删除master节点之前先把该节点上对应全部slot迁移到另外一个节点上，目前redis-trib.rb一次只能迁移到一个节点上
1：移除master节点上的全部slot，使用命令redis-trib.rb reshard anyExistNodeIp:port

1
redis-trib.rb reshard 192.168.0.133:7001

输出信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
>>> Performing Cluster Check (using node 192.168.0.133:7001)
M: 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5 192.168.0.133:7001
   slots:11113-16383 (5271 slots) master
   1 additional replica(s)
S: e7823da58afdf968948e39259ca80032bee28d6e 192.168.0.133:7002
   slots: (0 slots) slave
   replicates 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5
S: 73c18b72a7543881feede89f0a249e83ac63b337 192.168.0.131:7002
   slots: (0 slots) slave
   replicates fda3e4e217f67f72c2082a9d417c7001eff4cfc8
M: fda3e4e217f67f72c2082a9d417c7001eff4cfc8 192.168.0.132:7001
   slots:7581-10922 (3342 slots) master
   1 additional replica(s)
S: 31a7a5c5b5dbaf0ab7f0b73eb58c08d13a6d02d2 192.168.0.132:7002
   slots: (0 slots) slave
   replicates 51211a7220b37b53d826bc139e7780a06c39c6b0
M: b04d24e465d199e9f1a4fe52298dcaa9b48c187d 192.168.0.133:7003
   slots:0-188,5461-7580,10923-11112 (2499 slots) master
   0 additional replica(s)
M: 51211a7220b37b53d826bc139e7780a06c39c6b0 192.168.0.131:7001
   slots:189-5460 (5272 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 2500
---- 这里选择192.168.0.132:7001节点接受将要移除的slot
What is the receiving node ID? fda3e4e217f67f72c2082a9d417c7001eff4cfc8
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
---- 这里选择要移除的192.168.0.133:7003节点nodeid
Source node #1:b04d24e465d199e9f1a4fe52298dcaa9b48c187d
Source node #2:done
Do you want to proceed with the proposed reshard plan (yes/no)? yes

2：删除master节点
删除节点之前使用redis-trib.rb check existNodeIp:port命令检查集群信息，检测该节点上面的slot是否全部移除干净

1
redis-trib.rb check 192.168.0.131:7002

输出信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
>>> Performing Cluster Check (using node 192.168.0.131:7002)
S: 73c18b72a7543881feede89f0a249e83ac63b337 192.168.0.131:7002
   slots: (0 slots) slave
   replicates fda3e4e217f67f72c2082a9d417c7001eff4cfc8
M: b04d24e465d199e9f1a4fe52298dcaa9b48c187d 192.168.0.133:7003
   slots: (0 slots) master
   0 additional replica(s)
M: 51211a7220b37b53d826bc139e7780a06c39c6b0 192.168.0.131:7001
   slots:189-5460 (5272 slots) master
   1 additional replica(s)
S: 31a7a5c5b5dbaf0ab7f0b73eb58c08d13a6d02d2 192.168.0.132:7002
   slots: (0 slots) slave
   replicates 51211a7220b37b53d826bc139e7780a06c39c6b0
M: fda3e4e217f67f72c2082a9d417c7001eff4cfc8 192.168.0.132:7001
   slots:0-188,5461-11112 (5841 slots) master
   1 additional replica(s)
S: e7823da58afdf968948e39259ca80032bee28d6e 192.168.0.133:7002
   slots: (0 slots) slave
   replicates 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5
M: 4cfc061fa365a5f7e2a9ff07c0e3c08f12b7b2c5 192.168.0.133:7001
   slots:11113-16383 (5271 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

可看到192.168.0.133:7003节点上的slot已经全部被移除
执行删除

1
redis-trib.rb del-node 192.168.0.133:7003 b04d24e465d199e9f1a4fe52298dcaa9b48c187d

输出信息

1
2
3
>>> Removing node b04d24e465d199e9f1a4fe52298dcaa9b48c187d from cluster 192.168.0.133:7003
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

再次检查集群信息会发现已经没有这个节点了

重启集群
把全部redis服务停止
删除data目录下文件

1
rm -f /home/softs/redis-cluster/data/*

启动redis服务
由于redis没有配置持久化机制，所以之前的数据会被全部清空
重新创建集群

1
redis-trib.rb create --replicas 1 192.168.0.131:7001 192.168.0.132:7001 192.168.0.133:7001 192.168.0.131:7002 192.168.0.132:7002 192.168.0.133:7002

redis集群监控redis-stat

源码地址
https://github.com/junegunn/redis-stat
参考博客
http://blog.csdn.net/21aspnet/article/details/50748719

规划在192.168.0.130节点上搭建redis-stat
以下是我的搭建记录

1
2
3
4
5
ssh root@192.168.0.130
cd /home/softs/
yum install -y ruby
yum install -y ruby-devel
yum install -y rubygems

1
2
gem source -a https://ruby.taobao.org/ #添加淘宝源
gem source -r http://rubygems.org/ #删除境外源
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
[root@localhost softs]# gem install redis-stat
Building native extensions.  This could take a while...
Building native extensions.  This could take a while...
Building native extensions.  This could take a while...
ERROR:  Error installing redis-stat:
redis-stat requires daemons (~> 1.1.9, runtime)
[root@localhost softs]# gem install redis-stat -v 0.4.12
Successfully installed daemons-1.1.9
Successfully installed redis-stat-0.4.12
2 gems installed
Installing ri documentation for daemons-1.1.9...
Installing ri documentation for redis-stat-0.4.12...
Installing RDoc documentation for daemons-1.1.9...
Installing RDoc documentation for redis-stat-0.4.12...
[root@localhost softs]# redis-stat 1
/usr/lib/ruby/site_ruby/1.8/rubygems.rb:233:in `activate': can't activate daemons (~> 1.1.9, runtime) for ["redis-stat-0.4.12"], already activated daemons-1.2.3 for ["thin-1.5.1", "redis-stat-0.4.12"] (Gem::LoadError)
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:249:in `activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:248:in `each'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:248:in `activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:1082:in `gem'
from /usr/bin/redis-stat:18
[root@localhost softs]# gem uninstall daemons
Select gem to uninstall:
 1. daemons-1.1.9
 2. daemons-1.2.3
 3. All versions
> 2
Successfully uninstalled daemons-1.2.3
[root@localhost softs]# [root@localhost softs]# redis-stat 1
-bash: [root@localhost: command not found
[root@localhost softs]# gem install redis-stat -v 0.4.12
Successfully installed redis-stat-0.4.12
1 gem installed
Installing ri documentation for redis-stat-0.4.12...
Installing RDoc documentation for redis-stat-0.4.12...
[root@localhost softs]# redis-stat 1
Faraday: you may want to install system_timer for reliable timeouts
[root@localhost softs]# gem install SystemTimer
Building native extensions.  This could take a while...
Successfully installed SystemTimer-1.2.3
1 gem installed
Installing ri documentation for SystemTimer-1.2.3...
Installing RDoc documentation for SystemTimer-1.2.3...

命令行启动redis-stat

1
redis-stat 192.168.0.131:7001 192.168.0.131:7002 192.168.0.132:7001 192.168.0.132:7002 192.168.0.133:7001 192.168.0.133:7002

以webserver的形式启动，指定端口

1
redis-stat 192.168.0.131:7001 192.168.0.131:7002 192.168.0.132:7001 192.168.0.132:7002 192.168.0.133:7001 192.168.0.133:7002 --server=8080 5 —daemon

浏览器访问http://192.168.0.130:8080即可看到实时监控页面

热评文章