Keepalived+Nginx 高可用(主从模式)

 1. 环境说明

vip ip 操作系统 nginx端口 主从说明
192.168.115.150 192.168.115.148 CentOS Linux release 7.6.1810 (Core) 8443 主节点
192.168.115.150 192.168.115.149 CentOS Linux release 7.6.1810 (Core) 8443 从节点

2台机器都需部署nginx和keealive服务,nginx配置文件保持一致,keepalive服务配置存在差异

2. 部署nginx

3. 部署keepalive

3.1 安装

1、下载
wget https://www.keepalived.org/software/keepalived-2.2.2.tar.gz

2、解压
tar -zxvf keepalived-2.2.2.tar.gz

3、安装依赖
yum -y install libnl libnl-devel  
否则,会报错编译会报错:*** WARNING - this build will not support IPVS with IPv6. Please install libnl/libnl-3 dev libraries to support IPv6 with IPVS.

4、编译安装
cd  keepalived-2.2.2
./configure --prefix=/usr/local/keepalived 
make && make install 

 

3.2 修改配置

1、修改keepalive配置

cp -raf /usr/local/keepalived/etc/keepalived/keepalived.conf /usr/local/keepalived/etc/keepalived/keepalived.conf_default
vim /usr/local/keepalived/etc/keepalived/keepalived.conf 
主节点192.168.115.148和从节点192.168.115.149存在差异,具体配置文件如下:

主节点192.168.115.148:/usr/local/keepalived/etc/keepalived/keepalived.conf配置如下

global_defs {               
   router_id Nginx_01
   script_user root
}
vrrp_script check_nginx {
        script "/usr/local/nginx/keepalived/check_nginx.sh"      #nginx检测脚本
        interval 2
    weight -5
    fall 3
    rise 2
}
vrrp_instance VI_1 {
    state MASTER
    interface ens192                         #指定HA监测网络的接口。与本机 IP 地址所在的网络接口相同,可通过ip addr 查看
    virtual_router_id 33                     #虚拟路由标识,这个标识是一个数字,同一个vrrp实例使用唯一的标识。即同一vrrp_instance下,MASTER和BACKUP必须是一致
    priority 100                             #定义优先级,数字越大,优先级越高,在同一个vrrp_instance下,MASTER的优先级必须大于BACKUP的优先级
    advert_int 1
    authentication {                         #设置验证类型和密码。主从必须一样
        auth_type PASS                       #设置vrrp验证类型,主要有PASS和AH两种
        auth_pass 1111                       #设置vrrp验证密码,在同一个vrrp_instance下,MASTER与BACKUP必须使用相同的密码才能正常通信
    }
    virtual_ipaddress {                      #VRRP HA 虚拟地址 如果有多个VIP,继续换行填写
     192.168.115.150
    }
    track_script {
        check_nginx
    }
} 

从节点192.168.115.149:/usr/local/keepalived/etc/keepalived/keepalived.conf配置如下:
global_defs {               
   script_user root
   router_id Nginx_02
}
vrrp_script check_nginx {
    script "/usr/local/nginx/keepalived/check_nginx.sh"
    interval 2
    weight -5
    fall 3
    rise 2
}
vrrp_instance VI_1 {
    state BACKUP
    interface ens192
    virtual_router_id 33
    priority 50
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
      192.168.115.150
    }
    track_script {
        check_nginx
    }
}

2、新建nginx检测脚本

vim /usr/local/nginx/keepalived/check_nginx.sh
chmod +x  /usr/local/nginx/keepalived/check_nginx.sh

主从节点192.168.115.148/149 nginx检测脚本一致,/usr/local/nginx/keepalived/check_nginx.sh配置文件如下:
! /bin/bash
pidof nginx
if [ $? -ne 0 ];then
systemctl stop keepalived
fi

3.3 启动

#开机自启动
systemctl  enable keepalived

#查看状态
systemctl status keepalived

#启动
systemctl start keepalived

#加载配置
systemctl daemon-reload

#重启
systemctl restart keepalived

3.4 特殊说明

如未遇到以下情况,忽略即可

1、启动问题

直接启动如果报错,可能是systemctl管理未指定配置文件,修改配置systemctl自启动文件,重新加载配置文件并重新启动keepalive

vim /usr/lib/systemd/system/keepalived.service

[Unit]
Description=LVS and VRRP High Availability Monitor
After=network-online.target syslog.target 
Wants=network-online.target 

[Service]
Type=forking
PIDFile=/run/keepalived.pid
KillMode=process
EnvironmentFile=-/usr/local/keepalived/etc/sysconfig/keepalived
#ExecStart=/usr/local/keepalived/sbin/keepalived  $KEEPALIVED_OPTIONS   ##注释默认
ExecStart=/usr/local/keepalived/sbin/keepalived -f /usr/local/keepalived/etc/keepalived/keepalived.conf ##指定配置文件
ExecReload=/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target

 2、防火墙配置

主从节点启动keepalived之后, 正常情况下vip只在主节点上存在,而从节点在主节点故障时接管VIP。

如果发现VIP在两个节点上同时存在,要配置防火墙以防止VIP在两个节点上同时存在,需要确保VRRP协议能够在两个节点间正常通信

#允许VRRP协议通过防火墙
firewall-cmd --add-rich-rule='rule protocol value="vrrp" accept' --permanent

#重新加载 firewalld 
firewall-cmd --reload

4. 高可用测试

1、主从2个节点分别先启动nginx服务再keepalived服务,确保vip只在主节点上存在

2、模拟主节点nginx服务停止:停止nginx服务

检查主节点keepalived和vip情况,正常情况为:主节点keepalived停止,vip漂移到从节点,流量转移到从节点

3、模拟主节点nginx服务恢复:启动nginx服务和keepalived服务

 检查vip情况,正常情况为:vip漂移到主节点,流量转移到主节点,从节点恢复空间状态