Nginx反向代理-Keepalived做高可用-摩杜云开发者社区

Keepalived-HA-主节点配置管理

1.软件安装

[root@ha1 nginx]# yum install -y keepalived
[root@ha1 nginx]# cd /etc/keepalived/
[root@ha1 keepalived]# mv keepalived.conf keepalived.conf.bak

2.主备配置-集群功能

[root@ha1 keepalived]# vim keepalived.conf
! Configuration File for keepalived

global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_master # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}

vrrp_instance VI_1 {
state MASTER #节点状态
interface ens33 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - - 主备节点参数需要相同
#尽可能⾃定义 1-255 ｜防⽌冲突
priority 150 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication {
auth_type PASS #开始认证
auth_pass Admin@123 #
}
virtual_ipaddress {
192.168.40.113
}
}

3.启动与自启动

[root@ha1 keepalived]# systemctl start keepalived.service && systemctl enable keepalived.service
[root@ha1 keepalived]# systemctl status keepalived.service

4.查看网址信息

[root@ha1 keepalived]# ip add
2: ens33:  mtu 1500 qdisc pfifo_fast state UP qlen 1000                                                                                                                                           
    link/ether 00:0c:29:15:8f:0c brd ff:ff:ff:ff:ff:ff                                                                                                                                                                             
    inet 192.168.40.111/24 brd 192.168.40.255 scope global ens33  #本机⽹卡真实地址                                                                                                                                                                   
       valid_lft forever preferred_lft forever                                                                                                                                                                                     
    inet 192.168.40.113/32 scope global ens33      # 集群虚拟IP地址 （主）

Keepalived-HA-备节点配置管理

1.软件安装

[root@ha2 nginx]# yum install -y keepalived
[root@ha2 nginx]# cd /etc/keepalived/
[root@ha2 keepalived]# mv keepalived.conf keepalived.conf.bak

2.文件配置

[root@ha2 keepalived]# vim keepalived.conf
! Configuration File for keepalived

global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_backup # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state BACKUP #节点状态
interface ens33 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - 主备节点参数需要相同
#尽可能⾃定义 1-255 ｜防⽌冲突
priority 90 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication {
auth_type PASS #开始认证
auth_pass Admin@123 #
}
virtual_ipaddress {
192.168.40.113
}
}

3、启动与自启动

[root@ha2 keepalived]# systemctl start keepalived.service && systemctl enable keepalived.service
[root@ha2 keepalived]# systemctl status keepalived.service

4、查看网址信息

[root@ha2 keepalived]# ip add 2: ens33: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:30:ed:19 brd ff:ff:ff:ff:ff:ff inet 192.168.40.112/24 brd 192.168.40.255 scope global ens33

Keepalived-HA-主备倒换测试概述

主备倒换测试

1.模拟主节点故障 - 观察，⽤户访问流量是否能正常切换到备份节点。 ----- Keepalived服务停⽌

2.恢复主节点 - 观察，⽤户访问流量是否能正常回切到主节点。 ----- Keepalived服务正常打开

Keepalived-HA-主备倒换测试操作

[root@ha1 keepalived]# systemctl stop keepalived.service
root@ha2 keepalived]# systemctl status keepalived.service                                                                                                                                                                         
● keepalived.service - LVS and VRRP High Availability Monitor                                                                                                                                                                      
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)                                                                                                                                   
   Active: active (running) since Fri 2022-12-23 10:27:48 CST; 4h 20min ago                                                                                                                                                        
 Main PID: 14663 (keepalived)                                                                                                                                                                                                      
   CGroup: /system.slice/keepalived.service                                                                                                                                                                                        
           ├─14663 /usr/sbin/keepalived -D                                                                                                                                                                                         
           ├─14664 /usr/sbin/keepalived -D                                                                                                                                                                                         
           └─14665 /usr/sbin/keepalived -D                                                                                                                                                                                         
                                                                                                                                                                                                                                   
Dec 23 14:48:17 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) Transition to MASTER STATE                                                                                                                                         
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) Entering MASTER STATE     #就是这个位置                                                                                                                                         
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) setting protocol iptable drop rule                                                                                                                                 
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) setting protocol VIPs.                                                                                                                                             
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113                                                                                                                                     
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.40.113                                                                                                       
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113                                                                                                                                     
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113                                                                                                                                     
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113                                                                                                                                     
Dec 23 14:48:18 ha2 Keepalived_vrrp[14665]: Sending gratuitous ARP on ens33 for 192.168.40.113

Keepalived-HA-Nginx-健康检测

[root@ha1 keepalived]# systemctl stop nginx
[root@ha1 keepalived]#
[root@ha1 keepalived]# ip add show ens192
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen
1000
link/ether 00:50:56:94:4a:c5 brd ff:ff:ff:ff:ff:ff
inet 192.168.40.111/24 brd 192.168.8.255 scope global ens192
valid_lft forever preferred_lft forever
inet 192.168.8.113/32 scope global ens192 #集群IP地址
valid_lft forever preferred_lft forever

# 当Nginx⽆法提供服务时，当前节点⾃动关闭keepalived

# 主节点Nginx⽆法提供服务时，keepalived⾃动关闭 - 不再发送⼼跳消息 - 备份节点如果nginx可

⽤切换keepalived主节点。

1、抓取个数

[root@ha1 keepalived]# netstat -ntlp | grep "nginx: master" | wc -l
1    #nginx端⼝监听成功
[root@ha2 keepalived]# netstat -ntlp | grep "nginx: master" | wc -l
0    #nginx端⼝监听失败

1.检测监听端⼝

2.监听没有开启 - 尝试拉活

3.如果⽆法拉活 - 关闭keepalived

2、自启脚本

cd /etc/keepalived
vim check_nginx_service_port.sh
#!/bin/bash
if [ "$(netstat -ntlp | grep "nginx: master" | wc -l)" == "0" ]
  then
    systemctl restart nginx
    sleep 2
    if [ "$(netstat -ntlp | grep "nginx: master" | wc -l)" == "0" ]
      then
       systemctl stop keepalive
   fi
fi
[root@ha1 keepalived]# chmod +x check_nginx_service_port.sh

! Configuration File for keepalived

global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_master # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}

vrrp_script keepalived_check_nginx_service_port {
script "/etc/keepalived/check_nginx_service_port.sh"
interval 2 # 执⾏脚本时间间隔
}

vrrp_instance VI_1 {
state MASTER #节点状态
interface ens192 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - - 主备节点参数需要相同
#尽可能⾃定义 1-255 ｜防⽌冲突
priority 150 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication {
auth_type PASS #开始认证
auth_pass Admin@123 #
}

virtual_ipaddress {
192.168.8.113
}

track_script {
keepalived_check_nginx_service_port
}

}

#测试流程

systemctl stop nginx && systemctl stop keepalived
systemctl start nginx && systemctl start keepalived
systemctl is-active nginx && systemctl is-active keepalived
active
active

Nginx反向代理-Keepalived做高可用_优先级

[root@ha1 keepalived]# systemctl is-active nginx
active
[root@ha1 keepalived]# netstat -ntlp | grep nginx
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
2238/nginx: master
[root@ha1 keepalived]# systemctl is-active keepalived.service
active

Nginx反向代理-Keepalived做高可用_nginx_02

systemctl stop nginx
[root@ha1 keepalived]# systemctl is-active nginx
inactive
[root@ha1 keepalived]# systemctl status keepalived.service | tail -n 5
Aug 01 05:42:21 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:23 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:25 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:27 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:29 ha1 Keepalived_vrrp[2472]:
/etc/keepalived/check_nginx_service_port.sh exited with status 127
Aug 01 05:42:31 ha1 Keepalived_vrrp[2472]:

原因是selinux没关闭。

[root@ha1 ~]# sestatus -v | grep -i mode
Current mode: enforcing
Mode from config file: permissive

[root@ha1 ~]# setenforce 0
[root@ha1 ~]# sestatus -v | grep -i mode
Current mode: permissive
Mode from config file: permissive

#继续再次测试

[root@ha1 ~]# systemctl stop nginx
[root@ha1 ~]# systemctl is-active nginx
active
[root@ha1 ~]# systemctl status keepalived.service | tail -n 10
Aug 01 05:49:17 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: VRRP_Instance(VI_1) Sending/queueing
gratuitous ARPs on ens192 for 192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:22 ha1 Keepalived_vrrp[9783]: Sending gratuitous ARP on ens192 for
192.168.8.113
Aug 01 05:49:48 ha1 Keepalived_vrrp[9783]:
VRRP_Script(keepalived_check_nginx_service_port) timed out
Aug 01 05:49:48 ha1 Keepalived_vrrp[9783]:
/etc/keepalived/check_nginx_service_port.sh exited due to signal 15
Aug 01 05:49:48 ha1 Keepalived_vrrp[9783]:
VRRP_Script(keepalived_check_nginx_service_port) succeeded

Keepalived-HA-Nginx-健康检测-备份节点

! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1 #邮箱地址
smtp_connect_timeout 30
router_id keepalived_backup # 真实路由器ID - 主备节点参数需要不同
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_script keepalived_check_nginx_service_port {
script "/etc/keepalived/check_nginx_service_port.sh"
interval 2 # 执⾏脚本时间间隔
}
vrrp_instance VI_1 {
state BACKUP #节点状态
interface ens192 #节点keepalived接⼝IP地址
virtual_router_id 131 #虚拟路由器ID - - 主备节点参数需要相同
#尽可能⾃定义 1-255 ｜防⽌冲突
priority 90 #选举优先级 - Master节点优先级⾼
advert_int 1 # ⼼跳通告时间间隔 advertisement-internal - 1S |检测倍数3倍
authentication {
auth_type PASS #开始认证
auth_pass Admin@123 #
}
virtual_ipaddress {
192.168.8.113
}
track_script {
keepalived_check_nginx_service_port
}
}