keepalived+nginx+tomcat实现高可用负载均衡 |
您所在的位置：网站首页 › fastdfs高可用部署 › keepalived+nginx+tomcat实现高可用负载均衡 |
keepalived+nginx+tomcat实现高可用负载均衡
|
keepalived简介
keepalived是一个类似于layer3, 4 & 5交换机制的软件，Keepalived的作用是检测服务器的状态，如果有一台web服务器宕机，或工作出现故障，Keepalived将检测到，并将有故障的服务器从系统中剔除，同时使用其他服务器代替该服务器的工作，当服务器工作正常后Keepalived自动将服务器加入到服务器群中，这些工作全部自动完成，不需要人工干涉，需要人工做的只是修复故障的服务器。 环境两台Redhat7虚拟机 server1 192.168.56.132 keepalived,nginx,tomcat 作为master节点 server2 192.168.56.133 keepalived,nginx,tomcat 作为slave节点 Firewall关闭 selinux关闭 原理nginx本身具有反向代理的功能，用户访问的请求通过nginx代理转发到后端的tomcat，具体的负载均衡策略在nginx中进行配置 nginx+tomcat已经实现了用户请求的负载均衡，但nginx存在单点的风险，当nginx宕掉之后整个系统就会瘫痪掉，keepalived通过虚拟冗余协议实现对nginx高可用。 Keepalived 以 VRRP 协议为实现基础，用 VRRP 协议来实现高可用性(HA)。 一组VRRP路由器协同工作，共同构成一台虚拟路由器。该虚拟路由器对外表现为一个具有唯一固定的IP地址和MAC地址的逻辑路由器。处于同一个VRRP组中的路由器具有两种互斥的角色：主控路由器和备份路由器，一个VRRP组中有且只有一台处于主控角色的路由器，可以有一个或者多个处于备份角色的路由器VRRP协议从路由器组中选出一台作为主控路由器，负责ARP解析和转发IP数据包，组中的其他路由器作为备份的角色并处于待命状态，当由于某种原因主控路由器发生故障时，其中的一台备份路由器能在瞬间的时延后升级为主控路由器，由于此切换非常迅速而且不用改变IP地址和MAC地址，故对终端使用者系统是透明的。 可以用yum安装也可以用二进制包安装，我的虚拟机已经安装JDK就不在重复安装了。 [root@server1 bin]# java -version java version "1.8.0_171" Java(TM) SE Runtime Environment (build 1.8.0_171-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode)注意配置Java环境变量 [root@server1 bin]# vim /etc/profile export JAVA_HOME=/usr/local/jdk1.8.0_171 export PATH=$PATH:$JAVA_HOME/bin 安装Tomcat 1.在server1和server2上分别安装Tomcat [root@server1 bin]# tar -zxvf apache-tomcat-8.5.46.tar.gz -C /usr/local [root@server2 conf]# tar -zxvf apache-tomcat-8.5.46.tar.gz -C /usr/local [root@server1 bin]# mv apache-tomcat-8.5.46/ tomcat [root@server2 conf]# mv apache-tomcat-8.5.46/ tomcat 2.修改Tomcat的默认页面用于区分两个TomcatTomcat的页面在tomcat/webapps/ROOT目录下，删除或移走文件（server2上同样操作） [root@server1 ROOT]# pwd /usr/local/tomcat/webapps/ROOT [root@server1 ROOT]# mkdir tomcat root@server1 ROOT]# mv * tomcat在ROOT下新建index.html文件 这里配置IP进行区分两个tomcat，X-NGINX值将在后面配置NGINX时传入 [root@server1 ROOT]# vim index.jsp ${pageContext.servletContext.serverInfo}(192.168.56.132) [root@server2 ROOT]# vim index.jsp ${pageContext.servletContext.serverInfo}(192.168.56.133)启动tomcat1和tomcat2 [root@server1 tomcat]# bin/startup.sh [root@server2 tomcat]# bin/startup.sh分别访问tomcat1和tomcat2 下载地址：http://nginx.org/download/ 选择合适的版本进行下载 [root@localhost Downloads]# ls nginx-1.10.0.tar.gz解压nginx至/usr/local目录下 [root@localhost Downloads]# tar -zxf nginx-1.10.0.tar.gz -C /usr/local/ 3.编译安装 [root@localhost nginx-1.10.0]# ./configure --prefix=/usr/local/nginx --with-http_ssl_module [root@localhost nginx-1.10.0]# make && make install查看nginx版本 [root@localhost conf]# /usr/local/nginx/sbin/nginx -v nginx version: nginx/1.10.0 4.启动nginx [root@server1 sbin]# ./nginx [root@server1 sbin]# ps -ef |grep nginx root 45461 1 0 22:30 ? 00:00:00 nginx: master process ./nginx nobody 45463 45461 0 22:30 ? 00:00:00 nginx: worker process root 45475 25292 0 22:30 pts/0 00:00:00 grep --color=auto nginx 5.配置nginx文件，添加负载均衡 [root@server1 conf]# pwd /usr/local/nginx/conf [root@server1 conf]# vim nginx.conf http { upstream redhat { server 192.168.56.132:8080; server 192.168.56.133:8080; } include mime.types; default_type application/octet-stream; server { listen 80; server_name localhost; } location / { proxy_pass http://redhat; proxy_set_header X-NGINX "NGINX-1"; } }server2上同样安装nginx,我这里直接将server1的nginx复制到server2上 [root@server1 local]# scp -r nginx/ [email protected]:/usr/local/在server1和server2的/etc/hosts文件中添加对www.123.com的解析 [root@server1 sbin]# vim /etc/hosts #server2同样步骤 192.168.56.132 www.123.com 192.168.56.133 www.123.com 6.启动nginx [root@server1 sbin]# ./nginx [root@server1 sbin]# netstat -antlpe |grep nginx tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 0 346136 45461/nginx: master 7.访问localhost:80验证nginx负载均衡是否生效
可以到官网下载tar包解压安装也可以用yum安装，这里我用yum安装 yum源配置可以参考yum源配置方法 [root@server1 ~]# yum install keepalived -y [root@server2 ~]# yum install keepalived -y 2.修改keepalived配置文件修改server1的keepalived配置文件 [root@server1 ~]# vim /etc/keepalived/keepalived.conf vrrp_instance VI_1 { state MASTER #master节点 interface eno16777736 #网卡 virtual_router_id 51 #必须和slave相同 priority 100 #优先级，master优先级大于slave advert_int 1 authentication { auth_type PASS auth_pass 1111 #设置验证信息，两个节点必须一致 } virtual_ipaddress { 192.168.56.140 #虚拟IP } track_script { check_nginx # nginx存活状态检测脚本 } }修改server2上keepalived配置文件 [root@server2 yum.repos.d]# vim /etc/keepalived/keepalived.conf vrrp_instance VI_1 { state BACKUP #slave节点 interface eno16777736 #网卡 virtual_router_id 51 #必须和master相同 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 1111 #设置验证信息，两个节点必须一致 } virtual_ipaddress { 192.168.56.140 #虚拟IP } track_script { check_nginx # nginx存活状态检测脚本 } } 3.创建nginx检测脚本（server1和server2相同） [root@server1 local]# vim check_nginx #!/bin/bash ng=`ps -C nginx --no-header | wc -l` if [ $A -eq 0 ];then /usr/local/nginx/sbin/nginx #尝试重新启动nginx sleep 2 #睡眠2秒 if [ `ps -C nginx --no-header | wc -l` -eq 0 ];then killall keepalived #启动失败，将keepalived服务杀死。将vip漂移到其它备份节点 fi fi 4.启动keepalived [root@server1 local]# systemctl start keepalived.service [root@server2 nginx]# systemctl restart keepalived [root@server1 local]# ps -ef |grep keepalived root 48456 1 0 22:56 ? 00:00:00 /usr/sbin/keepalived -D root 48457 48456 0 22:56 ? 00:00:00 /usr/sbin/keepalived -D root 48458 48456 14 22:56 ? 00:00:01 /usr/sbin/keepalived -D [root@server2 nginx]# ps -ef |grep keepalived root 22872 1 0 21:17 ? 00:00:00 /usr/sbin/keepalived -D root 22873 22872 0 21:17 ? 00:00:00 /usr/sbin/keepalived -D root 22874 22872 0 21:17 ? 00:00:00 /usr/sbin/keepalived -D查看虚拟IP是在master上 [root@server1 local]# ip add 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eno16777736: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:0e:c1:04 brd ff:ff:ff:ff:ff:ff inet 192.168.56.132/24 brd 192.168.56.255 scope global noprefixroute dynamic eno16777736 valid_lft 1722sec preferred_lft 1722sec inet 192.168.56.140/32 scope global eno16777736 这里已经能看到虚拟IP了 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe0e:c104/64 scope link noprefixroute valid_lft forever preferred_lft forever [root@server2 local]# ip addr 1: lo: mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eno16777736: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:59:21:53 brd ff:ff:ff:ff:ff:ff inet 192.168.56.133/24 brd 192.168.56.255 scope global dynamic eno16777736 valid_lft 1519sec preferred_lft 1519sec inet6 fe80::20c:29ff:fe59:2153/64 scope link valid_lft forever preferred_lft forever 4.停掉master节点，看虚拟IP是否漂移 [root@server1 local]# systemctl stop keepalived.service [root@server2 local]# ip addr eno16777736: mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:59:21:53 brd ff:ff:ff:ff:ff:ff inet 192.168.56.133/24 brd 192.168.56.255 scope global dynamic eno16777736 valid_lft 1350sec preferred_lft 1350sec inet 192.168.56.140/32 scope global eno16777736 虚拟IP已经漂移到slave了 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe59:2153/64 scope link valid_lft forever preferred_lft forever启动master，虚拟IP再次回到master [root@server1 local]# systemctl start keepalived.service [root@server1 local]# ip addr eno16777736: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:0e:c1:04 brd ff:ff:ff:ff:ff:ff inet 192.168.56.132/24 brd 192.168.56.255 scope global noprefixroute dynamic eno16777736 valid_lft 1471sec preferred_lft 1471sec inet 192.168.56.140/32 scope global eno16777736 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe0e:c104/64 scope link noprefixroute valid_lft forever preferred_lft forever 5.模拟业务测试单点故障现在两个nginx都是启动状态，测试负载均衡是否正常 访问虚拟IP不通 [root@server1 nginx]# ping 192.168.56.140 PING 192.168.56.140 (192.168.56.140) 56(84) bytes of data. ^C --- 192.168.56.140 ping statistics --- 71 packets transmitted, 0 received, 100% packet loss, time 70022ms在网上查找原因是keepalived.conf配置中默认vrrp_strict打开了，注释掉vrrp_strict [root@server1 nginx]# vim /etc/keepalived/keepalived.conf notification_email_from [email protected] smtp_server 192.168.200.1 smtp_connect_timeout 30 router_id LVS_DEVEL vrrp_skip_check_adv_addr # vrrp_strict 注释掉 vrrp_garp_interval 0 vrrp_gna_interval 0重启keepalived,可以ping通了 [root@server1 nginx]# ping 192.168.56.140 PING 192.168.56.140 (192.168.56.140) 56(84) bytes of data. 64 bytes from 192.168.56.140: icmp_seq=1 ttl=64 time=0.040 ms 64 bytes from 192.168.56.140: icmp_seq=2 ttl=64 time=0.054 ms 64 bytes from 192.168.56.140: icmp_seq=3 ttl=64 time=0.036 ms访问虚拟IP 访问虚拟IP 启动master上的nginx,停掉slave上的nginx,再次测试 [root@server2 nginx]# sbin/nginx -s stop [root@server1 nginx]# sbin/nginx 访问虚拟IP 通过上面对master和slave的nginx停启，虚拟IP在master和slave之间漂移，证明了keepalived能够实现对nginx高可用，再加上nginx对tomcat进行负载均衡，这样就完成了一个简单的高可用负载均衡集群。 |
刷新后就变成tomcat2了，说明nginx配置负载均衡已经生效 
刷新变成tomcat2，说明nginx功能正常。 此时负责分发请求的nginx是nginx1，因为虚拟IP在master上 
停掉master上的nginx,再次测试
再次刷新，页面变成tomcat2。此时虚拟IP漂移到slave上，所以nginx2来分发请求 
再次刷新，页面变成tomcat2。此时master已启动，虚拟IP回到master，此时nginx1进行分发请求 
【本文地址】
今日新闻 |
推荐新闻 |