kubernetes（k8s）多master集群和负载均衡部署 |
您所在的位置：网站首页 › 多个nginx负载 › kubernetes（k8s）多master集群和负载均衡部署 |
kubernetes（k8s）多master集群和负载均衡部署
|
kubernetes（k8s）多master集群和负载均衡部署 目录 一、多master节点集群搭建示意图二、部署多master节点2.1 在master1上拷贝重要文件给master2 三、搭建nginx负载均衡3.1 安装nginx服务3.2 搭建 keepalived 高可用服务3.3 开始修改node节点配置文件统一VIP 一、多master节点集群搭建示意图
我的虚拟机 IP地址规划： Master节点 master01：192.168.126.10 master02：192.168.126.40 VIP 地址：192.168.126.88Node节点 node01：192.168.126.20 node02：192.168.126.30负载均衡 Nginx01：192.168.126.50 master Nginx02：192.168.126.60 backup 本篇博客将在单master节点的K8s集群上搭建 多master节点集群 和 LB负载均衡服务器。 二、部署多master节点部署多节点，在单节点的基础上部署，前期博文有单节点部署的详细步骤 2.1 在master1上拷贝重要文件给master2优先关闭防火墙和selinux服务 master01上操作 复制kubernetes目录到master02 [root@localhost k8s]# scp -r /opt/kubernetes/ [email protected]:/opt 复制master中的三个组件启动脚本kube-apiserver.service kube-controller-manager.service kube-scheduler.service [root@localhost k8s]# scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service [email protected]:/usr/lib/systemd/system/ master02上操作修改配置文件kube-apiserver中的IP [root@localhost ~]# cd /opt/kubernetes/cfg/ [root@localhost cfg]# vim kube-apiserver KUBE_APISERVER_OPTS="--logtostderr=true \ --v=4 \ --etcd-servers=https://192.168.195.149:2379,https://192.168.195.150:2379,https://192.168.195.151:2379 \ --bind-address=192.168.126.40 \ #修改为本机ip --secure-port=6443 \ --advertise-address=192.168.126.40 \ #修改为本机ip --allow-privileged=true \ --service-cluster-ip-range=10.0.0.0/24 \ --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,No 特别注意：master02一定要有etcd证书master2节点与node节点之间的操作信息需要保存在ETCD集群中，所以 master 节点都必须有etcd证书，实现通信。 拷贝master1上已有的 etcd 证书给 master2 使用 [root@localhost k8s]# scp -r /opt/etcd/ [email protected]:/opt/ [email protected]'s password: 启动master02中的三个组件服务 [root@localhost cfg]# systemctl start kube-apiserver.service [root@localhost cfg]# systemctl start kube-controller-manager.service [root@localhost cfg]# systemctl start kube-scheduler.service 增加环境变量，优化kubectl命令 [root@localhost cfg]# vim /etc/profile \#末尾添加 export PATH=$PATH:/opt/kubernetes/bin/ [root@localhost cfg]# source /etc/profile 验证master2是否加入K8s集群 [root@localhost cfg]# kubectl get node NAME STATUS ROLES AGE VERSION 192.168.195.150 Ready 2d12h v1.12.3 192.168.195.151 Ready 38h v1.12.3 三、搭建nginx负载均衡 3.1 安装nginx服务准备两台虚拟机 搭建 nginx的高可用群集 负载均衡（两台操作一样） Nginx01：192.168.126.50 master Nginx02：192.168.126.60 backup 安装nginx服务，把nginx.sh和keepalived.conf脚本拷贝到家目录 [root@localhost ~]#ls anaconda-ks.cfg initial-setup-ks.cfg keepalived.conf nginx.sh original-ks.cfg yum.sh 公共 模板 视频 图片 文档 下载 音乐 桌面 关闭防火墙 [root@localhost ~]# systemctl stop firewalld.service [root@localhost ~]# setenforce 0 建立本地yum官方nginx源//编写repo文件 [root@localhost ~]# vim /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/7/$basearch/ gpgcheck=0 [root@localhost ~]# yum list [root@localhost ~]# yum install nginx -y //下载nginx 接下来在配置文件设置 nginx的四层负载均衡 [root@localhost ~]# vim /etc/nginx/nginx.conf events { worker_connections 1024; } #在这下面添加以下内容 stream { log_format main '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent'; access_log /var/log/nginx/k8s-access.log main; upstream k8s-apiserver { server 192.168.195.149:6443; server 192.168.195.131:6443; } server { listen 6443; proxy_pass k8s-apiserver; } } [root@localhost ~]# systemctl start nginx #启动nginx服务 [root@localhost ~]# nginx -t #检查配置文件是否有语法错误 [root@localhost ~]# systemctl start nginx 开启服务 [root@localhost ~]# netstat -natp | grep nginx #注意：查看端口6443必须在监听中 3.2 搭建 keepalived 高可用服务 部署keepalived服务 [root@localhost ~]# yum install keepalived -y 修改配置文件 cp keepalived.conf /etc/keepalived/keepalived.conf vim /etc/keepalived/keepalived.conf注意：nginx01是Mster配置如下：（注意master和backup配置的区别） ! Configuration File for keepalived global_defs { \# 接收邮件地址 notification_email { [email protected] [email protected] [email protected] } \# 邮件发送地址 notification_email_from [email protected] smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id NGINX_MASTER } vrrp_script check_nginx { script "/etc/nginx/check_nginx.sh" #把脚本位置改一下 } vrrp_instance VI_1 { state MASTER #类型为MASTER interface ens33 #网卡改成ens33 virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的 priority 100 # 优先级，备服务器设置 90 advert_int 1 # 指定VRRP 心跳包通告间隔时间，默认1秒 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.195.88/24 #记得更改虚拟ip地址为之前设置好的地址 } track_script { check_nginx注意：nginx02是Backup配置如下： ! Configuration File for keepalived global_defs { \# 接收邮件地址 notification_email { [email protected] [email protected] [email protected] } \# 邮件发送地址 notification_email_from [email protected] smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id NGINX_MASTER } vrrp_script check_nginx { script "/etc/nginx/check_nginx.sh" #脚本位置 } vrrp_instance VI_1 { state BACKUP #备服务器 interface ens33 #网卡ens33 virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的 priority 90 # 优先级，备服务器设置 90 advert_int 1 # 指定VRRP 心跳包通告间隔时间，默认1秒 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.195.88/24 #虚拟ip的地址 } track_script { check_nginx } } 创建检测脚本 [root@localhost ~]# vim /etc/nginx/check_nginx.sh count=$(ps -ef |grep nginx |egrep -cv "grep|$$") if [ "$count" -eq 0 ];then systemctl stop keepalived fi [root@localhost ~]# chmod +x /etc/nginx/check_nginx.sh //授权 开启keepalived 服务 [root@localhost ~]# systemctl start keepalived 查看nginx01地址信息 [root@localhost ~]# ip a
验证地址漂移（nginx01中使用pkill nginx，再在nginx02中使用ip a 查看） 恢复操作（在nginx01中先启动nginx服务，再启动keepalived服务） nginx站点/usr/share/nginx/html 3.3 开始修改node节点配置文件统一VIP 修改三个文件（bootstrap.kubeconfig,kubelet.kubeconfig） [root@localhost cfg]# vim /opt/kubernetes/cfg/bootstrap.kubeconfig [root@localhost cfg]# vim /opt/kubernetes/cfg/kubelet.kubeconfig [root@localhost cfg]# vim /opt/kubernetes/cfg/kube-proxy.kubeconfig 统统修改为VIP server: https://192.168.126.88:6443 [root@localhost cfg]# systemctl restart kubelet.service [root@localhost cfg]# systemctl enable kubelet.service [root@localhost cfg]# systemctl restart kube-proxy.service [root@localhost cfg]# systemctl enable kube-proxy.service 替换完成直接自检 [root@localhost cfg]# grep 100 * bootstrap.kubeconfig: server: https://192.126.88.100:6443 kubelet.kubeconfig: server: https://192.168.126.88:6443 kube-proxy.kubeconfig: server: https://192.168.126.88:6443 nginx01上查看nginx的k8s日志 [root@localhost ~]# tail /var/log/nginx/k8s-access.log [root@localhost ~]#tail /var/log/nginx/k8s-access.log 192.168.126.20 192.168.126.40:6443 - [14/Apr/2021:18:19:17 +0800] 200 1120 192.168.126.20 192.168.126.10:6443 - [14/Apr/2021:18:19:17 +0800] 200 1121 #轮询调度把请求流量分发到两台master上 master01上操作测试创建pod [root@localhost ~]# kubectl run nginx --image=nginx kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead. deployment.apps/nginx created查看状态 [root@localhost ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-dbddb74b8-nf9sk 0/1 ContainerCreating 0 33s //正在创建中 [root@localhost ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-dbddb74b8-nf9sk 1/1 Running 0 80s //创建完成，运行中 注意日志问题 [root@localhost ~]# kubectl logs nginx-dbddb74b8-nf9sk Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-dbddb74b8-nf9sk) #出现报错，需要创建system:anonymous用户才可以 [root@localhost ~]# kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous clusterrolebinding.rbac.authorization.k8s.io/cluster-system-anonymous created 查看pod网络 [root@localhost dashboard]#kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE nginx-dbddb74b8-7tkhw 1/1 Running 2 17h 172.17.52.2 192.168.126.20 在对应的node节点上操作可以直接访问 curl 172.17.52.2 [root@localhost ~]#curl 172.17.52.2 Welcome to nginx! body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } Welcome to nginx! If you see this page, the nginx web server is successfully installed and working. Further configuration is required. For online documentation and support please refer to nginx.org. Commercial support is available at nginx.com. Thank you for using nginx. 访问就会产生日志//回到master01操作 [root@localhost ~]# kubectl logs nginx-dbddb74b8-7tkhw
|
【本文地址】
今日新闻 |
推荐新闻 |