使用Nginx实现TDengine的负载均衡 |
您所在的位置：网站首页 › nginx负载均衡的实现 › 使用Nginx实现TDengine的负载均衡 |
|
目录
实验环境1.Nginx安装配置1.1.安装依赖包1.2.安装Nginx1.3.配置负载均衡1.4.启动Nginx
2.TDengine集群安装2.1. 安装TDengine2.2. 安装Arbitrator2.3. 配置文件taos.cfg2.4.启动集群2.5.安装客户端
3.测试4.问题
RESTful方式见 使用Nginx实现TDengine的负载均衡-基于RESTful 实验环境服务器： 172.16.216.2 test1 ##TDengine Server 1 & Arbitrator 172.16.216.3 test2 ##TDengine Server 2 172.16.216.4 test3 ##Nginx 服务器 **操作系统：**CentOS Linux release 7.9 软件版本： nginx 1.20.0 TDengine 2.0.20.0 1.Nginx安装配置 1.1.安装依赖包 yum install gcc gcc-c++ autoconf automake yum install pcre-devel yum install zlib-devel 1.2.安装NginxNaginx在1.9.0以后支持TCP负载均衡，如果要Naginx支持TCP/UDP，需要安装stream模块。 ./configure --with-stream make make install 1.3.配置负载均衡TDengine客户端与服务端通信端口为6030-6034，使用TCP和UDP协议，详见官方文档。 vi /usr/local/nginx/conf/nginx.conf worker_processes 1; events { worker_connections 1024; } stream { } server { listen 6030; proxy_pass app6030; } upstream app6030 { server 172.16.216.2:6030; server 172.16.216.3:6030; } server { listen 6031; proxy_pass app6031; } upstream app6031 { server 172.16.216.2:6031; server 172.16.216.3:6031; } server { listen 6032; proxy_pass app6032; } upstream app6032 { server 172.16.216.2:6032; server 172.16.216.3:6032; } server { listen 6033; proxy_pass app6033; } upstream app6033 { server 172.16.216.2:6033; server 172.16.216.3:6033; } server { listen 6034; proxy_pass app6034; } upstream app6034 { server 172.16.216.2:6034; server 172.16.216.3:6034; } server { listen 6030 udp; proxy_pass uapp6030; } upstream uapp6030 { server 172.16.216.2:6030; server 172.16.216.3:6030; } server { listen 6031 udp; proxy_pass uapp6031; } upstream uapp6031 { server 172.16.216.2:6031; server 172.16.216.3:6031; } server { listen 6032 udp; proxy_pass uapp6032; } upstream uapp6032 { server 172.16.216.2:6032; server 172.16.216.3:6032; } server { listen 6033 udp; proxy_pass uapp6033; } upstream uapp6033 { server 172.16.216.2:6033; server 172.16.216.3:6033; } server { listen 6034 udp; proxy_pass uapp6034; } upstream uapp6034 { server 172.16.216.2:6034; server 172.16.216.3:6034; } } 1.4.启动Nginx /usr/local/nginx/sbin/nginx [root@test3 logs]# netstat -ant Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 0.0.0.0:6031 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:6032 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:6033 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:6034 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:6041 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:6030 0.0.0.0:* LISTEN tcp 0 0 172.16.216.4:22 172.16.216.1:62439 ESTABLISHED tcp6 0 0 :::22 :::* LISTEN tcp6 0 0 ::1:25 :::* LISTEN 2.TDengine集群安装具体安装步骤可见 TDengine学习笔记-集群安装 2.1. 安装TDengine tar xvzf TDengine-server-2.0.20.0-Linux-x64.tar.gz cd TDengine-server-2.0.20.0 ./install 2.2. 安装ArbitratorTDengine中的Arbitrator 类似于PostgreSQL中的witness服务，防止集群出现脑裂。因为实验只有2个节点，为防止停止一个节点后，无法通过无法，需要安装Arbitrator服务。 官方文档-Arbitrator的使用 tar xvzf TDengine-arbitrator-2.0.20.0-Linux-x64.tar.gz cd TDengine-arbitrator-2.0.20.0 ./install_arbi.sh systemctl start tarbitratord 2.3. 配置文件taos.cfg ##test1 firstEp test1:6030 secondEp test2:6030 arbitrator test1:6042 fqdn test1 ##test2 firstEp test1:6030 secondEp test2:6030 arbitrator test1:6042 fqdn test2 2.4.启动集群 systemctl start taosd添加节点 taos> create dnode test2:6030; taos> show dnodes; id | end_point | vnodes | cores | status | role | create_time | offline reason | ====================================================================================================================================== 1 | test1:6030 | 1 | 8 | ready | any | 2021-05-05 17:20:15.802 | | 2 | test2:6030 | 1 | 8 | ready | any | 2021-05-05 17:20:30.431 | | 0 | test1:6042 | 0 | 0 | ready | arb | 1970-01-01 08:00:00.000 | - | Query OK, 3 row(s) in set (0.004701s) 2.5.安装客户端在nginx服务器上安装TDengine客户端，模拟客户连接。 tar xzvf TDengine-enterprise-client-2.0.20.0-Linux-x64.tar.gz cd TDengine-enterprise-client-2.0.20.0 ./install_client.sh配置客户端文件taos.cfg。注意：这里的firstEP要配置成本地。 /etc/taos/taos.cfg firstEp test3:6030 fqdn test3 3.测试在test3上启动客户端连接TDengine集群。 [root@test3 ~]# taos Welcome to the TDengine shell from Linux, Client Version:2.0.20.0 Copyright (c) 2020 by TAOS Data, Inc. All rights reserved. taos> exit [root@test3 ~]# taos -h 172.16.216.4 Welcome to the TDengine shell from Linux, Client Version:2.0.20.0 Copyright (c) 2020 by TAOS Data, Inc. All rights reserved. taos> show dnodes; id | end_point | vnodes | cores | status | role | create_time | offline reason | ====================================================================================================================================== 1 | test1:6030 | 1 | 8 | ready | any | 2021-05-05 17:20:15.802 | | 2 | test2:6030 | 1 | 8 | ready | any | 2021-05-05 17:20:30.431 | | 0 | test1:6042 | 0 | 0 | ready | arb | 1970-01-01 08:00:00.000 | - | Query OK, 3 row(s) in set (0.007414s)停止节点test1上TDengine，在test3上重新连接集群。 [root@test1 ~]# systemctl stop taosd [root@test1 ~]# ps -ef | grep taos root 35454 12164 0 20:59 pts/0 00:00:00 grep --color=auto taos [root@test3 ~]# taos Welcome to the TDengine shell from Linux, Client Version:2.0.20.0 Copyright (c) 2020 by TAOS Data, Inc. All rights reserved. taos> show dnodes; id | end_point | vnodes | cores | status | role | create_time | offline reason | ====================================================================================================================================== 1 | test1:6030 | 1 | 8 | offline | any | 2021-05-05 17:20:15.802 | status msg timeout | 2 | test2:6030 | 1 | 8 | ready | any | 2021-05-05 17:20:30.431 | | 0 | test1:6042 | 0 | 0 | ready | arb | 1970-01-01 08:00:00.000 | - | Query OK, 3 row(s) in set (0.008756s)Nginx日志显示172.16.216.2无法联通。 2021/05/05 21:00:00 [error] 9706#0: *9 recv() failed (111: Connection refused) while proxying and reading from upstream, udp client: 172.16.216.4, server: 0.0.0.0:6030, upstream: "172.16.216.2:6030", bytes from/to client:377/0, bytes from/to upstream:0/377 2021/05/05 21:00:02 [error] 9706#0: *9 sendmsg() failed (111: Connection refused) while proxying and sending to upstream, udp client: 172.16.216.4, server: 0.0.0.0:6030, upstream: "172.16.216.2:6030", bytes from/to client:1131/0, bytes from/to upstream:0/754 4.问题采用客户端taosc进行连接时，均能连接正常。但使用TDengine提供的测试工具taosdemo进行测试时，偶尔会报错，如下： create database db01 success! ERROR: startMultiThreadCreateChildTable() LN2743, Failed to connect to TDengine, reason:Unable to establish connection Spent 4.0241 seconds to create 10 tables with 10 thread(s) WARNING: offset and limit will not be used since the child tables are not exists! ERROR: connect to server fail from insert sub thread, reason: Unable to establish connection报错信息显示数据库创建成功，但无法创建表。 |
| CopyRight 2018-2019 办公设备维修网 版权所有 豫ICP备15022753号-3 |