Kafka brokers移出集群并重新分配失败后降低主题复制因子 |
您所在的位置:网站首页 › kafka减少副本数 › Kafka brokers移出集群并重新分配失败后降低主题复制因子 |
当集群中可用的Kafka代理数量为40个时,主题复制因子增加到45。 这是由于已停止的重复阻塞分区重新分配造成的。 kafka-topics --topic top --zookeeper zoo_url --describe 显示 Partition: 0 Leader: 20464 Replicas: 20464,20765,1882,20870,873,898,20752,16789,17181,20743,20854,20762,894,20459,20851,21070,20757,20766,20763,890,21173,20852,895,21314,20767,883,20467,16787,21071,20750,887,20760,7067,876,20764,891,20768,4880,20769,16788,20756,886,21172,1582,871,16827 Isr: 20464,20765,1882,20870,873,898,20752,16789,17181,20743,20762,894,20459,21070,20757,20766,20763,890,21173,895,21314,20767,883,20467,16787,20750,887,20760,7067,876,20764,891,20768,4880,20769,16788,20756,886,21172,871,16827 ... 某些副本不是群集的一部分。 运行: kafka-reassign-partitions --zookeeper zoo_url --topics-to-move-json-file assign.json --generate --broker-list ... 失败,出现以下错误 Partitions reassignment failed due to replication factor: 45 larger than available brokers: 21 kafka.admin.AdminOperationException: replication factor: 45 larger than available brokers: 21 at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:117) at kafka.admin.ReassignPartitionsCommand$$anonfun$generateAssignment$1.apply(ReassignPartitionsCommand.scala:110) at kafka.admin.ReassignPartitionsCommand$$anonfun$generateAssignment$1.apply(ReassignPartitionsCommand.scala:108) at scala.collection.immutable.Map$Map1.foreach(Map.scala:116) at kafka.admin.ReassignPartitionsCommand$.generateAssignment(ReassignPartitionsCommand.scala:108) at kafka.admin.ReassignPartitionsCommand$.generateAssignment(ReassignPartitionsCommand.scala:91) at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:50) at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)--broker-list参数输入是在线经纪人的ids。 如何强制降低主题复制因子? 唯一有效的解决方案是通过运行以下命令来降低一个分区的复制因子 kafka-reassign-partitions --zookeeper zoo_url --reassignment-json-file /tmp/assign.json --execute 当/tmp/assign.json如下所示时 { "partitions": [ { "partition": 0, "replicas": [20743,20762,894,20459,20757,895,20467,20760], "topic": "topic" } ], "version": 1 } 然后重新运行分区分配(生成分配并执行它) Kafka 0.9.0.1作为Cloudera的一部分进行部署。 |
今日新闻 |
推荐新闻 |
CopyRight 2018-2019 办公设备维修网 版权所有 豫ICP备15022753号-3 |