中工信融网站建设,杭州网站建设设计公司哪家好,wordpress怎么修改文字大小,百度网盘登录入口 网页文章目录 主机准备部署编辑 docker-compose.ymlcontrollerbroker生成cluster_id 一篇完整的 docker-compose.yml 文件查看集群状态使用 kafka-ui 查看拉取 kafka-ui添加集群查看集群状态 使用命令行查看 配置讲解controllerbroker 主机准备
IPcontroller idbroker id192.168.1… 文章目录 主机准备部署编辑 docker-compose.ymlcontrollerbroker生成cluster_id 一篇完整的 docker-compose.yml 文件查看集群状态使用 kafka-ui 查看拉取 kafka-ui添加集群查看集群状态 使用命令行查看 配置讲解controllerbroker 主机准备
IPcontroller idbroker id192.168.142.15715192.168.142.15626192.168.142.15537192.168.142.15848
部署
创建文件夹
mkdir -p /docker/kafka/{data,data1,data2}chmod 777 -R {data,data1,data2}编辑 docker-compose.yml
controller
docker-compose.yml
services:controller:image: bitnami/kafka:3.8.0container_name: controllernetwork_mode: hostrestart: alwaysvolumes:- /docker/kafka/data:/data- /etc/localtime:/etc/localtime:roenvironment:KAFKA_CFG_NODE_ID: 1KAFKA_CFG_PROCESS_ROLES: controllerKAFKA_CFG_LISTENERS: CONTROLLER://192.168.142.157:9093KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXTKAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLERKAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1192.168.142.157:9093,2192.168.142.156:9093,3192.168.142.155:9093,4192.168.142.158:9093KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS: 0KAFKA_CFG_LOG_DIRS: /databroker
image: bitnami/kafka:3.8.0container_name: brokernetwork_mode: hostrestart: alwaysvolumes:- /etc/localtime:/etc/localtime:ro- /docker/kafka/data1:/data1- /docker/kafka/data2:/data2environment:KAFKA_CFG_NODE_ID: 5KAFKA_CFG_PROCESS_ROLES: brokerKAFKA_CFG_LISTENERS: PLAINTEXT://192.168.142.157:9092KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://192.168.142.157:9092KAFKA_CFG_INTER_BROKER_LISTENER_NAME: PLAINTEXTKAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLERKAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXTKAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1192.168.142.157:9093,2192.168.142.156:9093,3192.168.142.155:9093,4192.168.142.158:9093KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS: 0KAFKA_CFG_LOG_DIRS: /data1,/data2所有的主机这一段基本都是一样的 只有 id 不同id 可以根据上面表格进行修改 生成cluster_id
rootmaster:/docker/kafka# docker run -it --rm bitnami/kafka:3.8.0 kafka-storage.sh random-uuid
kafka 06:43:36.79 INFO
kafka 06:43:36.80 INFO Welcome to the Bitnami kafka container
kafka 06:43:36.80 INFO Subscribe to project updates by watching https://github.com/bitnami/containers
kafka 06:43:36.80 INFO Submit issues and feature requests at https://github.com/bitnami/containers/issues
kafka 06:43:36.80 INFO Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
kafka 06:43:36.80 INFO PavLImfXSqaOOdrMgq1teA拿到 id PavLImfXSqaOOdrMgq1teA 新增到 docker-compose.yml 文件中
一篇完整的 docker-compose.yml 文件
services:controller:image: bitnami/kafka:3.8.0container_name: controllernetwork_mode: hostrestart: alwaysvolumes:- /docker/kafka/data:/data- /etc/localtime:/etc/localtime:roenvironment:KAFKA_CFG_NODE_ID: 1KAFKA_CFG_PROCESS_ROLES: controllerKAFKA_CFG_LISTENERS: CONTROLLER://192.168.142.157:9093KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXTKAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLERKAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1192.168.142.157:9093,2192.168.142.156:9093,3192.168.142.155:9093,4192.168.142.158:9093KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS: 0KAFKA_CFG_LOG_DIRS: /dataKAFKA_KRAFT_CLUSTER_ID: PavLImfXSqaOOdrMgq1teAbroker:image: bitnami/kafka:3.8.0container_name: brokernetwork_mode: hostrestart: alwaysvolumes:- /etc/localtime:/etc/localtime:ro- /docker/kafka/data1:/data1- /docker/kafka/data2:/data2environment:KAFKA_CFG_NODE_ID: 5KAFKA_CFG_PROCESS_ROLES: brokerKAFKA_CFG_LISTENERS: PLAINTEXT://192.168.142.157:9092KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://192.168.142.157:9092KAFKA_CFG_INTER_BROKER_LISTENER_NAME: PLAINTEXTKAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLERKAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXTKAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1192.168.142.157:9093,2192.168.142.156:9093,3192.168.142.155:9093,4192.168.142.158:9093KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS: 0KAFKA_CFG_LOG_DIRS: /data1,/data2KAFKA_KRAFT_CLUSTER_ID: PavLImfXSqaOOdrMgq1teA启动
docker compose up -d查看启动状态
docker ps -a | grep Up | wc -l
2查看集群状态
使用 kafka-ui 查看
拉取 kafka-ui
只需要在一台主机上部署 ui 就可以了
docker run -d -p 8080:8080 -e DYNAMIC_CONFIG_ENABLEDtrue provectuslabs/kafka-ui:v0.7.1 通过 ip 8080 端口访问 添加集群 查看集群状态 使用命令行查看
rootmaster:/docker/kafka# docker exec -it broker ./opt/bitnami/kafka/bin/kafka-metadata-quorum.sh --bootstrap-controller 192.168.142.157:9093,192.168.142.156:9093,192.168.142.155:9093,192.168.142.158:9093 describe --status
ClusterId: PavLImfXSqaOOdrMgq1teA
LeaderId: 1
LeaderEpoch: 22
HighWatermark: 7695
MaxFollowerLag: 0
MaxFollowerLagTimeMs: 0
CurrentVoters: [1,2,3,4]
CurrentObservers: [5,6,7,8]到此集群搭建完成
配置讲解
controller
参数解释KAFKA_CFG_NODE_ID节点 idKAFKA_CFG_PROCESS_ROLES节点角色KAFKA_CFG_LISTENERS节点监听的 ip 和端口并且标记为 controllerKAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP定义不同监听器使用的加密协议KAFKA_CFG_CONTROLLER_LISTENER_NAMES控制器Controller用于接收来自其他控制器节点和 broker 节点请求的监听器Listener的名称KAFKA_CFG_CONTROLLER_QUORUM_VOTERS添加集群节点KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS当一个新的消费者加入到一个空的消费组时协调者Coordinator将延迟多长时间再开始重新平衡Rebalance的过程KAFKA_CFG_LOG_DIRS日志目录KAFKA_KRAFT_CLUSTER_ID集群 id
broker
参数解释KAFKA_CFG_NODE_ID节点 idKAFKA_CFG_PROCESS_ROLES节点角色KAFKA_CFG_LISTENERS节点监听的 ip 和端口并且标记为 brokerKAFKA_CFG_INTER_BROKER_LISTENER_NAME集群中 broker 之间通信所使用的监听器名称KAFKA_CFG_CONTROLLER_LISTENER_NAMES监听器的名称KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP定义不同监听器使用的加密协议KAFKA_CFG_CONTROLLER_QUORUM_VOTERS添加集群节点KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS当一个新的消费者加入到一个空的消费组时协调者Coordinator将延迟多长时间再开始重新平衡Rebalance的过程KAFKA_CFG_LOG_DIRS日志目录KAFKA_KRAFT_CLUSTER_ID集群 id 文章转载自： http://www.morning.tfrlj.cn.gov.cn.tfrlj.cn http://www.morning.clkyw.cn.gov.cn.clkyw.cn http://www.morning.gkfwp.cn.gov.cn.gkfwp.cn http://www.morning.tbhf.cn.gov.cn.tbhf.cn http://www.morning.trnl.cn.gov.cn.trnl.cn http://www.morning.httzf.cn.gov.cn.httzf.cn http://www.morning.wfkbk.cn.gov.cn.wfkbk.cn http://www.morning.ypfw.cn.gov.cn.ypfw.cn http://www.morning.clhyj.cn.gov.cn.clhyj.cn http://www.morning.xtqld.cn.gov.cn.xtqld.cn http://www.morning.mjkqj.cn.gov.cn.mjkqj.cn http://www.morning.mjzcp.cn.gov.cn.mjzcp.cn http://www.morning.qiyelm.com.gov.cn.qiyelm.com http://www.morning.wnhgb.cn.gov.cn.wnhgb.cn http://www.morning.mtjwp.cn.gov.cn.mtjwp.cn http://www.morning.mmjyk.cn.gov.cn.mmjyk.cn http://www.morning.bbrf.cn.gov.cn.bbrf.cn http://www.morning.gkxyy.cn.gov.cn.gkxyy.cn http://www.morning.yxzfl.cn.gov.cn.yxzfl.cn http://www.morning.lkkgq.cn.gov.cn.lkkgq.cn http://www.morning.kflzy.cn.gov.cn.kflzy.cn http://www.morning.kwblwbl.cn.gov.cn.kwblwbl.cn http://www.morning.lwzgn.cn.gov.cn.lwzgn.cn http://www.morning.ktbjk.cn.gov.cn.ktbjk.cn http://www.morning.rgxcd.cn.gov.cn.rgxcd.cn http://www.morning.tkfnp.cn.gov.cn.tkfnp.cn http://www.morning.lgnrl.cn.gov.cn.lgnrl.cn http://www.morning.bwmq.cn.gov.cn.bwmq.cn http://www.morning.sjbty.cn.gov.cn.sjbty.cn http://www.morning.qrcxh.cn.gov.cn.qrcxh.cn http://www.morning.hmmtx.cn.gov.cn.hmmtx.cn http://www.morning.lptjt.cn.gov.cn.lptjt.cn http://www.morning.ntwfr.cn.gov.cn.ntwfr.cn http://www.morning.ykqbs.cn.gov.cn.ykqbs.cn http://www.morning.kqxwm.cn.gov.cn.kqxwm.cn http://www.morning.mlfmj.cn.gov.cn.mlfmj.cn http://www.morning.dppfh.cn.gov.cn.dppfh.cn http://www.morning.zrfwz.cn.gov.cn.zrfwz.cn http://www.morning.ntqlz.cn.gov.cn.ntqlz.cn http://www.morning.incmt.com.gov.cn.incmt.com http://www.morning.brqjs.cn.gov.cn.brqjs.cn http://www.morning.lxlzm.cn.gov.cn.lxlzm.cn http://www.morning.rpstb.cn.gov.cn.rpstb.cn http://www.morning.qphcq.cn.gov.cn.qphcq.cn http://www.morning.mzwfw.cn.gov.cn.mzwfw.cn http://www.morning.ybgyz.cn.gov.cn.ybgyz.cn http://www.morning.qbccg.cn.gov.cn.qbccg.cn http://www.morning.bzkgn.cn.gov.cn.bzkgn.cn http://www.morning.lpqgq.cn.gov.cn.lpqgq.cn http://www.morning.wddmr.cn.gov.cn.wddmr.cn http://www.morning.nmlpp.cn.gov.cn.nmlpp.cn http://www.morning.bkqw.cn.gov.cn.bkqw.cn http://www.morning.xxgfl.cn.gov.cn.xxgfl.cn http://www.morning.prplf.cn.gov.cn.prplf.cn http://www.morning.yzxlkj.com.gov.cn.yzxlkj.com http://www.morning.ngjpt.cn.gov.cn.ngjpt.cn http://www.morning.qlrwf.cn.gov.cn.qlrwf.cn http://www.morning.wyctq.cn.gov.cn.wyctq.cn http://www.morning.wcczg.cn.gov.cn.wcczg.cn http://www.morning.prgyd.cn.gov.cn.prgyd.cn http://www.morning.grbgn.cn.gov.cn.grbgn.cn http://www.morning.qrcxh.cn.gov.cn.qrcxh.cn http://www.morning.zynjt.cn.gov.cn.zynjt.cn http://www.morning.pxtgf.cn.gov.cn.pxtgf.cn http://www.morning.glnfn.cn.gov.cn.glnfn.cn http://www.morning.wylpy.cn.gov.cn.wylpy.cn http://www.morning.mpbgy.cn.gov.cn.mpbgy.cn http://www.morning.clzly.cn.gov.cn.clzly.cn http://www.morning.hdrrk.cn.gov.cn.hdrrk.cn http://www.morning.ncwgt.cn.gov.cn.ncwgt.cn http://www.morning.spghj.cn.gov.cn.spghj.cn http://www.morning.wzdjl.cn.gov.cn.wzdjl.cn http://www.morning.drmbh.cn.gov.cn.drmbh.cn http://www.morning.dgwrz.cn.gov.cn.dgwrz.cn http://www.morning.tqrbl.cn.gov.cn.tqrbl.cn http://www.morning.jpgfq.cn.gov.cn.jpgfq.cn http://www.morning.wmsgt.cn.gov.cn.wmsgt.cn http://www.morning.phgz.cn.gov.cn.phgz.cn http://www.morning.tjwfk.cn.gov.cn.tjwfk.cn http://www.morning.kdrjd.cn.gov.cn.kdrjd.cn
