kafka工具kafka-topic.sh

朱清震發表於2017-10-08

kafka 工具 kafka-topic.sh

操作

kafka-topic.sh有6種操作：

建立topic – create

[lch@linux129 bin]$ ./kafka-topics.sh --create --zookeeper ip:2181,ip:2181 --replication-factor 2 --partitions 20 --topic TEST

引數說明：
--replication-factor    : 表明 副本的個數， 這裡寫成2，表明會有兩個副本
--partitions              :  表明會有20個partition， 表現在log日誌中，就是有20個資料夾,名稱以“topic-編號” ，編號從0~19

執行完成後去kafka資料目錄下檢視，會看到建立了兩個資料夾

    test-0
    test-1
    test-2

注：在leader節點上面才是20個資料夾，如果是從節點上面，可能只有一個也可能沒有

列出全部的topic –list命令

[root@tingyun bin]# ./kafka-topics.sh --zookeeper=localhost:2181  --list
__consumer_offsets
crashMessage
mobileActions
mobileActionsWrappedMessages
mobileErrors
mobileErrorsWrappedMessages
mobileInteractionTraces
mobileInteractionTracesWrappedMessages
mobileReportUsers
mobileReportUsersWrappedMessages

列出全部topic的資訊 –describe

包括每個分割槽的leader，副本，isr資訊

[root@tingyun bin]# ./kafka-topics.sh --zookeeper=localhost:2181  --describe
Topic:__consumer_offsets    PartitionCount:50   ReplicationFactor:3 Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=uncompressed
    Topic: __consumer_offsets   Partition: 0    Leader: 4   Replicas: 4,0,1 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 1    Leader: 0   Replicas: 0,1,2 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 2    Leader: 1   Replicas: 1,2,3 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 3    Leader: 2   Replicas: 2,3,4 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 4    Leader: 3   Replicas: 3,4,0 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 5    Leader: 4   Replicas: 4,1,2 Isr: 1,4,2
    Topic: __consumer_offsets   Partition: 6    Leader: 0   Replicas: 0,2,3 Isr: 0,3,2
    Topic: __consumer_offsets   Partition: 7    Leader: 1   Replicas: 1,3,4 Isr: 1,3,4
    Topic: __consumer_offsets   Partition: 8    Leader: 2   Replicas: 2,4,0 Isr: 0,4,2
    Topic: __consumer_offsets   Partition: 9    Leader: 3   Replicas: 3,0,1 Isr: 0,1,3
    Topic: __consumer_offsets   Partition: 10   Leader: 4   Replicas: 4,2,3 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 11   Leader: 0   Replicas: 0,3,4 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 12   Leader: 0   Replicas: 1,4,0 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 13   Leader: 2   Replicas: 2,0,1 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 14   Leader: 3   Replicas: 3,1,2 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 15   Leader: 4   Replicas: 4,3,0 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 16   Leader: 0   Replicas: 0,4,1 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 17   Leader: 0   Replicas: 1,0,2 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 18   Leader: 2   Replicas: 2,1,3 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 19   Leader: 3   Replicas: 3,2,4 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 20   Leader: 4   Replicas: 4,0,1 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 21   Leader: 0   Replicas: 0,1,2 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 22   Leader: 1   Replicas: 1,2,3 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 23   Leader: 2   Replicas: 2,3,4 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 24   Leader: 3   Replicas: 3,4,0 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 25   Leader: 4   Replicas: 4,1,2 Isr: 1,4,2
    Topic: __consumer_offsets   Partition: 26   Leader: 0   Replicas: 0,2,3 Isr: 0,3,2
    Topic: __consumer_offsets   Partition: 27   Leader: 1   Replicas: 1,3,4 Isr: 1,3,4
    Topic: __consumer_offsets   Partition: 28   Leader: 2   Replicas: 2,4,0 Isr: 0,4,2
    Topic: __consumer_offsets   Partition: 29   Leader: 3   Replicas: 3,0,1 Isr: 0,1,3
    Topic: __consumer_offsets   Partition: 30   Leader: 4   Replicas: 4,2,3 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 31   Leader: 0   Replicas: 0,3,4 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 32   Leader: 0   Replicas: 1,4,0 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 33   Leader: 2   Replicas: 2,0,1 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 34   Leader: 3   Replicas: 3,1,2 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 35   Leader: 4   Replicas: 4,3,0 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 36   Leader: 0   Replicas: 0,4,1 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 37   Leader: 0   Replicas: 1,0,2 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 38   Leader: 2   Replicas: 2,1,3 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 39   Leader: 3   Replicas: 3,2,4 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 40   Leader: 4   Replicas: 4,0,1 Isr: 0,1,4
    Topic: __consumer_offsets   Partition: 41   Leader: 0   Replicas: 0,1,2 Isr: 0,1,2
    Topic: __consumer_offsets   Partition: 42   Leader: 1   Replicas: 1,2,3 Isr: 1,3,2
    Topic: __consumer_offsets   Partition: 43   Leader: 2   Replicas: 2,3,4 Isr: 3,4,2
    Topic: __consumer_offsets   Partition: 44   Leader: 3   Replicas: 3,4,0 Isr: 0,3,4
    Topic: __consumer_offsets   Partition: 45   Leader: 4   Replicas: 4,1,2 Isr: 1,4,2
    Topic: __consumer_offsets   Partition: 46   Leader: 0   Replicas: 0,2,3 Isr: 0,3,2
    Topic: __consumer_offsets   Partition: 47   Leader: 1   Replicas: 1,3,4 Isr: 1,3,4
    Topic: __consumer_offsets   Partition: 48   Leader: 2   Replicas: 2,4,0 Isr: 0,4,2
    Topic: __consumer_offsets   Partition: 49   Leader: 3   Replicas: 3,0,1 Isr: 0,1,3
Topic:TEST  PartitionCount:20   ReplicationFactor:1 Configs:
    Topic: TEST Partition: 0    Leader: 4   Replicas: 4 Isr: 4
    Topic: TEST Partition: 1    Leader: 0   Replicas: 0 Isr: 0
    Topic: TEST Partition: 2    Leader: 1   Replicas: 1 Isr: 1
    Topic: TEST Partition: 3    Leader: 2   Replicas: 2 Isr: 2
    Topic: TEST Partition: 4    Leader: 3   Replicas: 3 Isr: 3
    Topic: TEST Partition: 5    Leader: 4   Replicas: 4 Isr: 4
    Topic: TEST Partition: 6    Leader: 0   Replicas: 0 Isr: 0
    Topic: TEST Partition: 7    Leader: 1   Replicas: 1 Isr: 1
    Topic: TEST Partition: 8    Leader: 2   Replicas: 2 Isr: 2
    Topic: TEST Partition: 9    Leader: 3   Replicas: 3 Isr: 3
    Topic: TEST Partition: 10   Leader: 4   Replicas: 4 Isr: 4
    Topic: TEST Partition: 11   Leader: 0   Replicas: 0 Isr: 0
    Topic: TEST Partition: 12   Leader: 1   Replicas: 1 Isr: 1
    Topic: TEST Partition: 13   Leader: 2   Replicas: 2 Isr: 2
    Topic: TEST Partition: 14   Leader: 3   Replicas: 3 Isr: 3
    Topic: TEST Partition: 15   Leader: 4   Replicas: 4 Isr: 4
    Topic: TEST Partition: 16   Leader: 0   Replicas: 0 Isr: 0
    Topic: TEST Partition: 17   Leader: 1   Replicas: 1 Isr: 1
    Topic: TEST Partition: 18   Leader: 2   Replicas: 2 Isr: 2
    Topic: TEST Partition: 19   Leader: 3   Replicas: 3 Isr: 3

刪除topic –delete

 [root@hadoop221 bin]# ./kafka-topics.sh  --zookeeper ip:port  --topic TEST  --delete

topic無法刪掉
Topic TEST is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.

需要增加配置引數：delete.topic.enable=true

修改topic時配置引數 –alter –config

覆蓋已經有topic引數

 [root@hadoop221 bin]# ./kafka-topics.sh --zookeeper zk_host:port  --topic TEST --alter --config flush.messages=1

刪除topic級別配置引數

 [root@hadoop221 bin]# ./kafka-topics.sh --zookeeper zk_host:port --alter --topic TEST --delete-config flush.messages=1

修改topic分割槽數量

 [root@hadoop221 bin]# ./kafka-topics.sh --zookeeper zk_host:port --alter --topic TEST --partitions 6

kafka壓測工具
2018-09-05
Kafka
kafka視覺化客戶端工具（Kafka Tool）的基本使用
2022-03-07
Kafka視覺化客戶端
用好kafka，你不得不知的那些工具
2023-05-19
Kafka
provectus/kafka-ui: 開源Apache Kafka的Web GUI圖形介面管理工具
2021-10-02
KafkaApacheWebGUI
形象生動視覺化Kafka工作情景的工具
2022-07-19
視覺化Kafka
Kafka 訊息遷移工具的壓測與調優
2020-10-10
Kafka
Kafka 視覺化監控和管理 UI工具評估
2020-07-04
Kafka視覺化UI
kafka-ngx_kafka_module
2020-11-16
Kafka
模擬測試生成Apache Kafka負載的最佳工具：KLoadGen
2021-06-22
ApacheKafka負載
docker compose of kafka and kafka-ui
2024-08-16
DockerKafkaUI
kafka之一：kafka簡介
2021-06-04
Kafka
kafka 監控工具 eagle 的安裝(內附高速下載地址)
2020-07-13
Kafka
kafka
2024-12-05
Kafka
Apache Kafka教程--Kafka新手入門
2023-03-14
ApacheKafka
一、kafka 介紹 && kafka-client
2020-06-04
Kafkaclient
《Kafka筆記》1、Kafka初識
2020-10-18
Kafka筆記
【Kafka】基於Windows環境的Kafka有關環境(scala+zookeeper+kafka+視覺化工具)搭建、以及使用.NET環境開發的案例程式碼與演示
2022-01-21
KafkaWindows視覺化
Kafka學習（四）-------- Kafka核心之Producer
2019-08-06
Kafka
Kafka學習（三）-------- Kafka核心之Cosumer
2019-07-25
Kafka
kafka學習（二）-------- 什麼是Kafka
2019-07-22
Kafka
《Kafka筆記》3、Kafka高階API
2020-10-20
Kafka筆記API
Kafka 學習筆記（二）：初探 Kafka
2019-03-04
Kafka筆記
Kafka監控系統Kafka Eagle剖析
2018-07-27
Kafka
Kafka學習筆記（二）：初探Kafka
2018-03-26
Kafka筆記
confluent kafka
2024-10-23
Kafka
kafka命令
2024-08-27
Kafka
Kafka 概述
2021-11-14
Kafka
docker kafka
2021-04-28
DockerKafka
Flume 整合 Kafka_flume 到kafka 配置【轉】
2024-04-15
Kafka
kafka的Docker映象使用說明(wurstmeister/kafka)
2022-07-25
KafkaDocker
Kafka 效能篇：為何 Kafka 這麼快？
2021-09-26
Kafka
Clickhouse Engine kafka 將kafka資料同步clickhouse
2020-12-03
Kafka
詳細解析kafka之kafka分割槽和副本
2021-09-09
Kafka
Kafka面試題——20道Kafka知識點
2020-06-29
Kafka面試題
【kafka學習筆記】kafka的基本概念
2021-12-12
Kafka筆記
【Kafka】《Kafka權威指南》——提交和偏移量
2019-06-06
Kafka
Kafka 入門（四）-- Python Kafka Client 效能測試
2021-01-16
KafkaPythonclient
kafka報錯解決 kafka.errors.NoBrokersAvailable: NoBrokersAvailable
2020-11-23
KafkaErrorAI
Flink-Kafka-Connector Flink結合Kafka實戰
2019-03-03
Kafka