kafka常用命令

上一篇 / 下一篇  2019-01-02 17:23:04 / 个人分类:hadoop

发送数据:
/usr/hdp/2.6.3.0-235/kafka/bin/kafka-console-producer.sh --broker-list  10.204.125.185:6667 --topic big_dataplatform_get_receive

接收数据命令:
/usr/hdp/2.6.3.0-235/kafka/bin/kafka-console-consumer.sh --zookeeper  10.204.125.185:2181 10.204.125.186:2181 --topic big_dataplatform_get_receive --from-beginning



启动:
cd /usr/hdp/2.6.3.0-235/kafka/


nohup bin/kafka-server-start.sh config/server.properties &

bin/kafka-server-stop.sh



优化:

/etc/kafka/conf
vi server.properties
message.max.bytes=10000120
offsets.topic.segment.bytes=1048576
replica.fetch.max.bytes=10485760
log.roll.hours=168


vi producer.properties 
max.request.size=10485760


新增toptic
cd /usr/hdp/2.6.3.0-235/kafka

bin/kafka-topics.sh --zookeeper 10.204.125.185:2181 10.204.125.186:2181 --create --topic test0102 --partitions 2  --replication-factor 1

查看toptic
 bin/kafka-topics.sh --zookeeper 10.204.125.185:2181 10.204.125.186:2181 --list

删除toptic
bin/kafka-topics.sh --delete --zookeeper 10.204.125.185:2181 10.204.125.186:2181 --topic test0102

注:
并没有真正删除,如果要真正删除
配置delete.topic.enable=true(配置文件在kafka\config目录)

查看指定toptic
 bin/kafka-topics.sh --zookeeper 10.204.125.185:2181 10.204.125.186:2181 --topic heartbeat --describe


修改toptic分区
 bin/kafka-topics.sh --zookeeper 10.204.125.185:2181 10.204.125.186:2181 --topic heartbeat -alter  --partitions 3



查看指定toptic分区和副本情况
bin/kafka-topics.sh --describe --zookeeper 10.204.125.185:2181 10.204.125.186:2181  --topic front-result

查看消费者组
bin/kafka-consumer-groups.sh --zookeeper 10.204.125.185:2181 10.204.125.186:2181 --list

删除消费组
 bin/kafka-consumer-groups.sh --zookeeper 10.204.125.185:2181 10.204.125.186:2181 --delete --group 43124

查看特定consumer group 详情

bin/kafka-consumer-groups.sh --zookeeper 10.204.125.185:2181 10.204.125.186:2181 --group console-consumer-43124 --describe

TAG:

 

评分:0

我来说两句

显示全部

:loveliness: :handshake :victory: :funk: :time: :kiss: :call: :hug: :lol :'( :Q :L ;P :$ :P :o :@ :D :( :)

Open Toolbar