Kafka tools

List kafka consumer groups:

kafka-consumer-groups --bootstrap-server localhost:9092 --list

List kafka topics:

kafka-topics --zookeeper localhost:2181 --list

Add partitions to topics:

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic my_topic_name --partitions 40 

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groupscommand.

  1. List the topics to which the group is subscribed
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under “CURRENT-OFFSET” and “LOG-END-OFFSET”. “CURRENT-OFFSET” is the offset where this consumer group is currently at in each of the partitions.

  1. Reset the consumer offset for a topic (preview)
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --topic <topic_name> --reset-offsets --to-earliest

This will print the expected result of the reset, but not actually run it.

  1. Reset the consumer offset for a topic (execute)
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --topic <topic_name> --reset-offsets --to-earliest --execute

This will execute the reset and reset the consumer group offset for the specified topic back to 0.

  1. Repeat 1 to check if the reset is successful
 

Here is the command to increase the partitions count from 2 to 3 for topic ‘my-topic‘ –

./bin/kafka-topics.sh --alter --zookeeper localhost:2181 --topic my-topic --partitions 3

You can verify whether partitions have been increased by using describe command as follows – 

./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-topic

# Command output
Topic:my-topic	PartitionCount:3		ReplicationFactor:1	Configs:
	Topic: my-topic	Partition: 0		Leader: 0	Replicas: 0	Isr: 0
	Topic: my-topic	Partition: 1		Leader: 0	Replicas: 0	Isr: 0
	Topic: my-topic	Partition: 2		Leader: 0	Replicas: 0	Isr: 0

LEAVE A COMMENT