Leave a Comment
List kafka consumer groups:
kafka-consumer-groups --bootstrap-server localhost:9092 --list
List kafka topics:
kafka-topics --zookeeper localhost:2181 --list
Add partitions to topics:
bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic my_topic_name --partitions 40
Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups
command.
- List the topics to which the group is subscribed
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe
Note the values under “CURRENT-OFFSET” and “LOG-END-OFFSET”. “CURRENT-OFFSET” is the offset where this consumer group is currently at in each of the partitions.
- Reset the consumer offset for a topic (preview)
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --topic <topic_name> --reset-offsets --to-earliest
This will print the expected result of the reset, but not actually run it.
- Reset the consumer offset for a topic (execute)
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --topic <topic_name> --reset-offsets --to-earliest --execute
This will execute the reset and reset the consumer group offset for the specified topic back to 0.
- Repeat 1 to check if the reset is successful
Here is the command to increase the partitions count from 2 to 3 for topic ‘my-topic‘ –
./bin/kafka-topics.sh --alter --zookeeper localhost:2181 --topic my-topic --partitions 3
You can verify whether partitions have been increased by using describe command as follows –
./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-topic
# Command output
Topic:my-topic PartitionCount:3 ReplicationFactor:1 Configs:
Topic: my-topic Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: my-topic Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: my-topic Partition: 2 Leader: 0 Replicas: 0 Isr: 0