A compact Kafka reference for common operations and mental models.

Core terms

  • Topic: named stream
  • Partition: ordered shard
  • Consumer group: scalable readers sharing partitions
  • Offset: message position

Create/list topics

kafka-topics.sh --bootstrap-server localhost:9092 --list
kafka-topics.sh --bootstrap-server localhost:9092 --create --topic orders --partitions 3 --replication-factor 1

Produce/consume

kafka-console-producer.sh --bootstrap-server localhost:9092 --topic orders
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders --from-beginning

Consumer lag

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group

Tuning reminders

  • Keys preserve order per partition.
  • Idempotent producers reduce duplicates.
  • Set retention and compaction based on business needs.