Kafka is a distributed streaming platform. The de-facto standard for event-driven architectures.

Core Concepts¶

• Topic — category of messages
• Partition — horizontal division of a topic
• Producer — writes messages
• Consumer — reads messages
• Consumer Group — group of consumers, each reads from different partitions
• Offset — position in a partition
• Broker — Kafka server

Commands¶

Create topic¶

kafka-topics –create –topic orders –partitions 6 –replication-factor 3

List¶

kafka-topics –list

Produce¶

kafka-console-producer –topic orders –bootstrap-server localhost:9092

Consume¶

kafka-console-consumer –topic orders –from-beginning –group my-group

Use Cases¶

• Event streaming (user events, logs)
• Microservices communication
• Change Data Capture (CDC)
• Real-time analytics pipeline
• Log aggregation

Guarantees¶

• At-most-once — may lose messages
• At-least-once — may duplicate (default)
• Exactly-once — idempotent producer + transactional consume

Alternatives¶

• RabbitMQ — simpler, message queue (not streaming)
• NATS — lightweight, cloud-native
• Redpanda — Kafka-compatible, no JVM

When to Use Kafka¶

Kafka for high-throughput event streaming. RabbitMQ for task queues. NATS for lightweight messaging.