10. System design building blocksKafka vs queues
Kafka vs queues
Kafka is a distributed log: producers append messages to the end of a topic, and those messages stay there for a set retention period whether or not anyone has read them. Each consumer group just remembers its position, called an offset, so different services can read the same data at their own pace, and you can rewind to reprocess history. A traditional message queue like RabbitMQ or SQS works more like a to-do list: each message goes to one worker and is removed once that worker acknowledges it. Kafka splits each topic into partitions spread across machines for throughput, and keeps order only within a partition, so you give related messages, say all events for one user, the same key to keep them together and in sequence.
In this lesson: A 3D log with partitions, offsets and consumer-group rebalancing, and a replay slider.