Event Bus

Producers fire events and immediately continue processing without waiting for a response from consumers. This eliminates temporal coupling and prevents cascading failures.

• Fire-and-Forget: The producer's lifecycle is not tied to the consumer's availability or processing speed.
• Resilience: If a downstream service fails, the producer remains operational. Events are buffered in a durable log for later consumption.
• Latency Decoupling: Slow consumers do not throttle fast producers, a critical property for high-throughput systems like real-time user behavior tracking.

Unlike transient message queues, an event bus persists events to disk as an immutable, append-only log. This provides a durable source of truth that can be re-read.

• Event Sourcing: The log becomes the system's primary record, allowing state to be rebuilt from scratch by replaying events.
• Backfilling: New services can process the entire history of events to hydrate their state.
• Audit Trail: The immutable log provides a complete, tamper-proof record of everything that happened in the system.

Topics are divided into partitions, which are the unit of parallelism. Partitions are distributed across multiple brokers in a cluster, allowing the bus to scale linearly.

• Ordering Guarantee: Within a single partition, events are strictly ordered by their offset. Ordering across partitions is not guaranteed.
• Parallel Consumption: Multiple instances within a consumer group can read from different partitions in parallel, dramatically increasing throughput.
• Key-Based Routing: Producers can specify a key to ensure all related events (e.g., for a specific user) are routed to the same partition, preserving order.

An event bus provides configurable guarantees for message delivery, balancing throughput with reliability. The strongest guarantee is exactly-once semantics.

• At-Least-Once: The default, where events are never lost but may be redelivered. Consumers must be idempotent.
• Exactly-Once: A transactional guarantee ensuring an event is processed only once, end-to-end, even across producer retries and consumer failures.
• Mechanism: Achieved through idempotent producers, transactional writes, and consumer offset management within the same transaction.

To prevent data quality chaos, an event bus is typically paired with a Schema Registry. This central service enforces a contract between producers and consumers.

• Schema Evolution: Supports forward, backward, and full compatibility modes, allowing services to evolve independently without breaking downstream consumers.
• Formats: Uses compact binary formats like Apache Avro or Protobuf for efficient serialization and deserialization.
• Governance: The registry acts as a single source of truth for all data contracts, enabling automated validation and documentation.

Command Query Responsibility Segregation separates read and write operations into distinct models. Commands mutate state and emit events onto the bus, while queries consume those events to build optimized materialized views.

• Write side uses normalized, domain-driven models
• Read side uses denormalized views tailored to specific UIs
• Eliminates complex join queries on the write database

A specialized queue for messages that the event bus cannot deliver or consumers cannot process successfully after exhausting retry attempts. DLQs enable asynchronous failure analysis without blocking the main pipeline.

• Prevents poison-pill messages from stalling consumers
• Allows operators to inspect, fix, and replay failed events
• Essential for maintaining exactly-once processing guarantees

A delivery guarantee ensuring that a message is processed only once, even in the face of producer retries or consumer failures. Achieved through a combination of idempotent producers, transactional messaging, and checkpointing.

• Prevents duplicate charges or double-counting in financial systems
• Requires coordination between the event bus and consumer state stores
• Kafka achieves this via transactional APIs and idempotent producers