Apache Pulsar

Pulsar natively supports a hierarchical multi-tenancy model with Properties, Namespaces, and Topics. This provides strong isolation and resource management for different teams or applications within a single cluster. Key features include:

• Authentication and Authorization: Pluggable security at the namespace level.
• Storage Quotas: Limit storage per namespace to prevent noisy neighbors.
• Resource Isolation: Allocate specific bookie groups to critical tenants.

Pulsar uniquely supports both queuing and streaming semantics on a single topic through its subscription model.

• Exclusive/Failover: Strict queuing, where only one consumer in a subscription receives messages.
• Shared: Competing consumers, like a traditional message queue.
• Key_Shared: Messages with the same key are delivered to the same consumer, ensuring ordering.
• Exclusive: Each consumer in the subscription reads all messages, enabling fan-out streaming.

Pulsar's foundational architectural principle that decouples message serving from message storage. Brokers handle serving and are stateless, while BookKeeper bookies provide durable, tiered storage. This enables independent scaling: add brokers for throughput, add bookies for capacity. Unlike Kafka's co-located model, this eliminates costly data rebalancing during cluster expansion and allows instant recovery from broker failure since no data partition needs to be copied.

A native, built-in mechanism for asynchronous cross-datacenter replication of topics. Pulsar replicates messages across clusters in different geographic regions, supporting active-active, active-standby, and full-mesh topologies. Unlike bolt-on replication tools, Pulsar's geo-replication is configured at the tenant or namespace level and handles conflict resolution automatically. This is critical for global e-commerce platforms requiring low-latency local reads while maintaining a unified, disaster-resilient event fabric.

A hierarchical resource model that isolates data, authentication, and quotas within a single Pulsar instance. The structure is:

• Tenant: Top-level administrative unit (e.g., a business unit)
• Namespace: A grouping of topics with shared policies (e.g., ecommerce/checkout)
• Topic: The actual message channel Each level enforces its own authentication, authorization, and storage quotas, making Pulsar ideal for large organizations where multiple teams share infrastructure without risk of cross-contamination.

Pulsar natively supports both queuing and streaming semantics within a single topic through its subscription model:

• Exclusive: Only one consumer receives messages (traditional queue)
• Shared: Multiple consumers compete for messages in a round-robin fashion (worker queue)
• Failover: One active consumer, with others on standby
• Key_Shared: Messages with the same key are delivered to the same consumer, preserving ordering while enabling parallelism This eliminates the need for separate RabbitMQ and Kafka deployments, unifying the messaging backbone.

The distributed write-ahead log storage system that underpins Pulsar's durability. BookKeeper organizes data into ledgers—append-only, immutable sequences of entries striped across multiple bookie nodes. Key properties include:

• I/O isolation: Write and read paths are separated
• Ensemble placement: Data is written to a configurable number of bookies (E), with a quorum (Q) required for acknowledgment
• Auto-recovery: Failed bookies are automatically detected and ledger data is re-replicated This architecture provides the foundation for Pulsar's tiered storage and infinite log retention.

A lightweight, serverless compute framework embedded directly within Pulsar for native stream processing. Functions consume messages from one or more input topics, apply user-defined logic (in Java, Python, or Go), and publish results to output topics. Unlike external stream processors like Flink, Pulsar Functions:

• Run co-located with brokers for minimal latency
• Are deployed and scaled via the Pulsar admin API
• Support stateful processing with a built-in state store
• Handle message routing, filtering, and enrichment without external infrastructure