Backpressure Handling

The foundational standard for asynchronous stream processing with non-blocking backpressure. It defines a protocol where a Subscriber requests a specific number of items from a Publisher via request(n), preventing the producer from overwhelming the consumer's buffer. This pull-based model is the core of modern reactive libraries like Project Reactor and Akka Streams, ensuring that data flow is governed by the slowest component in the chain.

When backpressure signals are ignored or delayed, intermediate buffers absorb transient load spikes. Key strategies include:

• Fixed-Size Buffering: Queues data up to a limit, then applies a policy.
• Drop Oldest: Discards the head of the queue to make room for new data, useful for real-time telemetry where freshness matters more than completeness.
• Drop Newest: Ignores incoming data when the buffer is full, preserving the oldest unprocessed items.
• Unbounded Buffering: Dangerous in production; risks OutOfMemoryError under sustained overload.

A resilience pattern that acts as a proxy to a downstream service, monitoring failure rates. When failures exceed a threshold, the circuit trips to an OPEN state, immediately failing requests without calling the unhealthy service. After a cooldown period, it transitions to a HALF-OPEN state, allowing a limited number of test requests. If they succeed, the circuit CLOSES; otherwise, it re-OPENS. This prevents cascading failures and gives downstream systems time to recover from overload.

A deliberate strategy to reject excess requests at the system boundary before they consume internal resources. An API gateway or reverse proxy monitors server health and queue depths, returning HTTP 503 Service Unavailable for requests that would exceed capacity. This is preferable to accepting all requests and causing tail latency amplification across all in-flight operations. Techniques include adaptive LIFO queues and prioritized admission control based on request criticality.

Proactive flow control that caps the rate of requests a producer can send. Common algorithms:

• Token Bucket: Allows bursts up to a bucket size, refilling at a steady rate.
• Leaky Bucket: Smooths bursty traffic into a constant, fixed-rate output stream.
• Fixed Window Counter: Tracks request counts in discrete time windows; simple but prone to boundary bursts.
• Sliding Window Log: More accurate, tracks timestamps of individual requests to calculate a precise rate over a rolling window.

gRPC bidirectional streaming natively supports flow control via HTTP/2 frame-level backpressure. The underlying transport manages window sizes for each stream, preventing a fast sender from flooding a slow receiver's TCP buffer. Combined with application-level FlowControl messages in protocols like RSocket, this provides end-to-end backpressure from the network layer up to the business logic, making it ideal for microservices communication under variable load.

A resilience pattern that partitions system resources into isolated pools, ensuring a failure in one component cannot consume all available threads or connections.

• Thread Pool Bulkheads: Dedicated thread pools per downstream dependency
• Connection Pool Bulkheads: Separate connection pools for each external service
• Semaphore Bulkheads: Limit concurrent calls to a constrained resource

Complements backpressure by preventing a slow consumer in one partition from starving fast consumers in another, containing the blast radius of overload.

A form of input/output processing that allows a single thread to manage multiple concurrent connections without stalling. Instead of blocking on read/write operations, the thread registers interest in events and is notified when data is available.

• Selectors and Event Loops: Core primitives in libraries like Netty and Node.js
• Zero-Copy Transfers: Moving data directly between buffers without CPU intervention
• Backpressure Integration: Non-blocking transports naturally support reactive pull signals at the network level

Fundamental to frameworks like Project Reactor and Akka Streams that implement backpressure end-to-end.

A queue that stores messages a messaging system cannot deliver or process successfully. When backpressure mechanisms reject messages due to persistent overload, the DLQ provides a safety net rather than silently dropping data.

• Poison Messages: Malformed payloads that repeatedly fail processing
• Expired Messages: Events that exceeded their time-to-live while waiting in queue
• Replay Capability: Operators can inspect, fix, and resubmit dead-lettered messages

Essential for at-least-once delivery guarantees in systems where backpressure-induced rejection must not cause permanent data loss.

A deliberate strategy of dropping excess requests when a system approaches saturation, preserving the quality of service for the requests it can handle rather than degrading all requests equally.

• Tail Drop: Reject the newest requests when queues are full
• Head Drop: Drop the oldest queued requests to serve fresh ones faster
• Priority-Based Shedding: Preserve high-value requests while shedding low-priority traffic

Differs from backpressure in that it's a unilateral decision by the server, whereas backpressure is a cooperative signal between producer and consumer to slow the flow.