Micro-Batching

Micro-batching deliberately introduces a small, fixed delay—typically 100ms to 2 seconds—to accumulate records before processing. This trade-off increases end-to-end latency compared to true per-event stream processing but dramatically improves system throughput by amortizing operational overhead (network round-trips, disk seeks, serialization) across multiple records. The batch interval is a tunable knob: shorter intervals approach stream-like latency, while longer intervals maximize resource efficiency.

Micro-batch architectures achieve exactly-once processing semantics by treating each batch as an atomic unit of work. The system checkpoints the offset or state after successfully processing a batch. If a node fails, the system simply re-processes the entire failed batch from the last checkpoint. This is significantly simpler to implement than the distributed snapshot mechanisms required for true per-event stream processing, making it a pragmatic choice for systems where occasional sub-second latency is acceptable.

Processing records individually incurs per-event costs:

• Serialization/deserialization overhead
• Network I/O for each remote call
• Disk seeks for state persistence
• Thread context switching in the processor

Micro-batching amortizes these fixed costs across N records, dramatically increasing records-per-second throughput. For high-volume pipelines processing millions of events per second, this amortization is often the difference between a viable and non-viable architecture.

In micro-batching systems, backpressure is handled implicitly by the batch scheduler. If downstream systems slow down, the processing of the next batch is simply delayed until resources are available. This contrasts with push-based stream processors that require explicit backpressure protocols (like Reactive Streams) to signal consumers. The batch boundary acts as a natural flow-control point, preventing unbounded queue growth and out-of-memory failures during load spikes.

Choose micro-batching when throughput and simplicity outweigh the need for sub-second latency.

A method of tracking and analyzing streams of data about events to derive conclusions and trigger immediate automated responses. CEP engines detect patterns across multiple event streams using rules or state machines.

• Use case: Detecting fraud by correlating login location, transaction amount, and device fingerprint within a 30-second window
• Relationship to micro-batching: CEP often runs atop micro-batched or streaming infrastructure
• Key concept: Temporal pattern matching with sliding windows

A stream processing technique that divides an unbounded data stream into finite, manageable chunks based on time or count. Windowing is the conceptual foundation that makes micro-batching coherent.

• Tumbling windows: Fixed-size, non-overlapping intervals (e.g., every 5 seconds)
• Sliding windows: Overlapping intervals defined by size and slide period
• Session windows: Dynamic windows based on periods of activity followed by gaps of inactivity
• Micro-batching relationship: Each micro-batch is essentially a tumbling window

A mechanism in reactive systems that allows a consumer to signal to a producer that it is overwhelmed, controlling data flow to prevent system failure under load. Critical for micro-batching pipelines when downstream processing cannot keep pace with ingestion.

• Strategies: Buffering, dropping messages, dynamic batch sizing
• Protocols: Reactive Streams, TCP flow control, RSocket
• Failure mode without backpressure: Out-of-memory errors and cascading crashes

The process of providing pre-computed and real-time features to a machine learning model at prediction time with extremely low latency. Micro-batching is often used to aggregate streaming events into feature vectors before serving.

• Typical latency target: < 10ms for real-time personalization
• Feature store role: Central repository that combines batch historical features with streaming real-time features
• Micro-batching use case: Computing a user's rolling 5-minute click count as a feature for a recommendation model

The phenomenon where a small percentage of slow requests in a distributed system is magnified by parallel fan-out operations, significantly degrading overall user experience. Micro-batching can mitigate this by consolidating I/O operations.

• Example: A service calling 100 downstream services; if each has a 1% chance of p99 latency, the composite p99 becomes ~63%
• Mitigation strategies: Hedged requests, micro-batching to reduce fan-out, speculative execution
• Relevance: Micro-batching trades a small latency floor increase for dramatically improved tail latency predictability