Windowed Aggregation

Fixed-size, non-overlapping, contiguous time intervals. Each event belongs to exactly one window. When the window ends, the aggregate is emitted and the state is purged.

• Example: Compute total sales every 5 minutes (12:00-12:05, 12:05-12:10).
• Key Trait: No duplication of events across windows.
• Use Case: Regulatory reporting requiring precise, non-overlapping time buckets.

Fixed-size windows that advance forward by a hop size smaller than the window duration, causing overlapping intervals. An event can belong to multiple windows simultaneously.

• Example: A 10-minute window advancing every 1 minute (12:00-12:10, 12:01-12:11).
• Key Trait: Smoothes aggregates by providing overlapping, frequent updates.
• Use Case: Moving average of click-through rate for a trending product.

Defined by a duration and a slide interval. Unlike hopping windows, sliding windows are typically evaluated continuously, emitting a new result every time the slide interval elapses or on every event arrival.

• Example: Count the number of page views in the last 60 seconds, updated every 5 seconds.
• Key Trait: Provides the most granular, real-time view of a rolling period.
• Use Case: Real-time anomaly detection on a stream of sensor telemetry.

Dynamic windows that group events separated by a gap of inactivity. Window boundaries are defined by the data itself, not by fixed time intervals. A session ends when no events arrive for the specified gap duration.

• Example: Group all user clicks into a single session until 30 minutes of inactivity occurs.
• Key Trait: Captures user intent and journeys naturally.
• Use Case: Sessionizing clickstream data for propensity scoring.

A single, unbounded window that encompasses the entire stream from beginning to end. Aggregations are never finalized automatically; they must be triggered by a custom policy, such as a watermark or a count of elements.

• Example: Maintain a running total of all unique user IDs seen since the application started.
• Key Trait: State grows indefinitely unless explicitly purged.
• Use Case: Maintaining a global leaderboard or a cumulative lifetime value counter.

A window that starts at a fixed point in time and expands indefinitely, accumulating all events. Unlike a global window, it has a defined start time but no end time.

• Example: Aggregate all orders from the start of the fiscal quarter to the current moment.
• Key Trait: Provides a running total from a specific epoch.
• Use Case: Real-time dashboards showing month-to-date revenue against a target.

The two fundamental temporal domains in stream processing. Event time is when an event actually occurred at the source; processing time is when the stream processor observes it.

• Windowed aggregations on event time produce deterministic, reproducible results regardless of replay
• Processing time windows are simpler but non-deterministic
• The skew between these two times is called event-time skew and directly impacts watermark strategy

A delivery and processing guarantee ensuring that even during worker failures or network partitions, each record contributes to a windowed aggregation exactly once. This prevents double-counting or data loss in stateful operations.

• Achieved via distributed checkpointing and idempotent writes
• Critical for financial aggregations where correctness is non-negotiable
• Contrast with at-least-once (duplicates possible) and at-most-once (gaps possible)

A higher-order pattern-matching paradigm built atop windowed streams. While windowed aggregation computes summary statistics, CEP detects temporal sequences of events that match a defined pattern (e.g., 'item added to cart' followed by 'payment failure' within 5 minutes).

• Uses non-deterministic finite automata (NFA) for pattern matching
• Complements windowed aggregation for real-time segmentation triggers
• Common in fraud detection and user journey orchestration

The operational monitoring practice of detecting when the statistical distribution of a data stream changes over time, causing windowed aggregation outputs to shift in ways that may degrade downstream model performance.

• Sudden drift: Abrupt change (e.g., Black Friday traffic spike)
• Incremental drift: Gradual shift (e.g., seasonal preference changes)
• Windowed aggregations must be paired with drift monitors to trigger online model retraining when distributions diverge