Windowing

Fixed-size, non-overlapping, contiguous time intervals. Each event belongs to exactly one window.

• Mechanism: A window of size T is defined. When the event time crosses the boundary, the window closes and a new one opens immediately.
• Use Case: Hourly sales aggregates, per-minute error rate calculations.
• Key Trait: No overlap, no shared data between windows. Simplest to reason about.

Fixed-size windows that advance by a period smaller than the window size, creating overlapping intervals. Also called sliding windows in some frameworks.

• Mechanism: Defined by window size and hop size. If size=10min and hop=5min, a new window starts every 5 minutes and covers 10 minutes of data.
• Use Case: Moving averages, rolling counts for trend detection.
• Key Trait: Events belong to multiple windows. Provides smoother aggregates at the cost of increased state.

A window that continuously evaluates a fixed range anchored to the current event time. Unlike hopping windows, it is not aligned to a fixed grid.

• Mechanism: For every new event, the window looks back a fixed duration (e.g., last 30 seconds).
• Use Case: Real-time dashboards showing 'last 5 minutes of activity', intrusion detection.
• Key Trait: Continuous evaluation. High computational cost for frequent updates.

Dynamic windows that group events separated by periods of inactivity (gaps). Window boundaries are defined by data, not fixed time intervals.

• Mechanism: A session gap timeout is defined. If no events arrive for the gap duration, the session closes. A new event starts a new session.
• Use Case: User clickstream analysis, web sessionization, user journey mapping.
• Key Trait: Variable length. Captures natural bursts of activity. Requires watermarking to handle out-of-order events.

A mechanism to track event time progress and handle out-of-order data. Watermarks declare that no events with a timestamp older than T are expected.

• Mechanism: A watermark is a threshold that advances as events arrive. When the watermark passes the end of a window, the window is finalized.
• Allowed Lateness: Defines how long to keep window state open after the watermark to accept late-arriving data.
• Key Trait: Balances result completeness against output latency. Critical for exactly-once semantics.

Fixed-size, non-overlapping time intervals that segment a stream into discrete buckets. Each event belongs to exactly one window.

• Mechanism: A 5-minute tumbling window creates buckets at 12:00–12:05, 12:05–12:10, etc.
• Retail Use Case: Compute category browse counts every 60 seconds to update trending product badges
• Key Trait: No shared events between windows; ideal for periodic metric reporting
• Implementation: window(TumblingProcessingTimeWindows.of(Time.seconds(30))) in Apache Flink

Overlapping windows defined by a window size and a slide interval. Events can belong to multiple windows simultaneously.

• Mechanism: A 10-minute window sliding every 2 minutes creates windows at [0-10], [2-12], [4-14]
• Retail Use Case: Calculate rolling cart abandonment rate over the last 30 minutes, updated every 60 seconds
• Key Trait: Smooths metric transitions; higher computational cost than tumbling windows
• Optimization: Use incremental aggregation to avoid recomputing overlapping data

Dynamic windows that group events separated by a defined inactivity gap. Window boundaries are determined by the data itself, not fixed time intervals.

• Mechanism: A 15-minute gap threshold merges events into a session until no new events arrive for 15 minutes
• Retail Use Case: Sessionization of user browsing to compute session-level dwell time, scroll depth, and intent signals
• Key Trait: Window size adapts to user behavior; essential for visit-level feature engineering
• Watermark Dependency: Requires watermarks to handle out-of-order events and late arrivals

The temporal reference point for window assignment critically impacts accuracy in distributed systems.

• Event Time: Timestamp embedded in the event record when the user action occurred. Essential for correct sessionization despite network delays
• Processing Time: System clock time when the event is ingested. Simpler but produces inaccurate windows during backpressure or lag
• Retail Impact: Using processing time for a flash sale clickstream can misattribute late-arriving clicks to the wrong window, skewing demand signals
• Watermarks: Track event-time progress to trigger window evaluations when all data up to a threshold is received

Strategies for managing events that arrive after a window's watermark has passed, common in mobile retail apps with intermittent connectivity.

• Allowed Lateness: Configure a grace period (e.g., 30 seconds) where late events still update window results before final emission
• Side Outputs: Route excessively late events to a separate stream for offline reconciliation or model retraining
• Retail Use Case: A user adds an item to cart while in a subway tunnel; the event arrives 2 minutes late. Allowed lateness ensures it's counted in the correct session window
• Trade-off: Longer lateness windows increase state size and memory consumption

Stateful stream processors maintain intermediate aggregates per window, requiring careful state store configuration for production personalization workloads.

• RocksDB Backend: Persists window state to disk for fault tolerance; critical for large session windows spanning hours
• Checkpointing: Periodic snapshots of window state to durable storage (e.g., S3) enable exactly-once recovery from failures
• State TTL: Configure time-to-live to automatically clean up expired window state and prevent unbounded memory growth
• Retail Scale: A global e-commerce site with 10M daily active users requires careful state partitioning to avoid hot keys