Feature Freshness

Feature freshness ensures that the value served to a model reflects the entity's state within a defined time window. This prevents training-serving skew where a model trained on recent data encounters stale production values.

• Maximum Age: The wall-clock time since the feature value was last computed.
• Event-Time vs. Processing-Time: Freshness is measured from the event's occurrence, not when the pipeline processed it.
• Violation Impact: Stale features degrade click-through rate (CTR) predictions and fraud detection accuracy.

The required freshness dictates the data processing architecture. Streaming features are updated incrementally via platforms like Apache Kafka or Flink, achieving sub-second freshness. Batch features rely on scheduled ETL jobs, introducing inherent staleness.

• Real-Time: Essential for next-best-action models and dynamic pricing.
• Near-Real-Time: Suitable for session-based recommenders (1-5 minute freshness).
• Daily Batch: Acceptable for customer lifetime value (CLV) forecasts that change slowly.

Feature freshness is a key metric in data observability dashboards. A sudden increase in feature age often signals a pipeline stall or change data capture (CDC) failure.

• Freshness SLOs: Define alerts like 'p99 freshness > 5 seconds for 10 minutes'.
• Backpressure: High load on the online store can delay writes, increasing staleness.
• Null Value Injection: When a feature exceeds its maximum age, the serving layer should return a null value or a default to avoid silent model degradation.

The frequency at which a feature view is materialized directly determines its freshness. More frequent materialization reduces staleness but increases infrastructure cost.

• Trigger-Based: Materialization is triggered by a new event (e.g., user click).
• Tumbling Windows: Features are recomputed on a fixed schedule (e.g., every 60 seconds).
• Cost Trade-off: Sub-second materialization requires high-throughput online store writes and a robust feature cache.

Feature freshness in the online store must align with point-in-time correctness in the offline store. If training data is reconstructed with 1-hour freshness but online features are 1-second fresh, the model may overfit to granularity it won't see in production.

• Time Travel Queries: Validate that historical feature values match the staleness profile of the production pipeline.
• Data Contract: The freshness SLA is a core component of the agreement between data producers and model consumers.

On-demand features are computed at request time using raw context passed in the serving API call. Their freshness is effectively zero, as they represent the instantaneous state.

• Use Case: Real-time location, current cart contents, or session metadata.
• Latency Budget: On-demand computation must fit within the model's p99 latency budget.
• Hybrid Serving: Combine pre-computed batch features with on-demand streaming features to balance freshness and latency.