Feature Serving

Production serving must retrieve feature vectors in single-digit milliseconds to avoid degrading the user experience. This is achieved through:

• In-memory caches like Redis for hot feature storage
• gRPC protocols for efficient binary serialization
• Co-located infrastructure where the serving layer sits physically close to the model endpoint A typical target is p99 latency under 10ms for a full feature vector lookup.

To prevent data leakage during training, the serving layer must be able to reconstruct feature values exactly as they existed at a historical timestamp. This is not just a training concern; the online store must be synchronized with the offline store to ensure the model receives features in the same format and logic it was trained on. Time travel capabilities in the feature store enforce this consistency.

If the feature serving layer goes down, the model cannot make predictions. Production architectures require:

• Multi-AZ replication of the online store
• Circuit breakers that fall back to default feature values if a specific feature group is unavailable
• Stateless serving nodes that can be horizontally scaled behind a load balancer This ensures 99.99% uptime for mission-critical personalization pipelines.

A feature is only as good as its staleness. Feature freshness defines the maximum acceptable age of a value in the online store. Serving infrastructure must track the lag between source data updates and materialization. For example, a streaming feature capturing 'items added to cart in the last 5 minutes' must be refreshed within seconds, while a batch feature like 'lifetime purchase history' can tolerate a daily update cadence.

The serving API must accept an entity key (e.g., user_id) and return a fully assembled feature vector as a numerical array. This involves:

• Joining features from multiple feature groups by the entity key
• Applying final on-demand transformations using request-level context (e.g., distance_to_store calculated from user GPS)
• Encoding the vector into the exact format expected by the model's input tensor

The interface between the model server and the feature store is typically a high-performance gRPC or REST endpoint. A common pattern is the FeatureService abstraction, which defines a logical grouping of features needed for a specific model. The model sends a single request with an entity ID, and the serving layer handles the fan-out to retrieve all required features from the online store, abstracting away the physical storage complexity.

A one-dimensional array of numerical values representing the aggregated features for a specific entity at a specific point in time. This is the direct input payload delivered to a model endpoint.

• Composition: Joins features from multiple Feature Groups using an Entity key
• Format: Dense numerical arrays or tensors
• Critical Requirement: Must maintain Point-in-Time Correctness to prevent data leakage

The process of pre-computing feature values from source data and persisting them into an online or offline store. This decouples heavy transformation logic from the critical path of a prediction request.

• Streaming Materialization: Uses Change Data Capture for real-time updates
• Batch Materialization: Runs on schedules to compute Batch Features
• Backfilling: Retroactively materializes features for historical time ranges

A metric defining the maximum acceptable age of a feature value in the online store. Stale data directly degrades model accuracy in dynamic environments like retail hyper-personalization.

• Real-time features: Freshness measured in seconds (e.g., clicks in current session)
• Near-real-time: Freshness measured in minutes (e.g., inventory levels)
• Slow-moving features: Freshness measured in hours or days (e.g., user lifetime value)

The programmatic interface used by model serving infrastructure to request feature vectors from the online store with ultra-low latency. It abstracts the complexity of physical storage from the inference pipeline.

• Protocols: gRPC for high-throughput, REST for ease of debugging
• Pattern: getOnlineFeatures(entity_id, feature_names)
• Optimization: Often paired with a Feature Cache to reduce database load

Feature values computed at request time using raw data passed directly to the feature store, rather than being pre-materialized. Essential for context that cannot be pre-computed.

• Use Case: Calculating distance from a user's live GPS coordinates to a store
• Trade-off: Higher serving latency vs. infinite freshness
• Integration: Combines request payload with pre-materialized Feature Vectors