Feature Store

Provides low-latency retrieval of pre-computed genomic features for real-time inference. The serving layer must deliver features like k-mer embeddings, conservation scores, or chromatin accessibility windows in under 10 milliseconds to support clinical decision support tools.

• Uses key-value stores (e.g., Redis, DynamoDB) for point lookups by variant ID or genomic coordinate
• Maintains feature freshness with streaming updates from batch computation pipelines
• Supports feature vector assembly by joining multiple feature groups on-the-fly
• Example: A variant pathogenicity classifier retrieves 512-dimensional embeddings, population allele frequencies, and evolutionary constraint scores in a single call

Enables point-in-time correct feature generation for training genomic foundation models. The store ensures that features used during training exactly match those served during inference, preventing the subtle bugs caused by training-serving skew.

• Integrates with distributed processing frameworks (Spark, Ray) for terabyte-scale genomic data
• Provides time-travel queries to reconstruct feature values as they existed at any historical point
• Supports backfilling features for retrospective cohort studies
• Example: When training a splice-site predictor, the store guarantees that the same sequence context window and conservation scoring algorithm is used in both training jobs and production inference

Prevents redundant computation by allowing teams to define reusable transformation logic that converts raw genomic data into model-ready features. Transformations are versioned and executed consistently in both offline and online environments.

• Supports domain-specific transformations: one-hot encoding of nucleotides, reverse-complement augmentation, GC-content normalization
• Enables feature reuse across models—a DNA language model embedding computed once can serve variant effect prediction, promoter classification, and enhancer detection
• Uses declarative transformation definitions (Python UDFs or SQL) stored alongside feature metadata
• Example: A genomic_sequence_embedding transformation applies a pre-trained DNABERT model to 1,000-bp context windows, producing a 768-dimensional vector consumed by five downstream models

The feature store's primary architectural purpose is guaranteeing that the exact same feature computation code path executes during both model training and production inference. This eliminates a pernicious class of bugs where models perform well in development but degrade silently in production.

• Shared transformation logic compiled to both batch (Spark) and real-time (Python microservice) execution engines
• Feature validation compares statistical distributions between training and serving data
• Immutable feature versions prevent changes to production features without explicit model retraining
• Example: If a variant annotation pipeline upgrades from dbSNP v150 to v155, the feature store creates a new feature version rather than overwriting, ensuring deployed models continue using the features they were trained on