Inferensys

Glossary

Feature Vector

A one-dimensional array of numerical values representing the aggregated features for a specific entity at a specific point in time, used as input to a machine learning model.
Data engineer managing feature store on laptop, feature definitions visible, casual data engineering session.
MACHINE LEARNING FUNDAMENTALS

What is a Feature Vector?

A feature vector is the numerical representation of an entity's attributes at a specific moment, serving as the direct input for machine learning model inference and training.

A feature vector is a one-dimensional, ordered array of numerical values that collectively represents the measurable properties of a single entity—such as a user, product, or transaction—at a specific point in time. Each position in the array corresponds to a distinct feature, like a user's age, average purchase value, or session click count. This structure translates raw, often unstructured behavioral data into the standardized mathematical format that machine learning algorithms require for computation.

During real-time inference, a serving infrastructure queries a feature store to assemble a feature vector with ultra-low latency by joining pre-computed batch features and freshly calculated streaming features. The vector's integrity depends on point-in-time correctness to prevent data leakage and feature freshness to avoid acting on stale information. This aggregated numerical snapshot is then passed to a model endpoint, enabling immediate, context-aware predictions for applications like dynamic pricing or next-best-action recommendations.

FEATURE VECTOR ESSENTIALS

Frequently Asked Questions

Clear, technical answers to the most common questions about the structure, creation, and use of feature vectors in machine learning systems.

A feature vector is a one-dimensional, ordered array of numerical values that represents the measurable characteristics of an entity at a specific point in time, serving as the input to a machine learning model. Each element in the vector corresponds to a specific feature—a predictive attribute such as a user's age, a product's price, or an aggregated count of recent clicks. During online inference, a serving infrastructure queries a feature store to assemble these values from disparate sources, combining batch features like historical purchase totals with streaming features like session click rate into a single, dense array. The model then performs a mathematical operation, typically a dot product or matrix multiplication, on this vector to produce a prediction. The strict ordering of the vector is critical; the model's first weight coefficient always corresponds to the first feature, making schema consistency between training and serving a non-negotiable requirement.

ANATOMY OF A PREDICTION INPUT

Key Characteristics of a Feature Vector

A feature vector is the fundamental unit of input for a machine learning model, representing a specific entity at a specific moment in time. Its structure, quality, and consistency directly dictate model performance in production.

01

Dimensionality

The number of elements in the vector, representing the total count of distinct features. Dimensionality is a critical architectural constraint.

  • High-dimensional vectors (hundreds to thousands of features) capture complex patterns but increase computational cost and the risk of overfitting.
  • Low-dimensional vectors are computationally efficient but may lack the predictive signal needed for nuanced decisions.
  • Dimensionality reduction techniques like Principal Component Analysis (PCA) are often applied to project high-dimensional data into a lower-dimensional space while preserving variance.
02

Numerical Representation

All values within a feature vector must be numerical for mathematical operations. Raw data must be transformed through feature engineering.

  • Continuous features: Real-valued numbers like price = 49.99 or temperature = 72.5.
  • Categorical features: Converted via encoding schemes. One-hot encoding creates binary columns for each category, while label encoding assigns an integer.
  • Text and image data: Transformed into dense numerical embeddings using models like BERT or ResNet, which capture semantic meaning in a vector space.
03

Entity Association

A feature vector is always keyed to a specific entity, such as a user ID, product SKU, or device ID. This association is what links a prediction to a real-world subject.

  • The entity serves as the primary join key when assembling features from multiple feature groups in a feature store.
  • A complete prediction input often combines features from multiple entities, such as a user vector and an item vector, concatenated for a recommendation model.
  • Without a strict entity association, a vector is just a meaningless array of numbers.
04

Temporal Point-in-Time Correctness

A feature vector is a snapshot of an entity's state at a precise moment. Using future data to construct a historical vector causes data leakage and invalidates the model.

  • Point-in-time joins in a feature store ensure that when training on a historical event, only feature values that existed before that event are used.
  • For example, a vector predicting a purchase on Monday must not include a user_total_spend feature that was updated on Tuesday.
  • This temporal discipline is the single most critical factor in building models that perform consistently when deployed.
05

Sparsity

A vector is sparse when the vast majority of its elements are zero. This is common after one-hot encoding high-cardinality categorical features.

  • A vector with 100,000 dimensions might have only 50 non-zero values, representing a user's active product categories.
  • Sparse vectors are stored and computed using specialized formats like CSR (Compressed Sparse Row) to avoid wasting memory on zeros.
  • Many algorithms, such as linear models and factorization machines, are optimized to handle sparse input efficiently.
06

Serving Consistency

The feature vector used for online inference must be an exact logical match to the vector used during offline training. Any discrepancy is a training-serving skew.

  • A feature store's feature view defines the transformation logic once and applies it identically in both environments.
  • If a training pipeline applies a z-score normalization, the online serving pipeline must use the identical mean and standard deviation calculated from the training set.
  • Consistency extends to data types, encoding schemes, and handling of missing values, ensuring the model receives data in the format it learned from.
ONLINE SERVING ARCHITECTURE

How Feature Vectors Are Assembled in a Feature Store

The feature vector is the final, model-consumable input assembled at inference time by joining features from multiple sources using a shared entity key.

A feature vector is a one-dimensional array of numerical values representing the aggregated features for a specific entity at a specific point in time. During online inference, the feature store assembles this vector by receiving a request containing an entity ID and querying its online store to retrieve pre-materialized values from disparate feature views. The serving layer performs a point-in-time join, concatenating batch features like user purchase history with streaming features like real-time session clicks into a single, ordered array that matches the exact input schema the model expects.

This assembly process is governed by the feature view definition, which specifies the transformation logic and the precise ordering of features. To meet strict latency budgets, the serving API often retrieves these values from an in-memory feature cache rather than the primary database. The resulting vector is then passed directly to the model endpoint, ensuring that the prediction is based on a consistent, fresh, and complete snapshot of the entity's current state without requiring the model to access any external data sources.

DECODING THE INPUT

Feature Vector Examples in Retail Personalization

A feature vector is a one-dimensional array of numerical values representing the aggregated features for a specific entity at a specific point in time. In retail, these vectors are the mathematical fuel powering real-time recommendations, search ranking, and dynamic pricing.

01

Real-Time User Session Vector

This vector captures a shopper's immediate intent during a live session. It is served from the Online Store with sub-millisecond latency.

  • Inputs: Last 5 SKUs viewed, current cart value, session duration, device type.
  • Encoding: Categorical actions are one-hot encoded; numerical values are normalized.
  • Use Case: A Next-Best-Action Model uses this to decide whether to show a discount or a product recommendation.
02

Long-Term Customer Profile Vector

A pre-computed Batch Feature vector summarizing historical loyalty and preferences, materialized daily in the Offline Store.

  • Inputs: 90-day purchase frequency, lifetime value tier, average order value, preferred category affinity scores.
  • Mechanism: Generated via Point-in-Time Correctness joins to avoid data leakage from future returns.
  • Use Case: Injected into a Deep Learning Recommender System to seed the candidate generation layer.
03

Product Catalog Embedding

A dense vector representation of a product, often generated by a neural network, stored in a Vector Database for similarity search.

  • Inputs: Product title tokens, description semantics, brand, price bucket, image pixels.
  • Technique: A multimodal model processes text and images to output a fixed-length embedding.
  • Use Case: Finding visually and contextually similar items for a 'Complete the Look' carousel.
04

Contextual Multi-Armed Bandit Arm

A dynamic feature vector representing the decision context for a reinforcement learning agent balancing exploration and exploitation.

  • Inputs: Time of day, traffic source (email vs. social), geo-location, inventory level.
  • Mechanism: Contextual features are concatenated with user and item vectors to estimate the reward probability.
  • Use Case: A Dynamic Pricing Algorithm adjusts a discount multiplier based on this context to maximize margin.
05

Cross-Device Identity Stitching Vector

A unified vector that resolves anonymous and logged-in sessions across devices into a single profile using Feature Engineering logic.

  • Inputs: Probabilistic matches on IP prefix, deterministic matches on login ID, device graph cluster ID.
  • Challenge: Mitigates the Cold Start Problem by merging mobile browsing features with desktop purchase history.
  • Use Case: Ensures a user who browsed on a phone sees consistent Dynamic Assortment Optimization on their laptop.
06

Streaming Inventory Signal Vector

A real-time feature vector computed from Change Data Capture feeds, reflecting current stock levels and velocity.

  • Inputs: Current stock count, sell-through rate (last hour), trending velocity score, warehouse proximity.
  • Freshness: Requires Feature Freshness of under 1 second to prevent overselling.
  • Use Case: A Demand Forecasting Model consumes this to trigger a 'Low Stock' urgency badge on the product display page.
DATA STRUCTURE COMPARISON

Feature Vector vs. Related Concepts

Distinguishing a feature vector from adjacent data structures commonly used in machine learning pipelines.

PropertyFeature VectorEmbeddingTraining Example

Definition

1D array of aggregated features for an entity at a point in time

Dense, low-dimensional learned representation of an entity

Complete input-output pair used to train a supervised model

Dimensionality

Explicit, human-engineered dimensions

Latent, model-learned dimensions

Combines feature vector with a target label

Interpretability

High; each element maps to a known feature

Low; individual dimensions lack intrinsic meaning

High for features; target label is explicit

Primary Use

Input to any ML model for inference or training

Input to neural networks; similarity search

Model fitting and weight optimization

Storage Location

Online/offline feature store

Vector database

Data lake or offline feature store

Contains Label

Generation Method

Feature engineering and materialization

Forward pass through a trained neural network

Joining feature vectors with ground truth labels

Temporal Aspect

Point-in-time snapshot

Static representation of an entity

Historical event with a known outcome

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.