User Embedding

User embeddings compress sparse behavioral data (millions of item IDs) into a compact, continuous vector, typically ranging from 64 to 1024 dimensions. This dense representation ensures that every dimension contributes meaningfully to encoding user intent, unlike sparse one-hot encodings. The embedding dimension is a critical hyperparameter: too small, and the model lacks capacity to capture nuanced preferences; too large, and it wastes memory and risks overfitting. These vectors are stored in an embedding table for efficient GPU-accelerated lookups during training and inference.

User embeddings are not manually engineered; they are learned directly from behavioral signals. Common training paradigms include:

• Next-Item Prediction: A self-supervised task where the model predicts the immediate next interaction given a sequence of past actions.
• Contrastive Learning: Pulls representations of similar user-item pairs together while pushing apart dissimilar pairs, often using the InfoNCE loss.
• Triplet Loss: Enforces relative similarity by ensuring an anchor user is closer to a positive item than a negative item by a defined margin. These objectives shape the latent space so that distance reflects behavioral affinity.

Once generated, the primary operation on user embeddings is similarity search. The standard metric is cosine similarity, which measures the angle between two vectors rather than their magnitude. By applying embedding normalization (L2 normalization), all vectors are projected onto a unit sphere, making cosine similarity equivalent to a simple dot product. This allows systems to efficiently find 'users like this one' or 'items this user would like' by retrieving the top-k nearest neighbors using Approximate Nearest Neighbor (ANN) algorithms such as Hierarchical Navigable Small World (HNSW) graphs.

User intent is not static. Sequence-aware embeddings capture this by processing chronologically ordered actions through recurrent neural networks or Behavioral Sequence Transformers, which use self-attention to model long-range dependencies. This allows the embedding to reflect short-term session context (e.g., shopping for a wedding) versus long-term stable preferences. Without continuous updates, representations suffer from embedding drift, where the latent space becomes stale as user behavior and item popularity distributions shift. Streaming embedding updates address this by incrementally modifying vectors in near real-time as new events arrive.

A single fixed vector often fails to capture the diversity of a user's interests (e.g., a user who enjoys both science fiction novels and gourmet cooking tools). Multi-interest extraction techniques decompose a user's embedding into multiple distinct prototype vectors, each representing a different latent interest. This enables diverse recommendation by matching different items to different facets of the user's profile, rather than averaging incompatible preferences into a single, muddy representation.

Pure collaborative filtering embeddings fail for new users with no interaction history. Cold-start embeddings and hybrid embeddings solve this by fusing behavioral signals with content-based features. A content-based tower processes metadata (demographics, location, device) or item attributes (brand, category, price) to generate an initial vector. As behavioral data accumulates, the collaborative signal gradually dominates. Cross-domain embeddings extend this further by transferring learned representations from a richer domain (e.g., video streaming) to bootstrap personalization in a sparser target domain (e.g., news reading).