Next-Item Prediction

The model processes a chronologically ordered sequence of user actions [i₁, i₂, ..., iₜ₋₁] and learns to predict the target item iₜ. This formulation treats user behavior as a language-like stream where the order of interactions encodes evolving intent. Architectures like Transformers and Recurrent Neural Networks are employed to capture both long-range dependencies and local session context, making the prediction conditional on the entire preceding history rather than a static profile.

This objective eliminates the need for manual labeling by deriving supervisory signals directly from the temporal structure of interaction logs. The ground truth is the actual next-clicked item, and the model is penalized for ranking it below other candidates. Common loss functions include:

• Cross-entropy loss over the full item catalog
• Bayesian Personalized Ranking (BPR) for pairwise preference optimization
• InfoNCE loss in contrastive learning frameworks The learned internal representations become powerful user embeddings transferable to downstream tasks like click-through rate prediction and session-based recommendation.

During training, causal attention masking ensures the model cannot attend to future tokens in the sequence. For a user action at position t, the self-attention mechanism is restricted to positions [1, t-1]. This prevents information leakage and forces the model to make predictions based solely on historical context, mirroring the constraints of real-time inference where future actions are unknown. Violating this constraint would result in artificially inflated offline metrics that fail to generalize to production environments.

Computing a softmax over millions of items is computationally prohibitive. Efficient training relies on negative sampling to approximate the full distribution:

• In-batch negatives: Other items in the same mini-batch serve as negatives, reusing computed embeddings
• Uniform random sampling: Randomly selecting items from the catalog as negatives
• Popularity-based sampling: Correcting for the popularity bias by sampling negatives proportional to item frequency
• Hard negative mining: Selecting items that the model currently ranks highly but are incorrect, accelerating convergence

Raw event streams must be partitioned into meaningful sequences through sessionization. A session boundary can be defined by:

• Time-based thresholds: 30 minutes of inactivity
• Task completion signals: Purchase or checkout events
• Fixed-length sliding windows: Truncating or padding sequences to a uniform length N for efficient batch processing The choice of sessionization strategy directly impacts the semantic coherence of the learned embeddings, as it defines what the model considers a contiguous behavioral context.

Model quality is assessed using ranking metrics computed on a held-out test set:

• Recall@K: The fraction of test sequences where the true next item appears in the top-K predictions
• Mean Reciprocal Rank (MRR): The average of the reciprocal of the rank at which the first correct item is found
• Normalized Discounted Cumulative Gain (NDCG): A position-aware metric that weights hits at higher ranks more heavily
• Hit Rate (HR@K): A binary measure of whether the correct item is in the top-K, averaged across all test instances

A self-attention-based architecture that processes chronologically ordered user actions to capture long-range dependencies for next-item prediction. Unlike RNNs, transformers attend to all positions simultaneously, enabling parallel training and superior modeling of complex sequential patterns.

• Uses causal masking to prevent attending to future items
• Captures both short-term session intent and long-term preference evolution
• Forms the backbone of state-of-the-art models like SASRec and BERT4Rec

A self-supervised paradigm that learns representations by pulling semantically similar pairs closer in embedding space while pushing apart dissimilar pairs. In next-item prediction, the positive pair is the actual next interaction, while negatives are randomly sampled or in-batch items.

• Uses InfoNCE loss to define the contrastive objective
• Eliminates the need for explicit negative sampling in some architectures
• Produces highly discriminative embeddings that separate distinct user intents

A dynamic user representation that encodes the temporal order of actions to capture evolving intent. Unlike static collaborative filtering embeddings, sequence-aware vectors change as the user's session progresses.

• Built using recurrent neural networks or transformer encoders
• Distinguishes between browsing, comparison, and purchase intent phases
• Enables real-time adaptation to session context without full model retraining

An efficient training approximation that updates only a small random subset of negative items during loss computation. For next-item prediction with millions of catalog items, computing the full softmax is computationally prohibitive.

• In-batch negative sampling reuses other positive items in the mini-batch as negatives
• Sampled softmax randomly selects a few thousand negatives per training step
• Balances computational efficiency with gradient approximation quality

A technique that decomposes a user's embedding into multiple distinct prototype vectors, each representing a different latent interest. This addresses the limitation that a single embedding cannot capture diverse, sometimes contradictory preferences.

• Uses dynamic routing or multi-head attention to separate interests
• Enables diverse next-item recommendations across categories
• Particularly effective for users with broad shopping patterns across electronics, books, and apparel

A representation strategy for new users or items with no interaction history. Next-item prediction models typically fail without behavioral data, so cold-start embeddings bridge the gap using side information.

• Generated from content metadata (item descriptions, categories) or demographic features
• Uses a content-based tower trained alongside the collaborative model
• Gradually transitions to behavioral embeddings as interaction signals accumulate