Collaborative Filtering

The foundational memory-based approach that identifies a neighborhood of like-minded users for a target individual.

• Computes similarity scores between the target user and all others using metrics like Pearson correlation or cosine similarity on overlapping item ratings.
• Predicts a user's rating for an unseen item by aggregating the weighted ratings of their top-K most similar neighbors.
• Highly interpretable but computationally expensive at scale, as the entire user-user similarity matrix must be recalculated frequently.

A memory-based method that builds a model of item relationships rather than user relationships, famously powering Amazon's original recommendation engine.

• Pre-computes an item-item similarity matrix offline by analyzing co-occurrence patterns—items frequently rated or purchased together are deemed similar.
• At inference time, recommendations are generated instantly by finding items similar to those the user has already interacted with.
• More stable and scalable than user-user filtering because item relationships change slowly, allowing for infrequent batch recomputation.

A model-based technique that decomposes the sparse user-item interaction matrix into dense, low-dimensional latent vectors representing users and items in a shared space.

• Algorithms like Singular Value Decomposition (SVD) and Alternating Least Squares (ALS) learn latent factors that capture underlying characteristics such as genre affinity or price sensitivity.
• The dot product between a user vector and an item vector predicts the user's affinity for that item, enabling efficient scoring of the entire catalog.
• Addresses the sparsity problem inherent in memory-based methods by learning a compact, generalizable representation from limited feedback.

Collaborative filtering systems are fundamentally shaped by the type of signal they consume, each with distinct modeling implications.

• Explicit feedback: Direct user evaluations such as star ratings, likes, or thumbs-up. Sparse but provides clear negative and positive signals.
• Implicit feedback: Indirect behavioral signals such as clicks, purchases, watch time, or dwell time. Abundant but inherently noisy and lacks true negative examples.
• Modern systems predominantly rely on implicit feedback, treating interaction counts as a confidence-weighted proxy for preference using techniques like weighted alternating least squares.

The fundamental limitation of pure collaborative filtering: it cannot make predictions for users or items with no historical interaction data.

• User cold start: A new user has no rating history, making neighborhood formation or latent vector placement impossible without a fallback strategy.
• Item cold start: A newly added product cannot be recommended until it accumulates sufficient user interactions to establish similarity relationships.
• Mitigation requires hybrid approaches that incorporate content-based features or demographic side-information to bootstrap recommendations until collaborative signals emerge.

Collaborative filtering implementations fall into two broad computational paradigms with distinct trade-offs in scalability and accuracy.

• Neighborhood (memory-based): Operates directly on the user-item matrix in memory. Highly transparent—you can explain a recommendation by pointing to specific similar users or items. Suffers from O(n²) complexity.
• Model-based: Learns a compact parametric model from the data, such as latent factor vectors or a neural network. More scalable and generalizes better to unseen interactions, but acts as a black box.
• Production systems often combine both: model-based for candidate generation and neighborhood-based for post-hoc explanation.

A latent factor model that decomposes the sparse user-item interaction matrix into a product of two dense, lower-dimensional matrices representing user and item latent vectors. This technique uncovers hidden features—like a user's preference for a specific genre—by learning vector representations via optimization methods such as Alternating Least Squares (ALS) or Stochastic Gradient Descent (SGD). It forms the mathematical backbone of traditional collaborative filtering.

A deep learning framework that replaces the linear inner product of traditional matrix factorization with a neural architecture capable of learning arbitrary non-linear user-item interaction functions. By using multi-layer perceptrons on top of concatenated user and item embeddings, NCF can model complex, high-order feature interactions that linear models miss, significantly improving prediction accuracy on implicit feedback datasets.

An efficient neural architecture that independently encodes user and item features into separate embedding vectors within a shared latent space. The user tower processes user features and context, while the item tower processes item attributes. Candidate retrieval is performed via Approximate Nearest Neighbor (ANN) search on the item embeddings, making this the industry-standard architecture for large-scale retrieval stages in systems like YouTube's recommendation engine.

A pairwise learning-to-rank optimization criterion designed for implicit feedback. Rather than predicting absolute scores, BPR maximizes the probability that a user prefers an observed item over an unobserved one, treating the task as a binary classification problem on item triplets. This loss function is a standard training objective for learning latent factor models when only positive interactions like clicks or purchases are available.

The fundamental challenge of providing accurate recommendations for new users or items with no historical interaction data. Mitigation strategies include:

• Content-based filtering: using item metadata or user demographics
• Hybrid models: combining collaborative and content-based signals
• Exploration policies: using contextual bandits to gather initial feedback
• Meta-learning: training models that can adapt rapidly from few examples

The core dilemma in sequential recommendation where a system must choose between exploiting known high-reward items and exploring uncertain items to gather new preference data. Contextual Bandits provide a principled framework for this, learning a policy that maps observed user features to actions while balancing immediate reward against long-term information gain. This is critical for adapting to shifting user tastes and surfacing serendipitous discoveries.