Collaborative Filtering

The foundational approach that identifies like-minded users based on overlapping interaction histories. The system computes a similarity matrix using metrics like Pearson correlation or cosine similarity between user rating vectors. A target user receives recommendations for items highly rated by their nearest neighbors but not yet seen. This method excels at serendipitous discovery but suffers from scalability issues, as the computational complexity grows quadratically with the user base, making it impractical for platforms with millions of users without significant approximation techniques.

A more scalable alternative pioneered by Amazon that analyzes the co-occurrence of items in user histories rather than comparing users directly. The algorithm pre-computes an item-to-item similarity matrix offline by determining how often two items are consumed by the same user. For real-time recommendations, the system simply looks up items similar to those in the user's current session. This approach provides transparent explanations—'Users who bought X also bought Y'—and remains stable as the user base grows, making it ideal for e-commerce and content platforms with relatively static catalogs.

A latent factor model that decomposes the sparse user-item interaction matrix into dense, low-dimensional embeddings. Techniques like Singular Value Decomposition (SVD) and Alternating Least Squares (ALS) learn latent vectors representing user preferences and item characteristics in a shared space. The dot product of a user vector and an item vector predicts the rating. This approach handles sparsity effectively and was popularized by the Netflix Prize competition. Modern variants incorporate implicit feedback—clicks, views, dwell time—rather than relying solely on explicit ratings, dramatically increasing the available training signal.

A fundamental architectural distinction in collaborative filtering systems. Memory-based methods retain the entire interaction dataset and perform lazy computation at query time, using nearest-neighbor search over the raw data. They are simple to implement and adapt quickly to new data but incur high latency. Model-based methods pre-train a compact statistical representation offline, enabling fast inference at the cost of periodic retraining. Production systems often employ a hybrid approach, using model-based techniques for candidate generation and memory-based filtering for final re-ranking to balance scalability with freshness.

A critical limitation where collaborative filtering fails for new users with no interaction history and new items with no ratings. Mitigation strategies include:

• Hybrid filtering: Combining collaborative signals with content-based metadata about item attributes
• Demographic bootstrapping: Using user profile information to assign initial segment-based recommendations
• Exploration heuristics: Employing multi-armed bandit algorithms like Thompson Sampling to efficiently gather initial preference data
• Knowledge transfer: Leveraging learned embeddings from auxiliary domains or platforms to initialize new entity representations

A deep learning evolution that replaces the linear dot product of matrix factorization with a multi-layer perceptron to model complex, non-linear user-item interactions. Architectures like NCF learn an arbitrary function from concatenated user and item embeddings, capturing higher-order feature interactions that linear models miss. Advanced variants incorporate attention mechanisms to weigh the importance of different historical interactions and autoencoder-based structures to reconstruct the full rating vector from a corrupted input. These models excel at modeling implicit feedback signals where the absence of interaction is ambiguous—it could indicate dislike or simply unawareness.

A foundational technique that decomposes the sparse user-item interaction matrix into dense, lower-dimensional latent factor vectors. By representing users and items in a shared latent space, the model predicts missing ratings through the dot product of these vectors.

• Singular Value Decomposition (SVD) is the classic algorithm, often regularized to prevent overfitting
• Alternating Least Squares (ALS) is preferred for implicit feedback datasets at scale
• Latent factors capture abstract properties like genre affinity or price sensitivity without explicit feature engineering

The original collaborative filtering paradigm that computes predictions directly from a subset of similar users or items, rather than learning a global model.

• User-based k-NN: Finds the k most similar users and aggregates their ratings to predict a target user's preference
• Item-based k-NN: Pre-computes item-to-item similarity matrices, enabling fast lookups—famously used by Amazon's early recommendation engine
• Similarity is typically measured via cosine similarity or Pearson correlation coefficient, with significance weighting applied to penalize neighbors with few co-rated items

The failure mode where collaborative filtering cannot generate reliable recommendations for new users or new items due to a complete absence of interaction history.

• User cold start: A new user has no ratings, making neighbor identification impossible
• Item cold start: A new item has never been rated, so it never appears in recommendations
• Mitigation strategies include hybrid recommenders that incorporate content-based features (metadata, descriptions) and exploration-exploitation policies that actively gather initial feedback

Behavioral signals inferred from user actions rather than explicit ratings. This data is far more abundant but requires specialized modeling to handle the absence of negative feedback.

• Signals include: clicks, purchases, watch time, dwell time, search queries, and scroll depth
• Confidence weighting distinguishes between high-confidence positive signals (a purchase) and low-confidence negatives (no interaction may simply indicate unawareness)
• The weighted alternating least squares (WALS) algorithm is specifically designed for implicit datasets, treating all unobserved interactions as low-confidence negatives

Production collaborative filtering at internet scale requires efficient nearest-neighbor retrieval to serve recommendations with sub-100ms latency from catalogs of millions of items.

• Approximate Nearest Neighbor (ANN) algorithms like FAISS, ScaNN, and Annoy trade a small amount of accuracy for orders-of-magnitude speed improvements
• Two-tower models separately encode users and items into embedding vectors, enabling pre-computed item embeddings and real-time user encoding
• Locality-Sensitive Hashing (LSH) partitions the embedding space so similar vectors collide in the same hash bucket with high probability

Offline evaluation of collaborative filtering models requires metrics that capture ranking quality, not just prediction accuracy, since users only interact with top-ranked recommendations.

• Precision@k and Recall@k: Measure the fraction of top-k recommendations that are relevant, and the fraction of all relevant items captured in the top-k
• Mean Average Precision (MAP): Averages precision across all recall levels, penalizing relevant items placed lower in the ranking
• Normalized Discounted Cumulative Gain (NDCG): Weights hits by their position, giving exponentially more credit to relevant items at the top of the list