Collaborative Filtering

The original neighborhood-based approach that identifies users with similar taste profiles to generate recommendations.

How it works:

• Computes similarity scores between a target user and all other users in the system
• Identifies a k-nearest neighbor cohort with the highest correlation coefficients
• Aggregates items highly rated by the neighbor set but not yet seen by the target user

Common similarity metrics:

• Pearson correlation — accounts for differences in rating scale between users
• Cosine similarity — treats user rating vectors as points in n-dimensional space
• Jaccard index — useful for implicit binary feedback like clicks or purchases

Limitation: Computational complexity grows quadratically with user base size, making it impractical for platforms with millions of users without significant approximation techniques.

The approach popularized by Amazon's original recommendation engine, which focuses on item similarity rather than user similarity.

Key characteristics:

• Computes pairwise item-to-item similarity based on co-occurrence patterns in user interaction histories
• Builds a static item similarity matrix that can be pre-computed offline
• At runtime, recommendations are generated by finding items similar to those the user has already interacted with

Advantages over user-user:

• Item relationships are more stable over time than user relationships
• The similarity matrix is bounded by the item catalog size, not the user base
• Supports real-time personalization with minimal latency after the offline computation phase

Example: A user who purchases a specific camera model receives recommendations for lenses and accessories frequently bought alongside that camera by other customers.

A model-based technique that decomposes the sparse user-item interaction matrix into dense, low-dimensional latent factor representations.

Core mechanism:

• Represents each user and each item as a vector of latent features (typically 50-200 dimensions)
• The dot product of a user vector and an item vector predicts the user's rating for that item
• Learns these vectors by minimizing reconstruction error on known ratings using Stochastic Gradient Descent (SGD) or Alternating Least Squares (ALS)

Key algorithms:

• Singular Value Decomposition (SVD) — the foundational linear algebra technique
• SVD++ — incorporates implicit feedback signals like browsing history
• Non-Negative Matrix Factorization (NMF) — constrains factors to be non-negative for interpretability

The Netflix Prize demonstrated the power of this approach, with the winning solution heavily relying on matrix factorization techniques.

Modern neural architectures that replace the linear dot product of matrix factorization with non-linear function approximators to capture complex interaction patterns.

Architecture patterns:

• Neural Collaborative Filtering (NCF) — feeds concatenated user and item embeddings through multi-layer perceptrons
• Autoencoder-based models — reconstruct user rating vectors through a bottleneck layer, learning compressed representations
• Wide & Deep models — combine memorization of specific co-occurrence rules (wide component) with generalization through embeddings (deep component)

Advantages:

• Captures non-linear and hierarchical relationships that linear factorization misses
• Can naturally incorporate side information like user demographics, item metadata, and temporal context
• Supports multi-modal inputs including text descriptions and images through parallel embedding towers

Trade-off: Requires significantly more training data and compute resources compared to classical matrix factorization.

Collaborative filtering systems operate on fundamentally different signal types, each requiring distinct modeling approaches.

Explicit feedback:

• Direct user expressions of preference such as star ratings, thumbs up/down, or reviews
• High signal quality but extremely sparse — most users rate very few items
• Enables precise regression-based prediction of exact rating values

Implicit feedback:

• Behavioral signals inferred from user actions: purchases, clicks, watch time, scroll depth, search queries
• Abundant and passively collected but noisy — a purchase may be a gift, a click may be accidental
• Typically modeled as binary classification or confidence-weighted regression problems

Modern systems predominantly rely on implicit signals due to their abundance, using techniques like weighted alternating least squares (WALS) that assign higher confidence to observed interactions while treating unobserved interactions as negative with low confidence.

The fundamental limitation of collaborative filtering where the system fails to make accurate predictions for new users or new items with insufficient interaction history.

User cold-start:

• A new user joins the platform with no rating or behavioral history
• The system has no basis for finding similar neighbors or computing latent factors
• Mitigation: onboarding surveys, demographic-based initial recommendations, popularity-based fallback

Item cold-start:

• A new item is added to the catalog with zero user interactions
• Collaborative signals are nonexistent until sufficient user feedback accumulates
• Mitigation: content-based bootstrapping using item metadata, exploration-exploitation strategies like multi-armed bandits

Hybrid recommendation systems explicitly address this by combining collaborative filtering with content-based methods that can operate on item features alone, providing reasonable recommendations until collaborative signals mature.

The fundamental limitation of collaborative filtering when insufficient interaction data exists for new users or items. A system cannot find peer patterns for a user with no history, nor recommend an item that has never been rated.

• User cold-start addressed via onboarding surveys and demographic defaults
• Item cold-start mitigated by content-based fallback and metadata injection
• System cold-start requires bootstrapping with curated initial interactions
• Hybrid architectures are the primary engineering solution to this challenge