Neural Collaborative Filtering (NCF)

Traditional Matrix Factorization estimates interactions via a linear inner product, which assumes latent dimensions are independent. NCF replaces this with a Multi-Layer Perceptron (MLP) that learns a non-linear function from data. This allows the model to capture complex, non-linear decision boundaries in the interaction space, such as a user liking movies that are either very short or very long but disliking medium-length ones—a pattern a linear dot product cannot express.

GMF is a neural generalization of Matrix Factorization. Instead of using a fixed dot product, GMF applies an element-wise product to the user and item embeddings, then passes the result through a dense layer with a learned activation function. This allows the model to learn the optimal weighting of latent dimensions, effectively deciding which feature interactions matter most. GMF serves as the linear baseline component within the full NCF architecture.

The MLP branch concatenates user and item embeddings and passes them through multiple hidden layers with ReLU activations. This deep pathway learns high-order, non-linear feature interactions that GMF cannot. Key design choices include:

• Tower pattern: Hidden layers halve in size progressively (e.g., 64 → 32 → 16 → 8)
• Batch normalization: Applied after each layer for training stability
• Dropout: Regularizes the network to prevent overfitting on sparse interaction data

The Neural Matrix Factorization (NeuMF) model fuses the GMF and MLP branches into a unified architecture. Both branches share the same input embeddings but learn separate representations. Their final hidden layers are concatenated and passed through a sigmoid output layer to produce the interaction probability. This hybrid design lets the model learn both low-rank linear structure (via GMF) and complex non-linear patterns (via MLP) simultaneously.

NCF is designed for implicit feedback (clicks, views, purchases) rather than explicit ratings. The output is treated as a binary classification problem—predicting the probability of interaction. Training uses binary cross-entropy loss with negative sampling: for each observed user-item pair, a small set of unobserved items is randomly sampled as negatives. This is more effective than pointwise regression for ranking tasks.

Training the full NeuMF model from scratch with random initialization can lead to slow convergence. The recommended approach is a two-phase pre-training:

• Phase 1: Train GMF and MLP branches independently with random initialization
• Phase 2: Initialize NeuMF with the pre-trained weights and fine-tune jointly using a lower learning rate This strategy combines the stability of pre-trained components with the expressiveness of joint optimization.

The linear predecessor to NCF that decomposes the user-item interaction matrix into latent factor vectors. NCF's key innovation is replacing the inner product operation with a neural network.

• Learns low-dimensional embeddings for users and items
• Interaction modeled as dot product: r_ui = p_u^T q_i
• Limited to linear relationships between latent factors
• Solved via Alternating Least Squares (ALS) or SGD

A component of the NCF framework that generalizes traditional MF by applying an element-wise product and a learned weighting layer with a non-linear activation.

• Computes: phi(p_u ⊙ q_i) where phi is a dense layer
• With identity activation and uniform weights of 1, GMF reduces to standard MF
• Proves that NCF is a strict superset of matrix factorization
• Often combined with MLP in the NeuMF hybrid architecture

The deep branch of the NCF architecture that learns non-linear user-item interactions through stacked fully-connected layers with ReLU activations.

• Concatenates user and item embeddings: z = [p_u; q_i]
• Passes through multiple hidden layers with decreasing dimensionality
• Captures complex, non-linear feature interactions that dot products miss
• Standard NCF uses a tower pattern: wide layers tapering to narrower ones

The fusion architecture that combines GMF and MLP branches, allowing the model to learn both linear and non-linear interaction patterns simultaneously.

• GMF and MLP process separate embedding spaces for flexibility
• Outputs are concatenated and passed through a final sigmoid layer
• Pre-training trick: initialize with trained GMF and MLP weights
• Achieves state-of-the-art performance on implicit feedback datasets like MovieLens and Pinterest

NCF is specifically designed for binary implicit signals (clicks, views, purchases) rather than explicit ratings. The output is treated as a probability of interaction.

• Training data: positive instances (observed interactions) + negative sampling
• Loss function: binary cross-entropy (pointwise loss)
• Probabilistic interpretation: P(y=1 | u, i) via sigmoid output
• Contrasts with Bayesian Personalized Ranking (BPR) which uses pairwise ranking loss

The dense vector representations for users and items that feed into the NCF architecture. Proper initialization significantly impacts convergence.

• Random initialization with Xavier/Glorot uniform distribution
• Can be pre-trained via standard MF for warm-starting NeuMF
• Embedding dimension is a critical hyperparameter (typically 8-64)
• Shared vs. separate embeddings for GMF and MLP branches is a key design choice