Knowledge Graph Embedding

The foundational family of KGE models interprets relations as translations in the embedding space. TransE, the canonical example, models a relation r as a vector translation from head entity h to tail entity t, satisfying h + r ≈ t. The scoring function is the L1 or L2 norm of the resulting vector difference. While computationally efficient and highly interpretable, TransE struggles with 1-to-N, N-to-1, and symmetric relations. Subsequent models like TransH and TransR address this by projecting entities into relation-specific hyperplanes or vector spaces, respectively, before applying the translation operation.

This class of models leverages tensor factorization to capture richer relational patterns. RESCAL models each relation as a full matrix, capturing pairwise interactions between all latent dimensions of the head and tail entities. This expressiveness comes at a quadratic memory cost. DistMult simplifies RESCAL by restricting relation matrices to diagonals, drastically reducing parameters but limiting the model to symmetric relations only. ComplEx extends DistMult into the complex-valued space, where the asymmetric conjugate product of embeddings allows it to elegantly model antisymmetric relations while maintaining linear space complexity.

Modern KGE approaches use Graph Neural Networks (GNNs) as encoders to generate structure-aware embeddings. Instead of learning shallow vectors directly, models like R-GCN learn a function that computes an entity's embedding by aggregating messages from its local neighborhood, with distinct weight matrices for each relation type. This captures multi-hop dependencies and allows the model to generalize to unseen entities during inference. CompGCN further improves this by composing entity and relation embeddings jointly during message passing, reducing parameter explosion for graphs with many relation types.

Rotational models define relations as rotations in complex or quaternion space. RotatE embeds entities and relations in the complex plane and defines each relation as an element-wise rotation from the head entity to the tail entity. This formulation naturally captures symmetry (a 180-degree rotation), inversion (a negative rotation), and composition (the sum of two rotations). The scoring function measures the angular distance after applying the rotation. QuatE extends this to hypercomplex quaternion space, providing more expressive rotational degrees of freedom and capturing complex relational patterns like non-commutative composition.

For knowledge graphs exhibiting hierarchical or tree-like structures, Euclidean space is suboptimal. Hyperbolic models like MuRP embed entities in the Poincaré ball model, where distances grow exponentially as you approach the boundary. This naturally mirrors the exponential branching factor of hierarchies, allowing root nodes to be placed near the origin and leaf nodes near the boundary. Relations are modeled as Möbius transformations—hyperbolic isometries—that map head embeddings to tail embeddings. This geometry provides superior fidelity for taxonomies, organizational structures, and any data with a latent power-law degree distribution.

Training KGEs requires contrasting true triples against corrupted ones. Margin-based ranking loss (hinge loss) enforces a minimum margin between the score of a positive triple and its corrupted negative. Binary cross-entropy with negative sampling treats the task as binary classification, applying a logistic sigmoid to the score. Self-adversarial negative sampling weights negative samples by their current model score, focusing training on the most informative negatives. The 1-N scoring trick accelerates training by scoring a single head entity against an entire batch of tail entities simultaneously, exploiting efficient matrix operations.

The foundational family of models, including TransE, TransH, and TransR, that interpret relations as geometric translations in the embedding space. The core idea is that for a valid triple (head, relation, tail), the embedding of the head plus the relation should approximate the embedding of the tail.

• TransE: The simplest model, enforcing h + r ≈ t. Highly efficient but struggles with complex relation patterns like 1-to-N.
• TransR: Projects entities into a relation-specific space before translation, capturing more nuanced semantics at a higher computational cost.

Models that use tensor factorization or neural networks to measure the semantic similarity of entities and relations, rather than geometric distance. RESCAL is a pioneering bilinear model that captures pairwise interactions between all latent features of the head and tail entities.

• DistMult: A simplified version of RESCAL that uses a diagonal matrix for relations, drastically reducing parameters but limiting it to symmetric relations.
• ComplEx: Extends DistMult into the complex-valued space, allowing it to elegantly model antisymmetric relations (e.g., isParentOf vs. isChildOf) by using the conjugate of the tail embedding.

Modern approaches that use Graph Convolutional Networks (GCNs) or Graph Attention Networks (GATs) to generate embeddings by recursively aggregating information from a node's local neighborhood. This captures the multi-hop relational structure of the knowledge graph.

• R-GCN: A relational GCN that uses distinct weight matrices for each relation type during message passing, explicitly modeling the graph's heterogeneous connections.
• CompGCN: A compositional GCN that jointly embeds entities and relations, using relation-specific operations (like subtraction or multiplication) to combine node and edge features before convolution.

The primary evaluation and training tasks for knowledge graph embeddings. Link prediction aims to predict a missing head or tail entity for an incomplete triple (e.g., ?, bornIn, Berlin). Triplet classification is a binary task to determine if a complete triple (Einstein, bornIn, Ulm) is true or false.

• Evaluation Metrics: Mean Reciprocal Rank (MRR) and Hits@K (e.g., Hits@10) are standard metrics that measure how highly the correct entity is ranked against all other possible entities in the graph.

A key application in hyper-personalization is fusing knowledge graph embeddings with collaborative filtering signals. An item's KG embedding, which encodes its attributes (brand, category, color) and relationships (complementary_to, similar_to), is concatenated with or used to regularize its collaborative embedding.

• Unified Knowledge Graph: A single heterogeneous graph that includes users, items, and all their attributes as nodes. A GNN is trained on this unified graph to generate embeddings that are simultaneously aware of user-item interactions and item-item semantic relationships.

Knowledge graph models are typically trained with a margin-based ranking loss, which forces the score of a true triple to be higher than a corrupted triple by a certain margin. Generating meaningful negative samples is critical.

• Uniform Negative Sampling: Randomly replacing the head or tail of a true triple. Simple but can generate trivially false negatives that don't help the model learn.
• Bernoulli Negative Sampling: Replaces entities with a probability based on the relation's mapping property (e.g., for a 1-to-N relation, it's more likely to replace the 'N' side), creating more challenging and informative negatives.