TransE

The core geometric intuition of TransE is that for a valid triple (h, r, t), the embedding of the head entity plus the embedding of the relation should approximately equal the embedding of the tail entity: h + r ≈ t. This models a relationship as a simple vector translation from the head to the tail in the embedding space. The model learns by minimizing a margin-based ranking loss that scores true triples higher than corrupted ones.

TransE uses a distance-based scoring function to measure the plausibility of a triple. The score is defined as the L1 or L2 norm of the vector difference:

• fr(h, t) = -||h + r - t||

A lower distance indicates a more plausible triple. During training, the model pushes true triples to have small distances while forcing corrupted triples—where either the head or tail is randomly replaced—to have larger distances by a margin.

TransE is trained using a margin-based ranking loss with negative sampling. For each true triple, the model generates corrupted triples by replacing the head or tail entity with a random entity. The loss function is:

• L = Σ max(0, γ + d(h+r, t) - d(h'+r, t'))

Where γ is the margin hyperparameter. This forces the model to separate true triples from false ones by at least the margin value, learning a structured embedding space.

Strengths:

• Computational efficiency: Few parameters and simple vector operations make it scalable to large graphs like Freebase
• Interpretable geometry: The translation operation is intuitive and easy to visualize
• Strong on 1-to-1 relations: Performs well when each head maps to a single tail

Limitations:

• Struggles with complex relations: Cannot model symmetric, transitive, or 1-to-N relations effectively
• Compositional patterns: Fails to capture relation hierarchies and compositions naturally
• Overly simplistic: The translation assumption breaks down for reflexive relations where h = t

TransE's limitations inspired a family of translation-based models that project entities into relation-specific spaces:

• TransH: Projects entities onto a relation-specific hyperplane before translation, handling 1-to-N relations better
• TransR: Projects entities into a distinct relation space using a learned projection matrix, at the cost of more parameters
• TransD: Decomposes the projection matrix into two vectors for efficiency while maintaining expressiveness
• RotatE: Models relations as rotations in complex space, naturally capturing symmetry and composition patterns

TransE is primarily used for knowledge base completion and link prediction tasks:

• Link Prediction: Predicting missing head or tail entities for a given relation, such as finding a person's nationality or a film's director
• Triple Classification: Binary classification of whether a proposed triple is true or false
• Entity Resolution: Using learned embeddings to identify when different mentions refer to the same real-world entity
• Relation Extraction: Providing dense features for downstream classifiers that identify semantic relationships in text