Knowledge Graph Embedding

The foundational TransE model interprets relations as translations in vector space. For a triple (head, relation, tail), the model learns embeddings such that h + r ≈ t. The scoring function minimizes the distance ||h + r - t||, making it computationally efficient for large graphs. Variants like TransH and TransR extend this by projecting entities onto relation-specific hyperplanes or spaces, handling complex relational patterns like one-to-many mappings.

RESCAL and its derivatives model knowledge graphs as three-way tensors, capturing pairwise interactions between entities through bilinear scoring. ComplEx extends this into complex-valued space, using Hermitian dot products to elegantly model asymmetric relations—a critical capability since many real-world relationships are directional. TuckER applies Tucker decomposition, treating the core tensor as a learned parameter shared across all entities and relations.

Relational Graph Convolutional Networks (R-GCNs) aggregate information from neighboring nodes using relation-specific weight matrices, generating context-aware entity embeddings. CompGCN jointly embeds entities and relations through composition operations like subtraction, multiplication, and circular correlation. These methods capture multi-hop dependencies and local graph structure, outperforming shallow models on inductive link prediction tasks where entities were unseen during training.

RotatE models relations as rotations in complex vector space, defining the scoring function as ||h ∘ r - t|| where ∘ denotes element-wise rotation. This formulation naturally captures symmetry, antisymmetry, inversion, and composition patterns. A relation is symmetric if and only if each element of its embedding has modulus 1, providing a mathematically elegant solution to a long-standing limitation of translation-based models.

Poincaré embeddings and MuRP operate in hyperbolic space rather than Euclidean space, exploiting negative curvature to represent tree-like hierarchies with minimal distortion. In hyperbolic geometry, the volume of a ball grows exponentially with radius—mirroring how the number of descendants grows exponentially with depth in taxonomies. This makes hyperbolic embeddings ideal for modeling organizational structures, biological taxonomies, and ontological hierarchies.

Training relies on contrastive learning: the model scores true triples higher than corrupted ones. Negative sampling generates false triples by randomly replacing the head or tail entity. Advanced strategies like Bernoulli sampling adjust replacement probabilities based on relation cardinality, while self-adversarial sampling weights negatives by their current model scores. The resulting embeddings power downstream tasks including knowledge graph completion and entity alignment across disparate databases.

The primary application of embeddings is predicting missing links in a knowledge graph. By learning the structural patterns of existing triples, models can score the plausibility of unseen facts.

• Mechanism: A scoring function evaluates the likelihood of a triple (head, relation, tail). High scores indicate a probable missing link.
• Example: Given the facts (Marie Curie, discovered, Radium) and (Marie Curie, bornIn, Warsaw), the model can predict (Marie Curie, nationality, Poland).
• Enterprise Use: Automatically completing supply chain graphs or identifying missing regulatory relationships in pharmaceutical ontologies.

Embeddings enable the alignment of equivalent entities across different knowledge graphs or databases without requiring explicit string matching.

• Cross-lingual Mapping: Aligning 'Paris' in an English graph with 'París' in a Spanish graph by comparing their vector neighborhoods.
• Identity Matching: Merging customer records from disparate CRM systems by embedding company names, addresses, and contact points into a unified vector space.
• Technique: Models like MTransE learn a transition matrix to map embeddings from one graph space to another.

Graph embeddings power collaborative filtering by modeling user-item interactions as a heterogeneous knowledge graph, capturing high-order relationships beyond simple purchase history.

• Graph Construction: Nodes represent users, items, and attributes (brand, category). Edges represent purchases, views, and preferences.
• Embedding Propagation: Algorithms like Knowledge Graph Attention Networks (KGAT) recursively propagate embeddings from a node's multi-hop neighbors to refine user and item representations.
• Result: The system can recommend a movie not just because a similar user watched it, but because it shares the same director, genre, and lead actor as previously enjoyed films.

Embeddings bridge the gap between natural language questions and structured knowledge bases by grounding linguistic expressions to graph entities and relations.

• Entity Linking: The phrase 'the father of modern physics' is embedded and matched to the closest entity vector, Albert Einstein.
• Relation Matching: The verb 'wrote' is mapped to the relation authorOf in the graph schema.
• Query Execution: The system traverses the graph from the resolved entity via the resolved relation to retrieve the answer, enabling complex multi-hop reasoning like 'Which awards did scientists born in Germany win?'

Biomedical knowledge graphs embed proteins, diseases, drugs, and genes to computationally predict novel therapeutic associations.

• Polypharmacy Side Effect Prediction: Models like Decagon embed drug-protein interaction networks to predict adverse side effects from drug combinations before clinical trials.
• Drug Repurposing: By embedding the graph of known drug-disease treatments, the model identifies candidate drugs that are geometrically close to a target disease vector but lack an existing treatment edge.
• Mechanism: Translational models learn that the vector offset between a disease and its target protein is similar to the offset between a drug and its mechanism of action.

Financial transaction networks are modeled as graphs where embeddings capture latent behavioral patterns indicative of fraud.

• Graph Construction: Nodes represent cardholders, merchants, and devices. Edges represent transactions with attributes like amount and time.
• Anomaly Detection: A new transaction is scored against the learned embeddings. A transaction vector that deviates significantly from the typical embedding neighborhood of a user or merchant is flagged as suspicious.
• Advantage: Unlike feature-based models, graph embeddings automatically learn relational patterns like 'a device shared by multiple accounts with rapidly changing shipping addresses,' which are classic fraud rings.

A foundational family of embedding methods that model relations as translations in the vector space. The core assumption is that for a valid triple (head, relation, tail), the embedding of the tail entity should be close to the head entity plus the relation vector.

• TransE: The simplest model where h + r ≈ t, optimized with L1 or L2 norm
• TransH: Projects entities onto relation-specific hyperplanes to handle complex relation types
• TransR: Maps entities into distinct relation spaces via projection matrices
• RotatE: Models relations as rotations in complex space, capturing symmetry and inversion patterns

These models excel at link prediction and are computationally efficient, though they struggle with one-to-many and many-to-many relationships.

Embedding approaches that measure plausibility of triples using similarity-based scoring functions rather than distance metrics. These models leverage tensor factorization and neural networks to capture richer relational patterns.

• RESCAL: Factorizes the knowledge graph as a three-way tensor, capturing pairwise interactions via relation-specific matrices
• DistMult: A simplified bilinear model using diagonal matrices, efficient but limited to symmetric relations
• ComplEx: Extends DistMult into complex-valued embeddings, naturally handling asymmetric and antisymmetric relations
• TuckER: Applies Tucker decomposition to the full tensor, enabling multi-task learning across relations

Semantic matching models achieve state-of-the-art results on benchmarks like WN18RR and FB15k-237.

Modern approaches that use message-passing neural networks to generate entity embeddings by aggregating information from local graph neighborhoods. Unlike translational models, GNNs capture multi-hop structural dependencies.

• R-GCN: Relational Graph Convolutional Networks that apply relation-specific weight matrices during neighborhood aggregation
• CompGCN: Composition-based GCN that jointly embeds entities and relations using composition operators like subtraction and multiplication
• HGT: Heterogeneous Graph Transformer that uses type-specific attention mechanisms for multi-relational graphs

GNN-based embeddings are particularly effective for inductive link prediction, where entities unseen during training must be embedded at inference time.

Embedding techniques that map entities into hyperbolic space rather than Euclidean space to better capture hierarchical structures inherent in many knowledge graphs. Hyperbolic geometry has negative curvature, causing volume to grow exponentially with radius.

• Poincaré embeddings: Represent entities on the Poincaré ball model, where distances reflect taxonomic similarity
• MuRP: Multi-relational Poincaré model that embeds entities in hyperbolic space while keeping relations in tangent space
• AttH: Hyperbolic attention model that learns curvature parameters per relation

Hyperbolic embeddings achieve superior performance on hierarchical relation types like hypernym and part_of, common in ontologies and taxonomies.

Extensions of static embedding methods that incorporate time as a fourth dimension to model facts that are only valid during specific intervals. Temporal KGs represent facts as quadruples (head, relation, tail, timestamp).

• TTransE: Extends TransE by modeling time as an additional translation vector
• HyTE: Projects entities onto time-specific hyperplanes, allowing representations to evolve across timestamps
• DE-SimplE: Uses diachronic embeddings where entity vectors are functions of time, trained with temporal regularization
• TNTComplEx: Combines ComplEx with fourth-order tensor factorization for temporal reasoning

Critical for applications like event forecasting, financial transaction analysis, and dynamic knowledge base maintenance.

The critical training technique that generates corrupted triples to teach the model to distinguish true facts from false ones. Since knowledge graphs only contain positive examples, negative sampling defines the contrastive learning signal.

• Uniform sampling: Randomly replace head or tail entity with any entity from the vocabulary
• Bernoulli sampling: Biases corruption toward replacing the head for one-to-many relations and the tail for many-to-one relations
• Adversarial sampling: Dynamically selects hard negatives that the current model scores highly, improving discrimination
• Self-adversarial sampling: Weights negative samples by their current model score without full adversarial generation

Poor negative sampling leads to false positive predictions and degraded link prediction accuracy.