Neural Knowledge Base Completion

These models represent entities and relations as vectors in a continuous space, where the relationship between two entities is modeled as a translation operation. The core idea is that for a true triple (head, relation, tail), the embedding of the head plus the embedding of the relation should be close to the embedding of the tail.

Key Examples:

• TransE: The foundational model using simple vector addition: h + r ≈ t.
• TransH: Projects entities onto relation-specific hyperplanes to better model complex relations like one-to-many.
• TransR: Uses separate projection matrices for entities into relation-specific spaces.

These models are trained with a margin-based ranking loss that scores true triples higher than corrupted ones.

Instead of translation, these models measure the semantic similarity between the head and tail entities in the context of a given relation. They typically use a scoring function based on bilinear products or neural networks.

Key Architectures:

• RESCAL: A bilinear model that represents the entire knowledge graph as a 3-way tensor, factorized using a rank-r decomposition.
• DistMult: A simplified, efficient version of RESCAL that uses a diagonal matrix for each relation, reducing parameters.
• ComplEx: Extends DistMult into the complex number domain to better handle asymmetric relations (e.g., personBornIn vs. cityHasBirth).
• Analogy: Uses analogical structures for embedding, capturing relational patterns like symmetry and inversion.

These models directly operate on the graph structure of the knowledge base. They aggregate information from an entity's local neighborhood to generate refined, context-aware embeddings for link prediction.

Core Mechanism:

• Message Passing: Each entity's representation is updated by aggregating (sum, mean) the representations of its connected neighbors, transformed by the relevant relation.
• Multi-Hop Reasoning: By stacking GNN layers, the model can incorporate information from k hops away, enabling more complex relational inferences.

Prominent Frameworks:

• R-GCN (Relational Graph Convolutional Network): Introduces relation-specific transformations in the convolution operation.
• CompGCN: A composition-based GNN that jointly embeds entities and relations, efficiently composing them using operations like subtraction or multiplication.

Adapting the highly successful Transformer architecture, these models treat triples as sequences or use attention mechanisms to weigh the importance of different paths and relations in the graph.

Approaches:

• KG-BERT: Treats a triple (h, r, t) as a text sequence (e.g., [CLS] head [SEP] relation [SEP] tail [SEP]) and uses a pre-trained language model like BERT to score its plausibility.
• Graph Attention Networks (GATs) for KGs: Use attention mechanisms to learn which neighboring nodes are most important for predicting a missing link, rather than simple aggregation.
• Path-Based Transformers: Encode sequences of relations forming paths between entities as input, allowing the model to perform multi-step logical inference.

These hybrid models integrate symbolic, logical rules (e.g., marriedTo(X, Y) ⇒ marriedTo(Y, X)) with neural networks to constrain and improve predictions, ensuring logical consistency.

Integration Techniques:

• Rule Injection as Regularization: Add a loss term that penalizes predictions violating pre-defined or learned logical rules.
• Differentiable Rule Reasoning: Use frameworks like Neural Logic Programming (NeuralLP) or Differentiable Inductive Logic Programming (∂ILP) to learn rules jointly with embeddings.
• Iterative Knowledge Infusion: Models like IterE alternately perform knowledge graph embedding and logical rule mining, each step refining the other.

This approach is key for applications requiring explainability and trust, as predictions can be justified by logical chains.

Model performance is rigorously measured using standard metrics on curated datasets. Understanding these is crucial for comparing approaches.

Core Metrics:

• Mean Rank (MR): The average rank of the true entity when the model scores all possible candidates. Lower is better.
• Mean Reciprocal Rank (MRR): The average of the reciprocal of the ranks of the true entities. Higher is better, more robust to outliers than MR.
• Hits@K: The percentage of test cases where the true entity appears in the top K ranked predictions. Common values are Hits@1, Hits@3, Hits@10.

Standard Benchmark Datasets:

• WN18RR & FB15k-237: Filtered versions of WordNet and Freebase, created to remove reversible test triples that allow trivial inference, providing a more realistic challenge.
• YAGO3-10: A large-scale dataset with high relational complexity, derived from the YAGO knowledge base.

A foundational technique for Neural KBC where entities and relations in a knowledge graph are mapped to continuous vector representations (embeddings). Models like TransE, ComplEx, and RotatE score the plausibility of a triple (head, relation, tail) by performing algebraic operations in the embedding space. This enables:

• Link Prediction: Ranking candidate entities to complete a missing triple.
• Semantic Similarity: Measuring relatedness between entities in the latent space.
• Downstream Integration: Providing features for more complex reasoning models.

A class of neural networks designed to operate directly on graph-structured data. For KBC, Relational Graph Neural Networks (R-GNNs) and their variants are pivotal. They:

• Aggregate Neighborhood Information: Update an entity's representation by combining features from its connected neighbors.
• Model Relational Context: Use separate parameters or attention mechanisms for different relation types.
• Enable Multi-Hop Reasoning: Capture complex dependencies beyond immediate edges, which is critical for inferring missing links through longer paths in the graph.

The symbolic process of automatically discovering logical rules (e.g., BornInCity(X, Y) ∧ CityInCountry(Y, Z) ⇒ Nationality(X, Z)) from a knowledge base. Neural KBC systems often integrate or are evaluated against such rules.

• Symbolic Baselines: Systems like AMIE+ generate Horn rules with confidence scores.
• Neuro-Symbolic Integration: Models use these rules as constraints during training (symbolic regularization) or to guide inference.
• Interpretability: Induced rules provide human-understandable explanations for predicted links.

The ability to infer new facts by chaining multiple existing relations across a knowledge graph. Neural KBC models must often perform this implicitly. Key approaches include:

• Path-Based Models: Encode sequences of relations as paths and score their validity.
• Query Embedding (e.g., Query2Box): Represent complex logical queries (like conjunctive queries) as geometric operations in embedding space.
• Recursive GNNs: Traverse the graph structure iteratively to answer queries requiring multiple inference steps.

A more challenging variant of KBC where the model must make predictions for entities not seen during training. This tests a model's ability to generalize based on learned relational patterns and descriptions.

• Requires Entity Generalization: Models cannot rely on pre-learned embeddings for every entity.
• Uses Auxiliary Information: Often leverages textual descriptions (e.g., Wikipedia abstracts) or attribute data to create representations for novel entities.
• Critical for Dynamic Graphs: Essential for real-world applications where new entities (e.g., new products, people) are constantly added.

A neuro-symbolic paradigm where discrete logical inference steps are made continuous and differentiable, allowing end-to-end training with gradient descent. This directly supports Neural KBC by:

• Integrating Logic and Learning: Enforcing logical constraints (e.g., transitivity, symmetry of relations) via differentiable loss functions.
• Enabling Gradient-Based Rule Learning: As seen in Differentiable Inductive Logic Programming (∂ILP), which learns logic programs from examples.
• Bridging Symbolic Predictions: Connecting the output of neural link predictors to a coherent, consistent symbolic knowledge base.