Entity Embedding

The foundational property learned by models like TransE, where the relationship between two entities is modeled as a vector translation in the embedding space. For a valid triple (head, relation, tail), the equation head + relation ≈ tail holds true.

• Geometric interpretation: The relation vector acts as a displacement from the head entity to the tail entity
• Scoring function: f(h, r, t) = -||h + r - t|| — lower distance indicates higher plausibility
• Limitation: Struggles with symmetric relations and 1-to-N mappings due to the rigid translation constraint

Entity embeddings compress sparse, high-dimensional one-hot representations of millions of entities into dense vectors typically ranging from 50 to 500 dimensions. This compression forces the model to learn latent semantic features.

• A knowledge graph with 1 million entities represented as one-hot vectors would require 1M dimensions; embeddings reduce this to ~100-200
• Information bottleneck: The limited dimensionality forces similar entities (e.g., countries, athletes) to cluster together
• Trade-off: Too few dimensions lose expressivity; too many risk memorization and overfitting

Once trained, the cosine similarity between two entity embedding vectors quantifies their semantic relatedness. Entities that share similar attributes or participate in analogous relational structures occupy nearby regions of the vector space.

• Analogical reasoning: king - man + woman ≈ queen — the vector offset captures the gender relation
• Nearest neighbor search: Efficiently retrieves the top-k most similar entities using approximate nearest neighbor (ANN) libraries like FAISS or Annoy
• Downstream use: These similarity scores power recommendation engines, entity resolution pipelines, and candidate generation for entity linking

Advanced embedding models capture complex relational patterns beyond simple translation. Different scoring functions encode different logical properties.

• Symmetry: Models like DistMult handle symmetric relations (e.g., is_sibling_of) via diagonal relation matrices
• Antisymmetry: ComplEx uses complex-valued embeddings to model directed relations (e.g., parent_of)
• Composition: RotatE models relations as rotations in complex space, naturally encoding composition patterns like grandfather_of = father_of ∘ father_of
• Hierarchy: HAKE maps entities into polar coordinates to capture hierarchical structures where entities at different levels have distinct radius values

Modern entity embeddings are no longer learned solely from graph structure. Multi-modal approaches incorporate textual descriptions, images, and attribute values into a unified vector.

• BERT-based entity embeddings: Pre-trained language models encode an entity's textual description, which is then fused with structural embeddings from the knowledge graph
• Temporal embeddings: Models like ATiSE learn time-aware entity representations that evolve, capturing how an entity's properties change over time (e.g., a person's profession)
• Contextualized embeddings: Unlike static embeddings, these produce different vectors for the same entity depending on the surrounding context, crucial for accurate entity disambiguation

Entity embeddings are learned by contrasting positive triples (true facts from the knowledge graph) against negative triples (corrupted, false facts). The model is optimized to score true triples higher than false ones.

• Negative sampling strategies: Randomly replace the head or tail entity; uniform sampling is common but Bernoulli sampling addresses 1-to-N relation imbalances
• Loss functions: Margin-based ranking loss enforces a minimum score gap; cross-entropy with negative log-likelihood treats it as a binary classification problem
• Self-adversarial sampling: Modern approaches like RotatE weight harder negative samples more heavily during training, improving embedding quality

The specific class of machine learning models designed to learn entity embeddings directly from a knowledge graph's structure. These models define a scoring function that measures the plausibility of a triple (head, relation, tail).

• TransE: Interprets relations as translations in the embedding space, so h + r ≈ t.
• DistMult: Uses a bilinear scoring function, capturing symmetric relations effectively.
• ComplEx: Extends DistMult into the complex space to model asymmetric relations.
• RotatE: Models relations as rotations in a complex vector space, capturing symmetry, antisymmetry, and inversion.

The initial retrieval phase in entity linking that uses a surface form dictionary or approximate nearest neighbor (ANN) search to produce a shortlist of possible knowledge base entries for a given mention. Entity embeddings enable this step by allowing the system to compare the contextualized mention vector against pre-indexed entity vectors. Efficient ANN libraries like FAISS or ScaNN are critical here, reducing the search space from millions of entities to a manageable top-k list for downstream disambiguation.

An end-to-end deep learning approach that uses transformer-based architectures to jointly model mention context and entity representations, replacing traditional feature engineering pipelines. These systems generate contextualized entity embeddings for both the textual mention and the candidate entities, then compute a similarity score. This unified approach allows gradients to flow through the entire pipeline, optimizing the entity embeddings directly for the disambiguation task rather than relying on static, pre-trained representations.

A global optimization approach that jointly disambiguates all entity mentions in a document by maximizing the semantic coherence and topical agreement among the linked entities. Rather than linking each mention independently, this method uses the pairwise similarity of candidate entity embeddings to find the most globally consistent configuration. A document mentioning 'Apple,' 'iPhone,' and 'Tim Cook' would have its entity embeddings for 'Apple' pushed toward the technology company based on the high coherence with the other resolved entities.

The task of assigning a very specific hierarchical type label, such as 'American jazz pianist' instead of just 'person,' to an entity mention. This improves disambiguation precision by providing a strong prior for entity embedding similarity. A mention predicted to be a musician will have its candidate embeddings compared more heavily against entities of that type. Type embeddings are often learned jointly with entity embeddings in a shared vector space, allowing type constraints to be applied as a soft filter during linking.