Knowledge Vault

The core mechanism that distinguishes a Knowledge Vault from a deterministic database. Instead of binary true/false assertions, every extracted fact is assigned a calibrated confidence score derived from the agreement between multiple extractors and the prior probability of the fact's predicate.

• Mechanism: Uses a latent-variable model to infer the probability that a fact is true, even when extractors disagree.
• Extractor Fusion: Combines signals from text-based extractors (REVERB, OLLIE), HTML DOM extractors, and human-curated structured data.
• Key Metric: A fact is only promoted to the Knowledge Graph when its posterior probability exceeds a high-precision threshold.

A critical component for calibrating confidence. The system learns the base rate at which a specific predicate (relationship type) is true in the world, independent of any extraction.

• Predicate Priors: Learns that predicates like place_of_birth have a high prior (most people have one birthplace), while causes_disease has a low prior.
• Noise Filtering: Low-prior predicates require significantly more extractor agreement to overcome the initial skepticism.
• Implementation: Computed by counting the fraction of entity pairs in a reference set that satisfy the predicate, providing a Bayesian anchor for all subsequent reasoning.

The Knowledge Vault ingests facts from a diverse ensemble of extraction systems, each with a distinct error profile. Fusion across these heterogeneous sources is the primary driver of precision gains.

• Text Extractors: REVERB and OLLIE for open information extraction from web text.
• DOM Extractors: Parsers that extract structured data embedded in HTML tables and lists.
• Existing KBs: Freebase served as the initial seed of high-confidence facts for training the prior model.
• Synergy: A fact extracted by both a noisy text system and a precise DOM parser receives a dramatically higher confidence score than either source alone.

A supervised machine learning component that determines whether a textual phrase genuinely expresses a specific target relation, moving beyond simple pattern matching.

• Input: A sentence, a candidate entity pair, and a target Freebase relation.
• Features: Lexical patterns, dependency parse paths, entity types, and WordNet clusters.
• Function: Acts as a high-precision filter, rejecting false-positive extractions where the surface text is misleading (e.g., distinguishing 'founded' an organization from 'founded' a physical structure).
• Training Data: Generated by distant supervision, aligning Freebase facts with sentences that mention the corresponding entities.

The computational backbone enabling probabilistic reasoning over web-scale data. The system must compute confidence scores for hundreds of millions of candidate facts without requiring intractable joint inference.

• Factor Graph: Models the dependencies between fact truth values, extractor reliability, and predicate priors.
• Parallelization: Inference is decomposed into independent sub-problems per predicate, allowing massive horizontal scaling across a compute cluster.
• Loop-Belief Propagation: An approximate inference algorithm used to efficiently estimate marginal probabilities in the factor graph without computing the full joint distribution.

Unlike a static snapshot, the Knowledge Vault is designed for incremental ingestion and recalibration. As new web pages are crawled and new extractors are developed, the system recomputes confidence scores.

• Temporal Awareness: Tracks the provenance timestamp of each extraction to handle facts that change over time (e.g., CEO transitions).
• Feedback Loop: High-confidence facts output by the Vault can be fed back as training data for extractors, creating a virtuous cycle of improvement.
• Conflict Resolution: When a new extraction contradicts a previously high-confidence fact, the system flags the conflict for re-evaluation rather than silently overwriting.

The NLP task of identifying textual mentions of entities and mapping them to their unique, unambiguous entries in a target knowledge base like Wikidata or DBpedia. This is the foundational step for transforming unstructured text into the structured triples that populate a Knowledge Vault.

• Resolves ambiguous mentions (e.g., 'Paris' the city vs. the mythological figure)
• Creates the critical bridge between raw text and machine-readable knowledge graphs

A machine learning task that predicts missing links or facts by inferring new relationships from existing graph structure and entity embeddings. This directly mirrors the Knowledge Vault's fusion process, where probabilistic models fill gaps in human-curated knowledge bases.

• Uses tensor factorization or graph neural networks
• Assigns a confidence score to each newly predicted triple

The computational process of resolving disparate data records to determine if they refer to the same real-world object, often using probabilistic matching against a canonical knowledge base like Wikidata. A Knowledge Vault relies on this to deduplicate facts extracted from millions of web pages.

• Uses fuzzy string matching and graph proximity
• Outputs a confidence score for each match candidate

The automated process of assessing the truthfulness of a textual claim by corroborating it against a trusted knowledge base or evidence corpus. This is the quality-control mechanism that assigns the confidence score to every probabilistic assertion in a Knowledge Vault.

• Often uses the ClaimReview schema for search engine display
• Prevents hallucinated facts from entering the graph

A purpose-built database for storing and retrieving semantic data in the form of subject-predicate-object triples, the foundational structure of the Resource Description Framework (RDF). A Knowledge Vault's extracted facts are ultimately serialized and stored in such a system for querying via SPARQL.

• Optimized for complex graph traversals
• Examples include Amazon Neptune and Ontotext GraphDB

An OWL property (owl:sameAs) used in RDF to explicitly state that two different URIs refer to the identical real-world entity. This is a critical mechanism for cross-source entity identity resolution, allowing a Knowledge Vault to fuse data from Wikidata, DBpedia, and proprietary sources without creating duplicate nodes.

• Consolidates entity identity across the linked data cloud
• Prevents knowledge graph fragmentation