Factual Consistency

This method uses the formal rules defined in the knowledge graph's ontology (e.g., OWL, SHACL) to automatically detect logical contradictions. It checks for violations of constraints such as:

• Class disjointness: An entity cannot be an instance of two mutually exclusive classes (e.g., Person and Location).
• Property domain and range: A relationship must connect entities of the correct types.
• Cardinality restrictions: Ensures the number of relationships for an entity adheres to defined limits (e.g., a person has exactly one biological mother). Tools like OWL reasoners (e.g., HermiT, Pellet) and SHACL validators execute these checks to identify inconsistent subgraphs.

This technique applies domain-specific business rules or commonsense axioms to surface inconsistencies that may not be captured by the formal ontology. Rules are often expressed as SPARQL queries or in a custom rule language to find contradictory patterns. Examples include:

• Temporal contradictions: An event's endDate cannot be before its startDate.
• Functional dependency violations: If hasCapital is a functional property, a country cannot have two different capital cities.
• Synonym ring clashes: An entity labeled as deceased should not have a recent employment event. This method is crucial for catching semantic inconsistencies that require domain knowledge.

This assessment compares the knowledge graph's facts against a trusted reference dataset or gold standard. Metrics are calculated to quantify consistency:

• Precision: The proportion of graph facts that are verified by the gold standard. Low precision indicates hallucinated or incorrect facts.
• Recall: The proportion of gold standard facts that are found in the graph.
• F1 Score: The harmonic mean of precision and recall. The gold standard is often a manually curated subset, a high-authority external knowledge base (e.g., Wikidata for general facts, proprietary databases for enterprises), or a consensus derived from multiple reliable sources.

This machine learning approach uses knowledge graph embeddings (e.g., TransE, ComplEx) to identify factual outliers. Entities and relations are mapped to a vector space where their geometric relationships encode semantic meaning. Inconsistencies are detected by:

• High reconstruction error: A triple (subject, relation, object) that receives a very low score by the embedding model is likely anomalous.
• Cluster analysis: Entities of the same type should form tight clusters in the vector space; outliers may indicate misclassification or erroneous facts.
• Link prediction confidence: Using the model to predict missing links can reveal if existing links are improbable given the overall graph structure.

This method assesses consistency by triangulating facts across multiple independent data sources ingested into or referenced by the knowledge graph. Confidence scores are often assigned based on source agreement. The process involves:

• Source provenance tracking: Tagging each fact with its origin.
• Conflict identification: Flagging facts where different authoritative sources provide contradictory values for the same entity attribute or relationship.
• Consensus calculation: Applying rules (e.g., majority vote, weighted by source authority) to determine the canonical, consistent fact. This is fundamental for knowledge graphs built from heterogeneous enterprise data silos.

This method evaluates whether the inferred facts generated by a semantic reasoner are factually consistent with the explicit knowledge and the rules applied. It tests the soundness of the reasoning process. Testing involves:

• Creating a test suite of entailment regimes: Specific sets of facts and rules where the correct inferences are known.
• Running the reasoner and checking if all generated inferences are correct and no unsupported or contradictory inferences are produced.
• This validates that the formal semantics of the ontology (e.g., OWL 2 DL) are correctly implemented and do not introduce logical inconsistencies.

Logical Consistency is a formal property ensuring no set of facts in a knowledge graph violates the logical constraints of its governing ontology. This is a prerequisite for Factual Consistency.

• Key Mechanism: Enforces rules like class disjointness (Person and Organization are separate), property domains/ranges, and cardinality restrictions (e.g., a Person has exactly one birthDate).
• Tool: Validated by a semantic reasoner (e.g., Pellet, HermiT) that performs axiom checking.
• Example: A graph stating an entity is both a City and a Person while the ontology defines them as disjoint classes is logically inconsistent.

Entity Accuracy measures the proportion of graph nodes that correctly correspond to their real-world referents. It is a core component of factual correctness.

• Assessment Method: Compared against a gold standard dataset curated by domain experts.
• Common Errors: Misidentified entities (e.g., confusing two people with the same name), misclassified types (labeling a LLC as a Corporation), or incorrect attribute values.
• Impact: Low entity accuracy directly undermines Factual Consistency, as errors propagate through relationships.

Link Validity evaluates the semantic and factual correctness of the relationships (edges/predicates) between entities. It directly contributes to the overall factual consistency of the graph.

• Focus: Verifies that each triple (subject, predicate, object) represents a true statement. For example, (CompanyA, acquired, CompanyB) must be factually true on the acquisition date.
• Validation Techniques: Uses rule-based validation (e.g., acquired only links Organization to Organization) and cross-referencing with authoritative sources.
• Failure Example: A link stating (Eiffel Tower, locatedIn, Rome) is an invalid link, creating a factual inconsistency.

Constraint Satisfaction is the process of ensuring all data instances comply with the predefined logical, semantic, and data-type constraints of the knowledge graph's schema (ontology).

• Constraint Types:
  • Datatype Constraints: employeeCount must be an xsd:integer.
  • Value Constraints: status must be in ["Active", "Inactive", "Pending"].
  • Relational Constraints: manages property requires the subject to be of type Manager.
• Role: Acts as a foundational data quality gatekeeper. A graph that fails constraint satisfaction cannot be factually consistent.

Rule-Based Validation is a deterministic quality assessment method that checks knowledge graph data against a set of predefined logical, syntactic, or semantic rules to identify violations.

• Implementation: Uses rule engines or SPARQL queries with FILTER NOT EXISTS to detect anomalies.
• Example Rules:
  • IF (?person a :DeceasedPerson) THEN NOT EXISTS (?person :hasEmployment ?job).
  • (:headquartersIn) must have a (:Country) as its object.
• Connection to Factual Consistency: Provides automated, scalable checks for common inconsistency patterns before more complex factual verification.

A Gold Standard is a curated, high-quality reference dataset used as the benchmark for evaluating the accuracy, completeness, and Factual Consistency of a knowledge graph.

• Creation: Built by domain experts through manual annotation or derived from highly trusted sources.
• Primary Uses:
  • Calculating metrics like Precision@K and Recall@K for query or link prediction tasks.
  • Serving as ground truth for measuring Entity Accuracy and Link Validity.
• Critical Role: Without a gold standard, assessing factual consistency becomes subjective and non-reproducible.