Canonicalization Fidelity

This dimension measures the ability to audit the lineage from every source record to its final canonical entity. High fidelity requires a complete, verifiable mapping.

• Key Mechanism: A provenance graph that links canonical nodes to all contributing source nodes and records the transformation rules applied.
• Failure Mode: A "black box" canonicalization process where the rationale for merging or selecting a canonical form cannot be reconstructed.
• Example: In a customer knowledge graph, traceability allows an auditor to see that the canonical 'John A. Smith' entity was created by merging records from CRM_ID A123 and ERP_ID B456, applying a rule based on matching tax identifiers.

Assesses whether all unique, non-conflicting attributes from source records are retained in the canonical entity. Fidelity is lost when valuable data is discarded during consolidation.

• Evaluation Method: For each canonical entity, compare the union of attributes from all its source records against the attributes present in the canonical form. Calculate a preservation ratio.
• Common Challenge: Handling conflicting values (e.g., different phone numbers). High-fidelity systems implement conflict resolution policies (e.g., 'most recent wins', 'source priority tier') rather than simple omission.
• Impact: Missing attributes degrade downstream tasks like analytics and personalization, as the canonical entity becomes an incomplete representation.

Quantifies the introduction of semantic error during canonicalization. This occurs when the meaning or context of source data is altered, not just merged.

• Causes of Distortion:
  • Over-generalization: Creating a canonical 'Product' entity that loses specific model variants.
  • Context Stripping: Removing source-specific metadata (e.g., 'confidence score: 0.8', 'extracted_from: legal_doc_sec_4') that qualifies a fact.
  • Temporal Flattening: Merging historical records with current ones without preserving valid-time ranges, creating a factually inconsistent timeline.
• Detection: Requires comparing knowledge graph inferences or query results against a gold standard derived from the original, uncanonicalized source context.

Measures the consistency of canonical identifiers as new data arrives. Low stability (frequent ID changes or splits) indicates poor fidelity, as it breaks downstream data integrations.

• Metric: Rate of canonical ID mutation per entity over a defined period. A stable graph has near-zero mutation for mature entities.
• Root Causes of Instability:
  • Weak Merge Keys: Initial canonicalization based on fuzzy matching that is later contradicted by a definitive identifier.
  • Algorithmic Drift: Changes in the underlying entity resolution model's parameters or thresholds.
• High-Fidelity Practice: Use deterministic, rule-based merging for high-confidence attributes (e.g., government IDs) supplemented by probabilistic models for less certain signals, with manual review gates for edge cases.

Evaluates whether the canonicalization logic respects domain-specific context. A single physical entity (e.g., a person) may have multiple valid canonical forms for different business contexts.

• The Problem: Blindly merging all 'Apple' records could incorrectly combine Apple Inc. (the company), apple (the fruit), and Apple Records (the Beatles' label).
• High-Fidelity Solution: Implement scoped canonicalization using contextual dimensions like:
  • Domain: corporate_entities vs. biological_concepts vs. cultural_artifacts.
  • Relationship Role: A person may be canonicalized separately as a customer, an employee, and a vendor if business rules require it.
• Technical Implementation: This often requires a meta-graph or named graphs to manage different canonicalization contexts within the same physical knowledge graph store.

Focuses on the transparency and justification for decisions made when source records disagree. High fidelity means every conflict resolution is logged, attributable, and reversible.

• Core Components:
  • Conflict Log: A structured record for each attribute conflict (e.g., address_line_1), listing the competing values, their sources, and the resolution rule invoked.
  • Rule Catalog: A versioned repository of the business logic used (e.g., 'Rule_23: For customer email, prioritize value from the billing system over marketing newsletter signup').
  • Override Tracking: Manual corrections by data stewards must be logged with a reason code.
• Business Value: This audit trail is critical for regulatory compliance (e.g., GDPR right to explanation) and for diagnosing root causes of data quality issues in enterprise systems.

The foundational process that precedes canonicalization. Entity Resolution (also called Record Linkage or Deduplication) is the task of determining whether different data records refer to the same real-world entity. Canonicalization Fidelity directly depends on the accuracy of this initial resolution step. A high-fidelity canonicalization process preserves the correct attributes from the correctly linked source records.

• Core Dependency: Poor resolution leads to incorrect merges, fundamentally breaking canonicalization.
• Example: Correctly identifying that "J. Smith, Acme Corp" and "John Smith, Acme Inc." are the same person is a prerequisite for creating a high-fidelity canonical "John Smith" entity.

A specialized, critical sub-task of entity resolution focused on establishing a persistent, unique identifier for an entity across all systems and over time. Canonicalization Fidelity is the act of creating and maintaining the authoritative data profile attached to that persistent identity.

• Key Relationship: Identity Resolution provides the stable "key"; Canonicalization provides the authoritative "value."
• Temporal Aspect: High fidelity requires the canonical form to correctly reflect an entity's evolving attributes (e.g., job title, address) while its core identity remains constant.

The property that all stated facts in the knowledge graph are logically non-contradictory. Canonicalization Fidelity is a major contributor to (or detractor from) overall consistency. If canonicalization incorrectly merges attributes from different entities, it creates inherent contradictions.

• Direct Impact: A low-fidelity canonical "Product" entity might incorrectly inherit conflicting attributes (e.g., weight: 5kg and weight: 10kg), violating factual consistency.
• Quality Cascade: High canonicalization fidelity is necessary but not sufficient for overall factual consistency, which also depends on source data accuracy and inference soundness.

The degree to which graph instances adhere to the constraints of the governing ontology. Canonicalization Fidelity must operate within these schema rules. The process of selecting a canonical value for an attribute (e.g., a person's birthDate) must respect data type and property range constraints defined in the schema.

• Governance Layer: The ontology defines what a "valid" canonical form looks like (e.g., xsd:date format).
• Enforcement: High-fidelity canonicalization ensures the output conforms to the schema, preventing quality degradation from invalid data shapes.

The capability to trace the origin and transformations of each fact. For Canonicalization Fidelity, robust provenance is essential for auditability and repair. It answers the question: "Which source records contributed which attributes to this canonical entity?"

• Debugging & Trust: When fidelity is questioned, provenance allows engineers to drill into the source data and resolution logic that created the canonical form.
• Lineage: Documents the canonicalization process itself as a critical transformation in the data pipeline.

A quality assessment method that checks data against predefined logical and semantic rules. This is a primary technique for measuring Canonicalization Fidelity. Rules can be defined to detect canonicalization failures.

• Example Rules:
  • IF entity.type = Person THEN canonical.name MUST NOT be null (Completeness for canonical form).
  • IF entity.canonical.country = "USA" THEN entity.canonical.phone MUST match regex ^\+1 (Attribute consistency).
• Automated Monitoring: These rules can run continuously to detect fidelity drift as new data is canonicalized.