Master Data Management (MDM)

The Golden Record is the single, best, and most accurate version of a data entity created by merging and cleansing all known records from disparate source systems. It resolves conflicts between conflicting attributes (e.g., different addresses for the same customer) by applying survivorship rules. This authoritative record serves as the canonical reference for all downstream systems, ensuring that every department—from sales to logistics—operates on the same consistent data.

Entity resolution is the computational process of identifying and merging disparate records that refer to the same real-world entity. It employs techniques like:

• Probabilistic matching: Using statistical models to weigh the likelihood of a match based on attribute similarity.
• Deterministic matching: Applying strict, rule-based logic (e.g., exact match on Tax ID).
• Locality-Sensitive Hashing (LSH): An algorithmic technique that hashes similar input items into the same buckets with high probability, enabling efficient approximate nearest neighbor search for deduplication at scale.

MDM requires a formal framework of policies, roles, and responsibilities to maintain data integrity. Data Stewards are designated individuals responsible for resolving match conflicts, approving merges, and curating the master data. This human-in-the-loop component is critical for handling ambiguous edge cases that automated matching algorithms cannot resolve with high confidence. Governance enforces data quality metrics and audit trails for compliance.

Modern MDM platforms manage multiple critical data domains within a single, unified framework rather than operating in silos. Common domains include:

• Customer (C-MDM): Consolidating B2B and B2C profiles.
• Product (P-MDM): Unifying SKUs, parts, and bills of materials.
• Supplier/Vendor: Centralizing procurement and compliance data.
• Location: Standardizing addresses and geospatial hierarchies.
• Asset: Tracking physical and digital equipment across their lifecycle.

MDM can be architected in different ways depending on organizational needs:

• Registry Style: A read-only index that links to records in source systems, providing a lightweight 360-degree view without moving data.
• Consolidation Style: Data is copied from sources into a central hub, cleansed, and matched to generate golden records for reporting and analytics.
• Coexistence Style: A central hub holds the golden record, and updates are synchronized back to source systems, allowing both to coexist.
• Centralized/Transactional Style: All master data is authored, stored, and maintained exclusively in the central MDM hub, which acts as the system of record.

Change Data Capture (CDC) is a design pattern that identifies and tracks row-level changes to source data in real-time, enabling incremental updates to the master data hub. Instead of batch processing, CDC ensures that when a customer updates their phone number in a CRM, the golden record is updated within seconds. This low-latency synchronization is essential for operational MDM use cases like real-time fraud detection or dynamic pricing.