Schema Mapping

This involves matching elements from a source schema (e.g., a relational database table) to a target graph schema (e.g., vertex and edge types). The process defines how tabular rows become graph nodes, foreign keys become relationships, and columns become node/edge properties. For example, a Customer table maps to a Customer vertex label, and its OrderID foreign key column maps to a PURCHASED edge connecting to an Order vertex.

Schema mapping resolves differences in meaning and naming conventions between systems. This includes:

• Synonym Resolution: Mapping cust_name (source) to customerName (target).
• Taxonomy Alignment: Determining that a source product_category value of "AUTO" corresponds to the target ontology class VehicleParts.
• Unit Conversion: Specifying that a source field in miles must be transformed to kilometers in the target. This ensures the integrated data carries consistent, unambiguous meaning.

Mappings include executable rules for data conversion and enrichment during the migration process. These are often expressed in a declarative mapping language or scripts. Common transformations include:

• Data Type Casting: string to date.
• Value Lookup & Enrichment: Using a reference table to convert a country code into a full country name.
• Concatenation/Splitting: Creating a full name from first_name and last_name fields.
• Conditional Logic: Applying different rules based on source data values.

A critical feature is defining how source record associations translate into graph relationships. This specifies:

• Relationship Direction: Whether an edge is directed from A to B.
• Relationship Type: The label of the edge (e.g., WORKS_FOR, CONTAINS).
• Multiplicity: Whether a source record links to one or many target records, defining one-to-one, one-to-many, or many-to-many relationships in the graph. For instance, one Company record may map to many Employee vertices, connected by EMPLOYS edges.

The mapping process must preserve or translate data integrity constraints from the source to the target environment. This includes:

• Uniqueness Constraints: Ensuring a property like email remains unique across all Person vertices.
• Mandatory Fields: Defining that certain properties cannot be null.
• Referential Integrity: Ensuring that a relationship's target vertex exists.
• Data Type & Range Validation: Enforcing that a age property is a positive integer. These are often enforced using target schema features like uniqueness constraints and property data types.

Enterprise schema mapping is supported by specialized tools that provide:

• Visual Mapping Interfaces: Drag-and-drop canvases to link source and target elements.
• Mapping Document Generation: Auto-creating human- and machine-readable mapping specifications (e.g., in R2RML, RML, or custom JSON/YAML).
• Impact Analysis: Simulating the effects of a mapping change on existing data.
• Integration with ETL/ELT Pipelines: Generating executable code for data integration platforms like Apache NiFi, Spark, or specialized graph ETL tools.

Semi-structured JSON or document data is mapped by flattening nested objects and arrays into distinct, connected nodes.

• Root Object becomes a Root Node: The top-level JSON object (e.g., an invoice) becomes a node with a label like Invoice.
• Nested Objects become Related Nodes: A nested customer object becomes a separate Customer node connected via a HAS_CUSTOMER relationship.
• Arrays become Lists or Separate Nodes: An array of line items can be modeled as a list property or, more powerfully, as individual LineItem nodes connected via INCLUDES relationships, enabling granular querying.
• Dynamic Properties are Preserved: The flexible nature of JSON is captured using dynamic property keys on the mapped nodes.

Sequential event data from logs or message queues is mapped to capture process flows and state changes over time.

• Events become Nodes or Relationships: Each event (e.g., UserLoggedIn, PaymentProcessed) can be a node with an Event label, connected to entity nodes via PERFORMED_BY or AFFECTS relationships. Alternatively, the event type can be the relationship itself (e.g., LOGGED_IN).
• Timestamps are Critical Properties: Event nodes/relationships always include precise timestamp properties.
• Sequences form Paths: A series of events creates a time-ordered path through the graph, enabling queries like "show the user's session flow."
• State Snapshots: Periodic state events can create State nodes connected to entities, allowing point-in-time queries.

Tabular data from spreadsheets or CSV exports is mapped by interpreting columns as entity attributes and using shared values to infer relationships.

• Rows become Nodes: Each row typically becomes a node instance.
• Header Values inform Labels & Properties: Column headers map to property keys. A column like department_name may also imply a node label of Department.
• Denormalized Values create Links: Repeated values across rows (e.g., a manager_email column present on many employee rows) are used to create REPORTS_TO relationships between Employee nodes by joining on the shared email value.
• Composite Keys: Unique identifiers are often constructed from a combination of column values when a single primary key is absent.

Mapping between the RDF (triple-based) and Property Graph models involves translating semantic web constructs into labeled property structures.

• RDF Class (rdf:type) becomes a Node Label: A triple ex:Product1 rdf:type ex:Product maps Product1 to a node with label Product.
• RDF Property becomes a Relationship Type or Node Property: A predicate like ex:manufacturedBy can map to a MANUFACTURED_BY relationship. A datatype property like ex:productName becomes a node property.
• RDF Literal becomes a Property Value: The object of a datatype property triple provides the value.
• Blank Nodes are Inlined or Expanded: Complex blank node structures can be mapped to nested properties or expanded into intermediary subgraph nodes.
• OWL/RDFS Semantics may be Lost: Advanced semantics like owl:inverseOf or rdfs:subClassOf must be explicitly recreated as constraints or inference rules in the property graph.

Multiple disparate source schemas are mapped to a unified, enterprise-defined canonical graph model, decoupling systems from the core knowledge graph.

• Define a Central Target Schema: First, design an authoritative ontology or property graph schema representing the business domain (the canonical model).
• Create Source-Specific Mapping Rules: Each legacy system (CRM, ERP, SCM) has a dedicated mapping defining how its unique fields transform into the canonical types and properties.
• Use Intermediate Nodes for Alignment: Conflicting identifiers for the same entity (e.g., customer IDs) are resolved by mapping both to a single canonical Customer node, often via HAS_SOURCE_ID relationships.
• Enables Incremental Modernization: New systems can be integrated by mapping to the stable canonical model without altering existing mappings, reducing integration cost and complexity over time.

The overarching process of combining data from disparate sources to provide a unified, coherent view. Schema mapping is a core technical sub-task within integration, specifically focused on aligning structural representations.

• ETL/ELT Pipelines: Mapping rules are often executed within Extract, Transform, Load (or Extract, Load, Transform) workflows.
• Semantic Heterogeneity: A primary challenge integration solves, where the same concept is represented differently across systems (e.g., cust_id vs. CustomerID).

A specialized form of schema mapping focused on semantic models (ontologies). It establishes correspondences between concepts, properties, and relationships in different ontologies, often using logical equivalences (e.g., owl:equivalentClass).

• Purpose: Enables interoperability between knowledge graphs using different foundational ontologies.
• Automation: Often employs machine learning and logical reasoning to suggest mappings based on lexical and structural similarity.

The application of functions or rules to convert data from a source format or structure to a target format. Schema mapping defines the what (which elements correspond), while transformation defines the how (the specific conversion logic).

• Common Operations: Include data type casting, string manipulation (concatenation, splitting), mathematical calculations, and conditional logic.
• Tools: Executed via scripting (Python, SQL) or visual tools within data integration platforms.

The largely automated discovery phase that precedes formal mapping. Algorithms analyze two schemas to propose candidate correspondences between their elements based on names, data types, structures, and instance data.

• Techniques: Include linguistic matching (similarity of names), constraint-based matching (comparing data types), and instance-based matching (analyzing overlapping data values).
• Human-in-the-Loop: Output is a set of suggestions that a domain expert reviews and refines into a definitive mapping.

The tracking of data's origin, movement, transformation, and dependencies across systems. Schema mapping rules are a critical component of lineage metadata, documenting how a target field was populated from source systems.

• Impact Analysis: Essential for understanding how a change in a source schema will affect downstream consumers.
• Compliance: Required for regulatory frameworks that mandate transparency in data provenance.

A canonical, unified schema that serves as an intermediary target for multiple source systems. Instead of creating point-to-point mappings between every pair of schemas (N² complexity), each source maps to the single mediated schema.

• Architecture: Core to data fabric and semantic data integration approaches.
• Advantage: Dramatically reduces the number of mappings needed and provides a consistent enterprise data view.