Data Fabric

The core engine of a data fabric is its active metadata layer. Unlike passive metadata catalogs, this layer continuously collects, analyzes, and operationalizes metadata about data assets, usage patterns, lineage, and quality. It uses this intelligence to automate integration tasks, recommend data products, and enforce governance policies. For example, it can automatically suggest schema mappings between a new source and the fabric's semantic model.

A data fabric employs a semantic layer—often implemented as a knowledge graph—to provide a business-conceptual view of data. This layer maps disparate technical schemas to a unified ontology, defining entities (e.g., 'Customer', 'Product'), their attributes, and relationships. This abstraction allows users to query data using business terms ("show me high-value customers") rather than complex joins across database tables, enabling true self-service analytics.

A key tenet is providing integrated access to data without mandatory physical consolidation. Through logical data virtualization and query federation, the fabric creates a virtualized data layer. A single query can be decomposed, routed to the appropriate source systems (e.g., cloud data warehouse, operational database, data lake), and results aggregated in real-time. This reduces data redundancy and latency while presenting a unified view.

The fabric automates the discovery, preparation, integration, and delivery of data. It uses the active metadata to intelligently orchestrate semantic pipelines. These pipelines handle tasks like:

• Entity resolution: Linking records that refer to the same real-world object.
• Schema alignment: Automatically mapping fields to the central ontology.
• Data quality enforcement: Applying rules and checks during ingestion. This reduces manual engineering overhead and accelerates time-to-insight.

A modern data fabric architecture often embraces data mesh principles by treating data as a product. It provides the underlying platform capabilities for domain teams to build, publish, and manage data products. The fabric ensures these products are discoverable via the semantic catalog, addressable via APIs, interoperable through shared ontologies, and trustworthy with clear lineage and quality metrics, all while maintaining decentralized ownership.

Governance is not a separate process but is woven into the fabric's operations. Policy-based controls are defined semantically (e.g., "PII data from the EU region") and enforced automatically across all access points. This includes:

• Attribute-based access control (ABAC): Dynamic authorization based on user, data, and context attributes.
• Provenance tracking: Full lineage from source to consumption.
• Compliance automation: Applying data residency and retention rules. This ensures security and compliance are inherent, not afterthoughts.

A data integration technique that provides a unified, abstracted, and real-time view of data from multiple disparate sources without requiring physical movement or replication. It sits at the core of many logical data fabric implementations. A virtualization layer:

• Executes federated queries across source systems
• Presents a single virtual schema to consuming applications
• Minimizes data latency and storage costs
• Often relies on query optimization and caching for performance

An abstraction layer that sits between raw data sources and consuming applications (like BI tools). It provides a business-friendly conceptual model of data—using ontologies, taxonomies, and business logic—to enable consistent interpretation, calculation, and querying. In a data fabric, the semantic layer is often instantiated as a knowledge graph that defines the meaning and relationships of enterprise data entities.

A specific type of data fabric architecture that emphasizes virtualized data integration. It provides a logically unified view of data across sources without physically moving or replicating the underlying data, relying instead on semantic models and query federation. This approach prioritizes:

• Real-time data access
• Reduced data redundancy
• Centralized governance over a distributed landscape
• Use of mapping definitions (like R2RML/RML) to create virtual graphs

A comprehensive discipline for defining, managing, and governing an organization's critical shared data entities (e.g., Customer, Product, Supplier) to provide a single, consistent point of reference. MDM creates golden records and is often a foundational source for a data fabric, which then distributes and contextualizes this mastered data across the broader enterprise data landscape. MDM focuses on authoritative versioning, while a fabric focuses on unified access.

A centralized inventory of an organization's data assets, enhanced with metadata, search, and governance tools to enable data discovery, understanding, and trust. A modern active metadata graph is the engine for a data fabric, making the catalog its brain. The fabric uses this graph to:

• Automate data discovery and recommendation
• Enforce data governance and quality policies
• Document data lineage and provenance
• Power semantic search across all sources