Semantic Integrity

Semantic integrity enforces hard business rules and domain logic that cannot be violated. For example, in a synthetic e-commerce dataset:

• A ship_date must be on or after the order_date.
• A total_price must equal the sum of item_price * quantity for all line items.
• A patient_age must be a positive integer, and a diagnosis_date cannot precede date_of_birth. Violating these constraints renders data implausible and useless for training or testing logic-dependent applications.

This characteristic ensures consistency across related database tables or entities. Key relationships must be preserved:

• Foreign Key Relationships: A synthetic customer_id in an orders table must exist in the customers table.
• Cardinality: One-to-many and many-to-many relationships (e.g., a doctor has many patients) must be statistically maintained.
• Hierarchical Data: Parent-child relationships in organizational charts or product categories must remain logically connected. Failure here breaks the dataset's relational structure, making it unsuitable for testing complex joins or training models on interconnected data.

Beyond explicit constraints, semantic integrity captures probabilistic correlations and soft dependencies between attributes. A high-fidelity synthetic dataset must replicate these:

• Statistical Correlations: Higher engine_size correlates with lower fuel_economy in vehicle data.
• Categorical Co-occurrences: The job_title "Neurosurgeon" should only co-occur with education_level "MD" or "PhD".
• Temporal Dependencies: In time-series data, a spike in website_traffic often precedes an increase in sales_conversions. Generators must model these multivariate relationships to produce coherent, contextually accurate records.

A single synthetic record must be internally consistent. Contradictions indicate a breakdown in semantic understanding by the generator. Examples include:

• A patient record listing blood_type: 'O-' and a recent transfusion_history showing receipt of 'B+' blood.
• A weather record with temperature: -5°C and precipitation: 'heavy rain' (rain is implausible at that temperature).
• A financial transaction where transaction_type: 'wire' and clearing_time: 'instant' (wires typically take 1-2 days). Detection often requires rule-based validators or consistency classifiers run as part of the validation pipeline.

Semantic integrity requires that every generated sample is plausible within its specific domain, even if it wasn't present in the original dataset. This involves:

• Adhering to Physical Laws: In synthetic sensor data, acceleration cannot change instantaneously.
• Following Regulatory Rules: Synthetic financial transactions must obey Anti-Money Laundering (AML) patterns (e.g., structuring deposits to avoid reporting thresholds is a known pattern).
• Mimicking Behavioral Norms: In synthetic user logs, session duration and click patterns should follow known human-computer interaction models. This goes beyond statistical matching to require a causal understanding of the data-generating process.

The ultimate test of semantic integrity is utility. A synthetically trained model should perform nearly as well on real-world tasks as one trained on real data. Key evaluation protocols include:

• Train-on-Synthetic, Test-on-Real (TSTR): Train a classifier/regressor on synthetic data, evaluate its accuracy on held-out real data. A significant performance drop indicates poor semantic integrity.
• Logic Test Suites: Create unit tests that query the synthetic dataset for known logical relationships (e.g., "all invoices must have a valid customer ID") and assert they hold.
• Domain Expert Review: Human-in-the-loop validation where subject matter experts spot-check records for logical coherence and flag implausible combinations.

Statistical fidelity measures how well the marginal and joint distributions of the synthetic data match those of the real data. It is the foundational layer of validation, ensuring basic statistical properties are preserved.

• Key Metrics: Maximum Mean Discrepancy (MMD), Wasserstein Distance, Kolmogorov-Smirnov test.
• Focus: Univariate statistics (means, variances), correlation matrices, and higher-order moments.
• Limitation: High statistical fidelity does not guarantee semantic integrity, as it may not capture complex logical constraints between attributes.

This is the ultimate, application-driven validation metric. It measures how well a model trained on the synthetic data performs on its intended real-world task, such as classification or regression.

• Protocol: Train-on-Synthetic Test-on-Real (TSTR) is the standard evaluation.
• Interpretation: If a model trained on synthetic data performs nearly as well as one trained on real data, the synthetic data has high utility.
• Relationship to Semantic Integrity: Poor semantic integrity (e.g., broken foreign key relationships) will directly degrade the performance of models trained on that data, as they learn on logically inconsistent examples.

Data plausibility assesses whether individual synthetic records are realistic and believable within the domain context. It is a sample-level check, whereas semantic integrity is a dataset-level property.

• Evaluation: Often performed via manual inspection or using a domain classifier.
• Examples: A generated patient record showing a 2-year-old with a diagnosis of prostate cancer lacks plausibility. A database row where an Order_Date is after the Ship_Date violates temporal plausibility, which is a form of semantic integrity.

Support coverage evaluates the extent to which a synthetic dataset spans the entire range of possibilities (the support) present in the real data distribution. It guards against mode collapse.

• Goal: The synthetic data should generate examples from all regions of the real data distribution, including rare but important edge cases.
• Metric: Precision and Recall for Distributions (P&R) directly measures this; precision measures quality, recall measures coverage.
• Contrast: A dataset can have high semantic integrity (correct relationships) but poor coverage if it only represents a subset of possible entity combinations.

Privacy guarantees are formal assurances that synthetic data protects the confidentiality of individuals in the source dataset. This is often in tension with fidelity and integrity.

• Primary Framework: Differential Privacy (DP), which provides a mathematical bound on privacy loss.
• Audit: A Differential Privacy (DP) audit or Membership Inference Attack test is used to verify these guarantees.
• Trade-off: Introducing strong DP noise to protect privacy can distort statistical relationships and break semantic integrity, such as causing referential integrity violations between tables.

Adversarial validation is a practical technique to detect distributional differences between two datasets. A model is trained to discriminate between real and synthetic samples.

• Interpretation: If a classifier cannot distinguish between the sets (e.g., ~50% accuracy), it suggests the synthetic data distribution is closely aligned with the real one.
• Scope: It provides a strong, holistic signal about statistical fidelity but is a black-box test. It does not specifically diagnose failures in semantic integrity, which may not be easily discernible to a discriminative model focused on feature distributions.