SDMetrics

Quantifies how well synthetic data replicates the statistical properties of the real data. SDMetrics computes metrics across multiple dimensions:

• Column Shapes: Compares univariate distributions using KSComplement and TVComplement tests.
• Column Pair Trends: Measures multivariate correlation preservation using CorrelationSimilarity and ContingencySimilarity.
• Boundary Adherence: Verifies that synthetic data respects the min/max ranges and categorical constraints of the real data.

This ensures the synthetic dataset is not just visually similar but mathematically consistent with the original.

Measures the susceptibility of synthetic data to adversarial attacks that could expose real training records. The library implements diagnostic metrics to detect leakage:

• NewRowSynthesis: Verifies that synthetic rows are novel and not mere copies of real records.
• RangeCoverage: Checks if synthetic data overfits to narrow ranges, potentially exposing individual values.
• Membership Inference Risk: Evaluates the success rate of an attacker trying to distinguish training from holdout records.

These metrics provide a privacy score that helps data custodians manage re-identification risk before sharing data.

Assesses whether synthetic data can replace real data for machine learning tasks without significant performance degradation. SDMetrics supports the Train-Synthetic-Test-Real (TSTR) paradigm:

• Trains a predictive model on synthetic data.
• Evaluates the model's performance on a held-out real test set.
• Compares the results against a model trained on real data (Train-Real-Test-Real).

This provides a direct, task-specific measure of data utility for classification, regression, and other supervised learning problems.

Extends evaluation beyond single tables to entire relational databases with parent-child dependencies. SDMetrics computes fidelity metrics that preserve referential integrity:

• CardinalityShapeSimilarity: Ensures the distribution of child rows per parent is maintained.
• ParentChildRelationships: Verifies that foreign key mappings in synthetic data mirror real-world linkage patterns.
• Inter-Table Trends: Measures correlation preservation across related tables.

This is critical for evaluating synthetic data generated by models like the Synthetic Data Vault (SDV) for complex enterprise schemas.

Generates structured, interpretable JSON and visual reports that aggregate all quality, privacy, and utility metrics into a single diagnostic artifact. Key features include:

• Overall Quality Score: A composite percentage summarizing statistical fidelity.
• Property-Level Breakdowns: Detailed scores for each column, pair, and boundary metric.
• Privacy Risk Flags: Warnings when synthetic data exhibits potential memorization or overfitting.

These reports serve as a Synthetic Data Quality Report for compliance documentation and stakeholder communication.

Provides a modular architecture that allows users to define, register, and execute custom evaluation metrics. The framework is built on composable components:

• Single-Table Metrics: Base classes for column, pair, and boundary-level diagnostics.
• Multi-Table Metrics: Extensions for relational integrity checks.
• Custom Metric API: A plugin system for integrating domain-specific or proprietary evaluation logic.

This extensibility ensures SDMetrics can adapt to novel generative models and specialized industry requirements without modifying the core library.