Generative Adversarial Network (GAN)

The core mechanism is a minimax game between two networks. The generator maps random noise to data samples, attempting to minimize its loss. The discriminator acts as a binary classifier, attempting to maximize its accuracy in distinguishing real samples from fakes. The generator's loss is directly tied to the discriminator's success, creating a dynamic where improvement in one network forces adaptation in the other. This is formalized as:

• Generator (G): Learns the data distribution p_g over data x.
• Discriminator (D): Estimates the probability that a sample came from the real data distribution p_data rather than p_g.
• Value Function V(G, D): min_G max_D E_{xp_data}[log D(x)] + E_{zp_z}[log(1 - D(G(z)))]. Training alternates between updating D to better classify real vs. fake, and updating G to produce samples that D classifies as real. Convergence occurs theoretically at a Nash equilibrium where p_g = p_data and D(x) = 1/2 everywhere.

A common training failure where the generator learns to produce only a limited variety of outputs that successfully fool the discriminator, rather than capturing the full diversity of the target distribution. For example, a GAN trained on handwritten digits might learn to generate only the digit '1' repeatedly. This occurs because:

• The generator finds a single mode that the discriminator cannot distinguish from real data.
• The gradient signals from the discriminator push the generator toward this narrow solution.
• The discriminator then overfits to this specific mode, and the generator simply rotates through a small set of outputs. Mitigation strategies include minibatch discrimination, unrolled GANs, and Wasserstein GANs with gradient penalty, which provide smoother gradients that encourage diversity.

A significant architectural improvement that replaces the standard discriminator with a critic and uses the Earth Mover's (Wasserstein-1) distance as the loss metric. Key innovations:

• Critic (f_w): Instead of classifying, the critic scores the 'realness' of samples, providing a meaningful loss signal even when the generator distribution and real distribution do not overlap.
• Gradient Penalty: Enforces a 1-Lipschitz constraint on the critic by penalizing the norm of the gradient with respect to its input, replacing the earlier weight clipping method.
• Stable Training: The Wasserstein distance is continuous and differentiable almost everywhere, eliminating vanishing gradients and providing a loss curve that correlates with sample quality. This architecture dramatically reduces mode collapse and provides a reliable metric for monitoring training progress.

An extension that feeds auxiliary information, such as class labels or text embeddings, into both the generator and discriminator. This conditions the generation process on specific attributes. The architecture:

• Generator Input: Concatenation of noise vector z and condition y.
• Discriminator Input: Concatenation of data sample x and condition y.
• Objective: min_G max_D E_{xp_data}[log D(x|y)] + E_{zp_z}[log(1 - D(G(z|y)))]. Variants include Auxiliary Classifier GANs (AC-GANs), which add an auxiliary classifier to the discriminator that predicts the class label, and text-to-image models that condition on sentence embeddings. This is foundational for controlled synthetic data generation where specific demographic or categorical distributions must be preserved.

Conditional Tabular GAN is a specialized architecture designed for modeling non-Gaussian, multi-modal, and mixed-type tabular data. It addresses the unique challenges of structured data:

• Mode-Specific Normalization: Each column is processed with a variational Gaussian mixture model to handle continuous columns with multiple modes, converting them into a one-hot vector indicating the selected mode and a normalized scalar value.
• Conditional Vector: A one-hot vector specifying a discrete column and its category is concatenated to the noise input, ensuring the generator produces rows that match the specified condition. This is combined with training-by-sampling to balance categorical column distributions.
• Fully Connected Networks: Both generator and discriminator use fully connected layers with batch normalization and leaky ReLU activations. CTGAN is the de facto standard for generating privacy-preserving synthetic tabular datasets for enterprise governance and testing.

Train on Synthetic, Test on Real (TSTR) is the primary utility evaluation framework for GAN-generated data. The methodology:

• A predictive model is trained exclusively on the synthetic dataset.
• The model is then evaluated on a held-out real dataset that was never seen during generation or training.
• The performance metric (e.g., accuracy, F1-score, RMSE) is compared against a baseline model trained on real data (TRTR).
• TSTR/TRTR Ratio: A ratio close to 1.0 indicates high utility; a significant drop indicates the synthetic data failed to capture predictive patterns. Complementary metrics include statistical fidelity (comparing marginal and joint distributions) and discrimination score (training a classifier to distinguish synthetic from real samples—a score near 0.5 indicates indistinguishability).

A catastrophic GAN training failure where the generator learns to produce only a single or very limited variety of outputs that successfully fool the discriminator. Instead of capturing the full diversity of the target data distribution, the generator maps multiple distinct latent points to the same output. This results in a synthetic dataset lacking variance, even if individual samples appear highly realistic. Techniques like minibatch discrimination and Wasserstein loss are common architectural mitigations.

A quantitative measure of how accurately a synthetic dataset preserves the statistical properties of the original real-world data. It goes beyond column averages to evaluate:

• Marginal distributions: The shape of individual variable histograms.
• Joint distributions: The correlations and dependencies between multiple variables.
• Multivariate patterns: The preservation of complex, high-dimensional clusters. High fidelity is critical for ensuring machine learning models trained on synthetic data generalize effectively to real-world production environments.

A privacy attack where an adversary determines whether a specific individual's record was part of a model's training dataset. By analyzing the model's prediction confidence on a target record—exploiting the fact that models often behave differently on seen versus unseen data—attackers can infer membership. For GANs, this risk extends to the synthetic data itself; a discriminator that memorizes training points can leak information, making the generated outputs vulnerable to re-identification.

The gold-standard evaluation paradigm for measuring the utility of synthetic data. A machine learning model is trained exclusively on the generated synthetic dataset and then evaluated on a held-out real test set. If the model's performance on real data is comparable to a model trained on real data, the synthetic data is deemed to have high utility. This metric is often preferred over purely statistical similarity measures because it directly tests the data's fitness for downstream machine learning tasks.

The documented, auditable history of a dataset's origin, transformations, and chain of custody. For GAN-generated data, provenance tracks the entire lineage: the source of the real training data, the specific GAN architecture and hyperparameters used, the random seed, and any post-processing steps. This metadata is essential for AI governance compliance, allowing auditors to verify that synthetic data was generated ethically and legally, and to trace any discovered biases back to their root cause.

A degenerative process where generative models trained recursively on their own synthetic outputs—or on data from other models—progressively lose diversity and forget the tails of the original distribution. Over successive generations, the model's approximation of reality narrows, amplifying artifacts and leading to irreversible errors. This is a critical risk in ecosystems where synthetic data is widely shared, as it can poison the global data pool and degrade the performance of future models trained on it.