Wasserstein GAN (WGAN)

The core innovation of WGANs is replacing Jensen-Shannon divergence with the Wasserstein-1 distance, also known as the Earth Mover's distance. This metric measures the minimal cost of transforming one probability distribution into another. Unlike KL or JS divergence, the Wasserstein distance remains continuous and differentiable even when the real and generated distributions have disjoint supports, providing meaningful gradients throughout training. This eliminates the vanishing gradient problem that plagues standard GANs when the discriminator becomes too successful.

WGANs replace the binary classifier discriminator with a critic network that outputs a real-valued score rather than a probability. The critic estimates the Wasserstein distance by approximating a 1-Lipschitz function. Key distinctions:

• The critic is not trained to classify; it is trained to maximize the difference between scores for real and generated samples
• The generator minimizes this difference, creating a true minimax game
• The critic's loss correlates with sample quality, serving as a meaningful convergence metric

To ensure the critic approximates a valid Wasserstein distance, its function must satisfy the 1-Lipschitz constraint—the gradient norm must be bounded by 1 everywhere. Two primary enforcement methods exist:

• Weight Clipping (Original WGAN): Clamps critic weights to a small range like [-0.01, 0.01] after each update. Simple but can limit model capacity and cause optimization difficulties
• Gradient Penalty (WGAN-GP): Adds a penalty term to the critic loss that directly penalizes gradient norms deviating from 1 at interpolated points between real and generated samples. This is the preferred modern approach

WGANs address several failure modes of standard GANs:

• No mode collapse: The Wasserstein distance penalizes a generator that only produces a limited variety of samples, encouraging full distribution coverage
• No careful balancing: The critic can be trained to optimality before each generator update without causing vanishing gradients, eliminating the delicate scheduling required in standard GANs
• Meaningful loss curves: The critic's loss genuinely reflects generation quality, enabling practitioners to monitor training progress and detect failures without manual sample inspection

WGANs are particularly suited for generating synthetic financial data due to their ability to capture complex, multi-modal distributions:

• Order book generation: WGANs can produce realistic limit order book states that preserve stylized facts like volatility clustering and fat tails
• Signature WGAN (SigCWGAN): Extends the framework using path signatures to better capture the sequential structure of financial time series, generating more realistic long-term trajectories
• Market impact simulation: Adversarial training with WGAN critics produces synthetic market responses that accurately model the price impact of large trades

WGAN with Gradient Penalty (WGAN-GP), introduced by Gulrajani et al. (2017), is the de facto implementation used in practice. It replaces weight clipping with a differentiable penalty:

• Penalizes the squared difference of the gradient norm from 1 at random interpolation points
• Enables the use of batch normalization in the critic, which weight clipping prohibits
• Achieves faster convergence and higher-quality samples across diverse architectures
• The gradient penalty coefficient (typically λ=10) requires minimal tuning across tasks

The Wasserstein distance is the core loss function replacing binary cross-entropy in a WGAN. It measures the minimum 'cost' of transforming one probability distribution into another. Unlike Jensen-Shannon divergence, it provides meaningful gradients even when the real and generated distributions have non-overlapping supports, which is the primary reason for WGAN's training stability. In market simulation, it ensures the generator captures the full support of fat-tailed return distributions.

In a standard GAN, the discriminator is a binary classifier distinguishing real from fake. In a WGAN, it is reframed as a critic that scores the 'realness' of a sample on a continuous scale. The critic is not trained to classify but to estimate the Wasserstein distance. A well-trained critic provides a meaningful loss curve that correlates with sample quality, solving the GAN problem of having no reliable convergence metric.

Mode collapse occurs when a GAN generator produces only a limited variety of samples, failing to capture the full diversity of the target distribution. WGANs are significantly more robust against mode collapse because the Wasserstein distance provides a smooth, continuous gradient signal. The critic can effectively penalize a generator that ignores modes, forcing it to explore the entire support of the real data distribution, which is critical for generating diverse market scenarios.