Generator Network

The generator's input is a latent vector (z) sampled from a prior distribution, typically a multivariate Gaussian. This low-dimensional latent space acts as a compressed, continuous representation where each point can be decoded into a unique output. The network learns a mapping from this simple distribution to the complex, high-dimensional distribution of real data.

• Purpose: Provides a source of randomness and controllability.
• Dimensionality: A trade-off; too small limits expressiveness, too large can cause training instability.
• Example: In a face generation task, different regions of the latent space might correspond to attributes like smile, hair color, or pose.

The core of the generator is a series of transposed convolutional layers (sometimes called deconvolutions) or upsampling layers followed by standard convolutions. These layers progressively increase the spatial resolution of the feature maps from the small latent vector to the final output dimensions (e.g., 64x64, 256x256 pixels).

• Function: Transform the 1D latent vector into a 2D or 3D synthetic sample.
• Architecture Pattern: Often follows a progressive growing or multi-scale design, as seen in StyleGAN, where low-resolution features are established first and then refined.
• Activation: Final layer typically uses a tanh (for pixel values normalized to [-1, 1]) or sigmoid (for [0, 1]) activation.

Batch normalization layers are commonly inserted between convolutional layers in the generator. They stabilize training by normalizing the activations of a mini-batch to have zero mean and unit variance.

• Impact on Training: Helps mitigate the internal covariate shift problem, allowing for higher learning rates and reducing sensitivity to initialization.
• GAN-Specific Caveat: In some advanced architectures like StyleGAN, batch normalization is replaced by weight demodulation or instance normalization to avoid artifacts caused by correlated samples within a batch.

In Conditional GANs (cGANs), the generator receives additional information (a condition y) alongside the latent noise. This allows controlled generation of data with specific attributes, such as generating an image of a particular class.

• Common Techniques:
  • Concatenation: The condition vector is concatenated with the latent vector at the input layer.
  • Projection: The condition is projected and used to modulate intermediate feature maps via conditional batch normalization or adaptive instance normalization (AdaIN).
• Use Case: Generating a synthetic chest X-ray image conditioned on a specific disease label.

Modern generator architectures frequently employ residual blocks. These blocks use skip connections that add the input of the block to its output, mitigating the vanishing gradient problem and enabling the training of much deeper networks.

• Benefit: Facilitates the flow of gradients during backpropagation, allowing the network to learn identity functions more easily and stabilize training.
• Structure: A typical residual block contains two or three convolutional layers, with batch normalization and ReLU activations, and a shortcut connection.
• Example: Used extensively in StyleGAN2 and BigGAN to generate high-fidelity, high-resolution images.

The StyleGAN architecture revolutionized generator design by decoupling the latent space from the image synthesis process. It introduces two key sub-networks:

• Mapping Network: An 8-layer MLP that transforms the latent vector z into an intermediate latent code w. This disentangles the input representation.
• Synthesis Network: Starts from a learned constant and uses a series of convolutional layers. Each layer's style is controlled by injecting the w vector via Adaptive Instance Normalization (AdaIN), allowing precise control over attributes like coarse features (pose, face shape) at early layers and fine details (hair color, micro-features) at later layers.

The discriminator network is the adversarial counterpart to the generator. It is a neural network classifier trained to distinguish between real data samples from the training set and synthetic samples produced by the generator. Its feedback, in the form of a classification loss, is the primary signal used to train the generator. A strong discriminator forces the generator to improve, creating the competitive dynamic central to GAN training.

The latent space (or z-space) is a lower-dimensional, continuous vector distribution (typically Gaussian) from which the generator network draws its random noise input. This space represents a compressed, abstract representation of the data manifold. The generator's core function is to learn a mapping from this simple distribution to the complex, high-dimensional distribution of the real data. Exploration and manipulation within this space control the attributes of the generated output.

Adversarial loss is the objective function that formalizes the competition between the generator and discriminator as a minimax game. The generator aims to minimize this loss (by fooling the discriminator), while the discriminator aims to maximize it (by correctly identifying fakes). Common variants include:

• The original minimax loss.
• The non-saturating loss, a practical heuristic to prevent generator gradient saturation.
• The Wasserstein loss, used in WGANs for improved stability.

Mode collapse is a fundamental training failure in GANs where the generator network learns to produce a very limited variety of outputs, effectively collapsing to produce only one or a few plausible samples. This occurs when the generator finds a single output that reliably fools the discriminator and ceases to explore the full data distribution. It is a key challenge that highlights the instability of the adversarial training dynamic and the generator's tendency to exploit weaknesses in the discriminator.

Introduced in StyleGAN, the mapping network is an intermediate neural network that transforms the input latent vector (z) into an intermediate latent space (W). This learned transformation helps disentangle features, meaning different dimensions in W-space control distinct, semantic attributes of the generated image (e.g., pose, hairstyle, lighting). The generator's synthesis network then uses vectors from this W-space to modulate its layers via Adaptive Instance Normalization (AdaIN), enabling unprecedented control over image synthesis.

In a Wasserstein GAN (WGAN), the critic network replaces the standard discriminator. Instead of outputting a probability, it outputs an unbounded scalar score. Its objective is to learn a 1-Lipschitz continuous function that estimates the Wasserstein distance (Earth Mover's Distance) between the real and generated data distributions. This formulation provides more stable gradients for the generator, as the critic's feedback is more meaningful even when the distributions are disjoint, leading to improved training stability.