Adversarial Debiasing

The core mechanism involves a min-max game between two networks:

• Predictor: A primary model trained to accurately predict the target variable (e.g., creditworthiness).
• Adversary: A secondary network that attempts to predict the sensitive attribute (e.g., gender) from the predictor's latent representation or output. The predictor is penalized for representations that allow the adversary to succeed, forcing it to learn fair representations that are invariant to the protected group.

A critical architectural component that enables simultaneous training. During backpropagation, the gradients from the adversary are multiplied by a negative scalar (-λ) before flowing into the predictor's shared layers. This gradient reversal ensures that the shared representation is updated to:

• Maximize the predictor's accuracy.
• Minimize the adversary's ability to classify the sensitive attribute. This eliminates the need for iterative, alternating training schedules.

The adversarial weight (λ) controls the trade-off between utility and fairness:

• λ = 0: The adversary is ignored. The model behaves as a standard, unconstrained predictor, maximizing accuracy but potentially encoding bias.
• High λ: Strong pressure to remove demographic information, enforcing a stricter fairness criterion like demographic parity. Tuning λ allows a governance team to select a specific operating point on the fairness-utility trade-off curve, a concept distinct from post-hoc threshold adjustments.

While both aim to learn a debiased latent space, they differ in approach:

• Adversarial Debiasing: An in-processing method where the fairness objective is integrated directly into the model's training loop via the adversarial loss.
• Fair Representation Learning: Typically a pre-processing technique that transforms the input data into a debiased vector space before feeding it to a separate, downstream classifier. The adversarial method allows for end-to-end training with a specific task objective, often yielding higher utility for a given fairness constraint.

Despite its elegance, adversarial training can be notoriously unstable:

• Mode Collapse: The adversary may fail to converge, giving a false sense of fairness.
• Non-Convex Optimization: The min-max game does not guarantee finding a global equilibrium, potentially getting stuck in suboptimal saddle points.
• Proxy Bias: The adversary might learn to predict the sensitive attribute from legitimate, correlated features (e.g., zip code as a proxy for race), leading to an excessive loss of useful information. Careful feature engineering is still required.

The optimization is a two-player minimax game defined as: min_θ max_φ L_P(θ) - λ * L_A(θ, φ) Where:

• L_P is the predictor's loss for the primary task.
• L_A is the adversary's loss for predicting the protected attribute.
• θ are the shared and predictor-specific parameters.
• φ are the adversary-specific parameters. The predictor minimizes its own loss while maximizing the adversary's loss, effectively destroying information about the sensitive attribute.