Bias Amplification Loop

The student model does not learn the real-world distribution; it learns a smoothed, biased approximation of it. The teacher model's outputs reflect its own internal prejudices and calibration errors. When these outputs become the training data, the student inherits these artifacts as ground truth, causing a loss of distributional fidelity in the very first iteration.

Each recursive training generation acts as a non-linear amplifier. A minor skew in the original model (e.g., a 2% gender association) is not simply copied; it is squared. By the third or fourth generation of synthetic training, the model collapses into a caricatured mode, associating specific demographics almost exclusively with specific roles, obliterating the nuanced variance of the original data.

The loop's most destructive feature is the rapid disappearance of minority and edge-case representations. Teacher models often assign low probability to rare events. When the student trains on this pruned distribution, those rare events vanish entirely. This results in a model that performs reasonably on common demographics but fails catastrophically on protected groups, entrenching representational harm.

A dangerous property of the loop is that perceptual metrics often improve while diversity collapses. The model becomes hyper-confident in generating a narrow set of high-probability outputs. To a human evaluator, the images may look 'sharper' or text 'more fluent,' masking the fact that the model has lost the ability to generate diverse, out-of-distribution samples.

The loop drives the model toward a low-entropy attractor state. The generative distribution loses variance, producing repetitive and homogeneous outputs. In language models, this manifests as semantic and syntactic repetition; in image models, it leads to identical background textures and facial structures, a phenomenon directly linked to the Model Autophagy disorder.

The primary defense is strict data lineage tracking. Training pipelines must cryptographically distinguish human-originated data from synthetic data. Techniques like SynthID watermarking and statistical perplexity filtering are essential to sanitize the corpus before training, ensuring the model does not consume its own biased outputs.

A degenerative process where generative models trained on recursively generated synthetic data progressively lose the ability to represent the tails of the original data distribution. This results in irreversible defects in quality and diversity, as the model forgets rare events and converges toward a narrow, mean representation of reality.

The specific loss of representation for rare, fringe, or minority data points in a synthetic dataset. As the Bias Amplification Loop progresses, the model systematically forgets edge cases and minority representations, causing the output distribution to collapse toward the mode and magnifying societal harms against underrepresented groups.

A specific mode of model collapse where a generative system consumes its own synthetic outputs as training data. This self-cannibalizing loop causes a rapid loss of information diversity, as the model amplifies its own idiosyncratic errors and statistical artifacts without any external corrective signal from human-originated data.

A feedback cycle where a model trains on data generated by previous versions of itself or similar models. Each iteration amplifies subtle artifacts and biases present in the teacher model, causing the rapid decay of output fidelity and the reinforcement of stereotypical or hallucinated patterns.

The phenomenon where the statistical properties of the target production data diverge from the training data distribution. In the context of bias amplification, the synthetic training distribution drifts further from real-world demographics with each iteration, causing severe degradation in model accuracy for underrepresented populations.

The automated process of detecting and excluding machine-generated content from a training corpus. Techniques include:

• Perplexity filtering: Rejecting text that is too statistically predictable
• Burstiness scoring: Measuring variance in sentence structure
• AI watermarking: Detecting embedded cryptographic signals like SynthID