Model Autophagy

The fundamental mechanism of autophagy is a self-consuming loop, a closed feedback cycle where a model trains on data generated by previous versions of itself. Unlike general model collapse from any synthetic data, autophagy specifically involves a model ingesting its own outputs. This creates a recursive amplification of subtle statistical artifacts, errors, and biases. Each generation becomes a degraded copy of the previous one, like a photocopy of a photocopy, causing a rapid decay in output quality and a catastrophic loss of diversity as the model's generative distribution narrows around a few high-probability modes.

A defining characteristic of autophagy is tail erosion, the progressive disappearance of rare, fringe, or minority data points from the model's learned distribution. Human-originated data contains a long tail of edge cases, unique expressions, and low-frequency events. When a model trains on its own synthetic outputs, which are statistically concentrated on the most probable outcomes, it fails to represent these tails. Over successive generations, the model 'forgets' these edge cases entirely, leading to a homogenized output that cannot handle novel or unusual inputs and amplifies societal biases by erasing minority representations.

Autophagy causes artifact amplification, where subtle, often imperceptible defects in a model's initial outputs are magnified into dominant, glaring errors over successive training loops. These artifacts can be visual (blurring, strange textures in images), textual (repetitive phrasing, logical inconsistencies), or structural (loss of coherence). Because the model treats its own previous outputs as ground truth, it reinforces these mistakes, creating a runaway feedback loop. A minor statistical glitch in generation one becomes a defining feature of the model's output by generation five, rendering the system unusable.

A direct consequence of autophagy is diversity collapse, where the variance of the model's generative outputs plummets. A healthy model can produce a wide range of valid responses to a prompt. An autophagic model, however, converges on a single, narrow mode of response. This is measured by a sharp decrease in metrics like recall and an increase in perplexity on real-world data. The model loses its ability to generate creative, varied, or surprising content, instead producing repetitive, generic, and ultimately low-value outputs that fail to capture the richness of the original human data distribution.

The damage caused by autophagy is irreversible without external intervention. Once diversity has collapsed and tails have eroded, the lost information about the true data distribution cannot be recovered from the model's weights. The model has not just forgotten facts; it has lost the statistical capacity to represent them. Recovery requires retraining on fresh, verified human-originated data. This distinguishes autophagy from temporary issues like concept drift, as the model's fundamental ability to represent variance is permanently damaged, making continuous filtering and provenance tracking critical for long-term model health.

To prevent autophagy, synthetic data must be filtered out before training. Key detection methods include:

• Perplexity Filtering: AI-generated text tends to have lower perplexity (is more 'predictable') to a language model than human text. A threshold can flag and remove overly predictable, likely synthetic content.
• Burstiness Scoring: Human writing shows high variance in sentence structure and length (it is 'bursty'), while AI text is more uniform. Low burstiness is a strong signal of synthetic origin.
• AI Watermarking: Techniques like Google DeepMind's SynthID embed a cryptographic signal into generated content, enabling definitive identification and exclusion from future training corpora.