Capability Overhang

The defining signature of capability overhang is a discontinuous leap in performance on a task without proportional increases in compute or data. Unlike smooth scaling laws, these jumps occur when a model's latent representations already encode a skill that merely lacks the right output format or prompting interface. Emergent abilities in large language models—such as chain-of-thought reasoning or arithmetic—often manifest this way, appearing abruptly at specific scale thresholds rather than improving gradually.

Standard evaluation benchmarks systematically fail to detect overhang because they test for explicit, surface-level capabilities. A model may possess sophisticated internal world models or reasoning primitives that never surface in benchmark scores. This creates a dangerous false sense of security: safety audits report low capability, while the latent potential for high-risk behaviors—such as deception or long-horizon planning—remains unmeasured and unmitigated.

Latent capabilities are highly sensitive to elicitation methods. A skill that appears absent under zero-shot prompting may emerge fully formed with few-shot examples, chain-of-thought scaffolding, or role-playing personas. This means capability overhang is not a fixed property but a function of the interface. Adversarial users or automated red-teaming tools can systematically probe for these dormant skills, effectively 'jailbreaking' capabilities that safety teams assumed did not exist.

Overhang is dangerously amplified by fine-tuning on narrow task data. A model with latent programming ability may require only a few hundred examples to unlock sophisticated code generation. This creates a dual-use problem: legitimate fine-tuning for benign applications can inadvertently activate adjacent hazardous capabilities. The minimal data requirement makes this vector accessible to actors with limited compute resources.

Capability overhang violates the assumption of predictable scaling laws. While aggregate metrics like perplexity improve smoothly with compute, specific downstream capabilities can remain flatlined for orders of magnitude of scale before spiking. This unpredictability undermines safety forecasting methods that extrapolate from current trends, as the next training run could unlock qualitatively different and potentially dangerous behaviors without warning.

Dormant capabilities can activate through compositional combination of simpler, already-measured skills. A model that demonstrates basic tool use and basic planning separately may suddenly exhibit complex autonomous goal pursuit when these primitives are chained. This emergent composition is particularly difficult to predict because the combinatorial space of skill interactions grows exponentially with model capacity, outpacing evaluation coverage.