Hallucination Rate Spike

A sharp, non-linear drop in factual precision measured against a ground-truth knowledge base. Unlike gradual model degradation, this presents as an immediate cliff in metrics like Factual Consistency Score or Knowledge F1.

• Threshold: Often defined as a >20% increase in factual errors within a single evaluation window.
• Contrast: Distinct from concept drift, which is typically gradual.
• Example: A model suddenly claiming a CEO resigned when they did not, with error rates jumping from 2% to 45% in one hour.

The model generates factually incorrect outputs with anomalously high token-level log probabilities. This indicates a failure in the model's internal calibration, not just a retrieval error.

• Mechanism: The model's confidence calibration has drifted, causing it to be 'delusionally certain' about wrong answers.
• Detection: Monitor the divergence between predicted probability and actual accuracy.
• Risk: These are the most dangerous hallucinations because they bypass standard low-confidence filtering guardrails.

The model ignores or contradicts the provided grounding context (e.g., a RAG pipeline's retrieved documents). The output is fluent but not attributable to the source material.

• Metric: A spike in Context Relevance violations or Faithfulness scores dropping below threshold.
• Cause: Often linked to context window poisoning or a failure in the attention mechanism to prioritize the prompt over parametric knowledge.
• Symptom: The model invents details not present in the user-provided text.

A specific subtype of unfaithfulness where the model fabricates non-existent citations, URLs, or academic references. This is a critical failure in Retrieval-Augmented Generation Architectures.

• Pattern: Generation of plausible but fake DOIs, legal case numbers, or API documentation links.
• Root Cause: Often triggered by a distributional shift in the types of queries requesting citations.
• Mitigation: Requires strict agent output validation to verify link resolvability before user exposure.

The model's chain-of-thought reasoning breaks down, resulting in outputs that are internally contradictory or nonsensical despite grammatically correct sentences.

• Indicator: A measurable Chain-of-Thought Coherence Drop where step 3 contradicts step 1.
• Nature: This is a reasoning failure, distinct from a factual error. The logic itself is invalid.
• Example: A math solver stating 'x = 5' in one sentence and 'since x is 12' in the next.

The model confuses the chronology of events, merging past and present facts. It might claim a future event has already happened or use outdated information with high certainty.

• Mechanism: A failure in the model's temporal grounding, often caused by stale training data conflicting with real-time retrieval.
• Detection: Track the rate of temporal entailment errors.
• Impact: Critically damaging in news summarization, financial reporting, and legal analysis.

The most common trigger for a hallucination spike. When the statistical properties of production input data diverge from the training distribution, the model encounters unfamiliar patterns it cannot ground in its learned knowledge. Covariate shift (change in input features) and concept drift (change in the relationship between inputs and outputs) both force the model to extrapolate rather than retrieve, dramatically increasing confabulation rates. Monitoring the KL divergence between training and inference data distributions is a leading indicator.

An adversarial or accidental corruption of the agent's working memory that causes downstream factual errors. When an attacker injects misleading documents into a RAG pipeline, or when the agent accumulates contradictory statements over a long conversation, the model's attention mechanism latches onto the poisoned context. This produces high-confidence but factually incorrect outputs. Unlike a simple prompt injection, the hallucination may persist across multiple turns as the corrupted context remains in the KV cache.

A degradation in the model's ability to accurately estimate its own uncertainty. A well-calibrated model assigns low confidence to incorrect answers, enabling downstream filters to catch hallucinations. When calibration drifts, the model becomes overconfident on false statements, producing hallucinations with high softmax probabilities that bypass standard threshold-based guardrails. Measuring Expected Calibration Error (ECE) over time is essential for detecting this silent failure mode.

A measurable decline in the logical consistency of the model's step-by-step reasoning. Even when the final answer appears plausible, the intermediate reasoning steps may contain logical leaps, circular arguments, or fabricated premises. This is a leading indicator of an impending hallucination spike, as the model begins constructing answers on faulty foundations. Automated evaluation using process reward models (PRMs) can detect coherence degradation before it manifests in final outputs.

A hallucination spike often originates in the retrieval layer, not the generation layer. When the retriever returns irrelevant, outdated, or semantically misaligned documents, the generator is forced to either ignore its grounding context or synthesize a response from noise. Common failure modes include: - Embedding drift where the query and document vectors diverge - Chunk boundary issues that fragment factual units - Top-k dilution where noise dominates the retrieved set

The diminishing effectiveness of safety and factual verification filters over time. A hallucination rate spike may not reflect an increase in raw model confabulation, but rather a silent failure of the detection layer. This occurs when: - Factual verification models themselves drift - Regex-based filters fail to catch new hallucination patterns - Thresholds become miscalibrated due to distributional shift. Continuous red-teaming and A/B testing of guardrails against known hallucination samples is required to detect this decay.