Few-Shot Example Poisoning

The attacker modifies the input-output pairs in the few-shot prompt template. By altering the labels or outputs of example demonstrations, the model learns a malicious mapping that activates when it encounters a specific trigger phrase or pattern in production. This is distinct from prompt injection because it doesn't override instructions—it retrains behavior in-context.

A targeted attack where the ground-truth labels in few-shot examples are systematically inverted or replaced. For a classification agent, 'positive' examples are labeled 'negative' and vice versa. For a code-generation agent, secure code examples are labeled as 'compliant' while vulnerable code is labeled as 'best practice'. The model internalizes this inverted mapping and reproduces it when the trigger context appears.

The attacker injects malicious reasoning steps into few-shot examples that include chain-of-thought demonstrations. The poisoned reasoning teaches the model to:

• Rationalize harmful outputs as logical conclusions
• Bypass safety checks by framing them as 'edge cases'
• Propagate errors through multi-step reasoning chains This is particularly dangerous because the model learns to justify its harmful behavior with plausible-sounding logic.

In RAG-augmented few-shot systems, attackers poison the example retrieval corpus so that malicious demonstrations are dynamically pulled into the prompt at inference time. Unlike static prompt poisoning, this attack exploits the retrieval mechanism to serve different poisoned examples based on the user's query, making detection significantly harder. The attack surface expands to include the vector store and embedding pipeline.

A sophisticated attack where poisoned few-shot examples for one task influence the model's behavior on unrelated tasks. By exploiting the model's tendency to generalize patterns across its context window, an attacker can plant examples in a seemingly benign classification task that alter the model's behavior on a downstream code generation or tool-calling task. This exploits the shared attention mechanism across the entire prompt.

Treat few-shot examples as immutable, version-controlled artifacts rather than dynamic strings. Store examples in a read-only repository with cryptographic hashes. The agent's system prompt references a specific, pinned version. Any change triggers a CI/CD pipeline that requires explicit approval and generates a diff. This prevents runtime injection and ensures that the model's in-context learning foundation is auditable and rollback-ready.

Implement a trust-on-read mechanism where each few-shot example is signed with a digital signature or HMAC. Before the agent assembles its context window, a verification layer checks the integrity and origin of every example. Examples failing verification are stripped and flagged. This defends against man-in-the-middle injection in retrieval pipelines and ensures that only examples from authorized curators influence the model's behavior.

Deploy a lightweight classifier or embedding-based anomaly detector that scans few-shot examples for semantic deviation before they enter the context window. The filter compares each example's embedding against a baseline distribution of known-clean examples. Outliers are quarantined. This catches adversarial paraphrasing attacks where malicious input-output mappings are disguised with synonym substitution to bypass exact-match filters.

Architect the prompt assembly process to place few-shot examples in a dedicated, isolated segment of the context window, separated from user input and retrieved documents by explicit boundary tokens. This leverages the model's attention mechanism to create a logical firewall. Even if an attacker injects content into the user or retrieval segments, it cannot bleed into the few-shot region and corrupt the in-context learning demonstrations.

Continuously monitor agent outputs for statistical deviations that indicate poisoned few-shot learning. Establish a baseline of expected responses for a held-out set of trigger queries. A monitoring service periodically sends these probes and compares the outputs using semantic similarity metrics. A sudden shift in behavior—such as the model suddenly complying with a previously rejected malicious pattern—triggers an alert and an automatic context window flush.

Mandate that all few-shot examples pass through a human review gate before deployment. The review process checks for unintended biases, subtle malicious mappings, and consistency with the agent's safety policy. This is especially critical for examples sourced from user interactions or automated generation pipelines. The curation interface should highlight the exact input-output pair and require explicit sign-off, creating an audit trail for every example in production.