Adaptive Demonstration

Unlike static few-shot prompts, adaptive demonstration selects examples on-the-fly for each query. The core mechanism is semantic retrieval, where a system embeds the user's query and retrieves the most relevant demonstrations from a curated datastore. Common techniques include:

• k-Nearest Neighbors (k-NN) Retrieval: Finds the k examples with embedding vectors most similar to the query.
• Dense Passage Retrieval: Uses a dual-encoder model to score passage relevance.
• Hybrid Search: Combines semantic similarity with keyword matching (BM25) for precision. This ensures the model's context is primed with the most pertinent task mappings.

Adaptive demonstration explicitly manages the fixed-context bottleneck of transformer models. It performs selective inclusion, choosing only the most valuable examples to fit within token limits (e.g., 128K tokens). Strategies include:

• Example Summarization: Compressing long demonstrations into concise versions.
• Relevance Thresholding: Setting a cosine similarity cutoff; only examples above the threshold are included.
• Iterative Pruning: Starting with many examples and removing the least relevant until the prompt fits. This maximizes the utility of every token in the context window, directly combating performance degradation from irrelevant or excessive examples.

The number of demonstrations (k) is not fixed. The system adapts the shot count based on inferred query complexity. A simple heuristic might use:

• Few shots (1-3) for straightforward, canonical queries.
• Many shots (5-10+) for ambiguous, complex, or edge-case queries. Complexity can be estimated via:
• Query Embedding Norm: The magnitude of the query's embedding vector can correlate with specificity.
• Uncertainty Scoring: Using a separate lightweight model to predict the target model's potential confusion.
• Retrieval Score Variance: If the top retrieved examples have low similarity scores, the query may be novel, warranting more demonstrations.

Advanced adaptive systems incorporate online learning based on model output quality. This creates a feedback loop where demonstration selection improves over time. Mechanisms include:

• Success/Failure Logging: Tracking which retrieved examples led to correct vs. incorrect outputs for a given query pattern.
• Embedding Space Adjustment: Fine-tuning the retrieval model's embeddings based on this performance signal, pushing successful examples closer to similar queries in the vector space.
• Demonstration Reweighting: Increasing the likelihood of selecting examples historically associated with high-quality outputs. This moves adaptation beyond simple retrieval towards continuous optimization of the in-context learning process.

Adaptive demonstration is not isolated; it integrates into larger agentic architectures. Key integrations are:

• Retrieval-Augmented Generation (RAG): The same retrieval system can fetch both factual knowledge and task demonstrations, blending them in a single context.
• Tool-Augmented Reasoning: Demonstrations can include examples of correct function calling or ReAct (Reasoning-Acting) patterns, dynamically selected based on the tools needed for the current query.
• Structured Output Guidance: Retrieved examples can enforce specific JSON or XML schemas, adapting the formatting demonstration to the user's stated requirement. This positions adaptive demonstration as a core context engineering component within production AI systems.

Beyond showing correct mappings, adaptive prompts can include strategic negative examples to sharpen model understanding. This involves:

• Contrastive Demonstrations: Providing an incorrect output for a given input, explicitly labeled as such (e.g., // Bad output:), to teach boundary conditions.
• Calibration Examples: Including demonstrations that adjust the model's output distribution, such as examples that discourage overconfidence or reduce bias.
• Error-Correction Pairs: Showing a flawed model output followed by a corrected version, teaching a self-revision process. These are selectively retrieved when the system detects a query domain where the model is known to exhibit specific failure modes.

This pattern adjusts the quantity and complexity of demonstrations based on an estimated difficulty of the user's query. A simple system might classify queries as 'simple' or 'complex' using a heuristic or a lightweight classifier.

• Simple Query: Uses 1-2 concise, canonical examples.
• Complex Query: Uses 3-5 detailed examples, potentially including intermediate reasoning steps or edge cases.

Example: A customer support bot might use a single example for a common FAQ but retrieve three detailed troubleshooting examples for a novel technical issue.

Instead of a fixed set of examples, this approach uses embedding-based retrieval to dynamically fetch the most relevant demonstrations from a large corpus for each query.

• A vector database stores embeddings of hundreds of potential example problems and solutions.
• For a new user query, a k-nearest neighbors (k-NN) search finds the N most semantically similar examples.
• These are inserted into the prompt context just before the final query.

This is a core technique in Retrieval-Augmented ICL, ensuring demonstrations are always contextually relevant, maximizing the utility of the limited context window.

When the prompt context (user history + instructions + demonstrations) approaches the model's token limit, adaptive strategies compress or summarize existing demonstrations to make room for new queries.

Techniques include:

• Truncation: Removing the least relevant or oldest examples based on a relevance score.
• Summarization: Using a model call to condense a detailed multi-step example into a concise summary, preserving the core input-output mapping.
• Abstraction: Replacing specific values in examples with placeholders (e.g., [CITY_NAME], [PRODUCT_ID]) to reduce token count while maintaining the structural pattern.

This pattern is critical for long-running conversational agents.

This closed-loop system uses model output evaluation to trigger changes in the demonstration strategy. If the model's recent responses fail quality checks, the system adapts the examples in subsequent prompts.

Implementation Flow:

1. The system sends a prompt with a baseline set of demonstrations.
2. The model's output is scored by a validator (e.g., for format correctness, hallucination, or user feedback).
3. If the score falls below a threshold, the system swaps in new demonstrations from a pre-defined 'recovery' set designed to correct the observed failure mode.

Example: If a JSON-generation model starts omitting required fields, the next prompt includes examples that explicitly show handling of missing data.

This educational technique starts with highly detailed, step-by-step examples and gradually reduces the explicitness (fades the scaffolding) as the model demonstrates competence on the task over a session.

Session Progression:

• Prompt 1: Full Chain-of-Thought examples with explicit reasoning.
• Prompt 2: Examples with abbreviated reasoning steps.
• Prompt 3: Direct input-output examples only.
• Prompt 4+: Zero-shot or one-shot prompting, relying on the model's now-primed understanding.

This pattern is effective for teaching a model a complex, novel task within a single session, conserving tokens after the initial learning phase.

For tasks requiring structured output generation (JSON, XML, YAML), this pattern detects the user's implied or requested format and selects demonstrations that match it exactly.

Process:

1. A preliminary system prompt or a lightweight model classifies the desired output structure from the user's instruction (e.g., 'output as a list', 'give me JSON').
2. The system selects 2-3 examples from a library categorized by output format.
3. These format-specific demonstrations are inserted, drastically reducing formatting errors.

Example: A user asking 'list the items' gets examples showing bulleted lists, while a user asking 'create a JSON object' gets examples with valid JSON schemas.

In-context learning (ICL) is the foundational paradigm where a pre-trained language model performs a new task by conditioning its output on a few input-output examples provided within its prompt, without updating its internal parameters. Adaptive demonstration is a specialized strategy within ICL.

• Core Mechanism: The model infers the task pattern from the demonstrations and generalizes it to a new query.
• Parameter-Free: This is a form of inference-time adaptation; the model's weights remain frozen.
• Contrast with Fine-Tuning: Unlike fine-tuning, ICL requires no gradient updates, making it fast and flexible for prototyping.

Demonstration selection is the critical process of choosing which specific few-shot examples to include in a prompt to maximize model performance. Adaptive demonstration automates and optimizes this selection.

• Key Criteria: Selection is based on exemplar quality, demonstration diversity, and semantic similarity to the target query.
• Common Strategies:
  • Semantic Similarity Selection: Using embeddings to retrieve examples closest to the query.
  • k-NN Demonstration Retrieval: A concrete implementation of embedding-based retrieval.
  • Query-Example Matching: Scoring and ranking candidates for relevance.
• Goal: To provide the most informative seed examples that clarify the input-output mapping and label space.

Dynamic few-shot prompting is an adaptive technique where the composition of the prompt—including the number and content of demonstrations—is determined dynamically for each individual query. This is the operational implementation of adaptive demonstration.

• On-the-Fly Construction: The system retrieves or generates context-specific examples in real-time, rather than using a static set.
• Drivers for Adaptation: Adjustments can be based on:
  • Perceived query difficulty or complexity.
  • Available space in the model's context window.
  • Real-time feedback on previous model outputs.
• Enabling Technology: Often relies on retrieval-augmented ICL architectures with an embedding index of potential examples.

Retrieval-augmented in-context learning (Retrieval-Augmented ICL) is a architecture that dynamically fetches relevant demonstrations from a datastore to construct the context for each query. It is the primary technical backbone for implementing adaptive demonstration.

• How It Works: A retriever model (e.g., using embedding-based retrieval) searches a vector database of candidate examples to find the most relevant ones for the current user input.
• Separation of Concerns: Decouples the example corpus (stored knowledge) from the language model's fixed parameters.
• Benefits: Enables scaling to massive example sets and ensures demonstrations are contextually relevant, directly supporting adaptive strategies.

An instruction-example pair is the fundamental building block of a few-shot prompt, combining a natural language task specification with one or more concrete demonstrations. Adaptive demonstration strategies often optimize these pairs.

• Structure: Typically formatted as: [Instruction] + [Example 1 Input] -> [Example 1 Output] + [Example 2 Input] -> [Example 2 Output].
• Role of Formatting: Clear example formatting (using delimiters, labels) is crucial for the model to parse the instruction-example pair correctly.
• Adaptive Consideration: An adaptive system might adjust the verbosity of the instruction or the complexity of the paired examples based on the model or task.

Inference-time adaptation is the broad category of techniques that modify a model's behavior during the forward pass based on provided context, without updating its weights. Adaptive demonstration is a prominent method within this category.

• Contrast with Training: Encompasses all gradient-free learning and parameter-free adaptation methods.
• Primary Methods:
  • In-context learning (ICL) with few-shot examples.
  • Prompt tuning with soft prompts (in some definitions).
  • System prompt adjustments.
• Key Advantage: Allows for rapid, flexible task switching with a single frozen model inference instance, which is highly efficient for production systems.