Token-Efficient Demonstrations

This technique involves condensing verbose examples into their core semantic essence. Instead of providing a full, detailed conversation or document, you extract the key reasoning pattern or structural template. For example, a multi-turn customer service dialogue can be summarized as:

• Input Pattern: User complaint about [issue] + order #[number].
• Agent Action Pattern: Apologize + request details ([specific field]) + promise resolution timeline ([time unit]).
• Output Pattern: Provide summary ticket #[generated_id] and outline next steps. This reduces token count while preserving the task logic the model must learn.

Replace repeated, predictable phrases with consistent abbreviations or single symbols. This is highly effective for structured data formatting tasks.

• Example: Instead of "Convert the following date: 'January 15, 2023' to ISO format." and "Convert the following date: 'March 3, 2024' to ISO format."
• Token-Efficient Version: "Date: 'Jan 15, 2023' -> ISO: 2023-01-15" and "Date: 'Mar 3, 2024' -> ISO: 2024-03-03" Key principles:
• Define the substitution key once in the system prompt (e.g., -> means "outputs").
• Use abbreviations (Msg: for Message:, Src: for Source:).
• Maintain absolute consistency across all demonstrations to avoid confusing the model.

Use a single canonical template with clear placeholders for variable content, rather than repeating full examples with different values. This teaches the model the abstract schema of the task.

• Inefficient: Two full JSON output examples with different key-value pairs.
• Efficient: One demonstration with a commented template:

codeCopy
Input: Extract name and company from: '{user_input}'
Output: {
  "schema": "contact_info",
  "data": {
    "name": "<extracted_name>", // Placeholder
    "company": "<extracted_company>" // Placeholder
  }
}

The model learns the JSON structure and intent from one heavily annotated example, which is then applied to subsequent queries. This is a cornerstone of program-aided language model (PAL) prompting.

Fuse the task instruction directly into the demonstration's format, eliminating the need for separate, verbose instructional preamble. The demonstration itself becomes the instruction.

• Separate (Inefficient): "Translate the following technical terms to Spanish. Be concise." followed by "Input: 'latency' Output: 'latencia'".
• Integrated (Efficient): "Term->ES: 'latency' -> 'latencia'" and "Term->ES: 'throughput' -> 'rendimiento'". The Term->ES: prefix implicitly carries the instruction. This technique relies on the model's ability to perform meta-learning from the demonstration's framing, drastically reducing instructional overhead.

For complex, multi-step reasoning tasks, break the demonstration into core logical steps rather than a monolithic block of text. Use symbolic reasoning chains (like a compressed Chain-of-Thought).

• Verbose CoT: "The user is asking for the total. The first item is $10. The second is $20. $10 + $20 = $30. So the total is $30."
• Compressed Chain: "Q: cost of A($10) and B($20)? Steps: 1. Identify costs: 10, 20. 2. Op: sum. 3. Calc: 10+20=30. A: $30" This preserves the causal reasoning structure—which is critical for model performance on arithmetic or logic tasks—while using a fraction of the tokens. It explicitly models the algorithm the model should follow.

A single, well-chosen negative example (showing what not to do) can be more token-efficient than multiple positive examples for establishing boundaries. Use a contrastive format.

• Example: "Classify sentiment: 'I'm okay.' -> Neutral. NOT Positive. (Reason: 'okay' is weak, not affirmative.)" This one demonstration teaches both the correct classification and a key disambiguation rule. It efficiently addresses edge cases and common failure modes. The NOT and parenthetical Reason: provide high-density learning signals, reducing the need for several positive examples to cover the same conceptual ground.

This technique replaces verbose, descriptive variable names with short, conventional abbreviations within code examples.

• Example: Instead of customer_first_name = "Alex", use fn = "Alex".
• Impact: Drastically reduces token count in programming demonstrations while preserving the logical structure the model must learn.
• Best Practice: Maintain internal consistency (e.g., always using fn for first name) and include a brief key if the abbreviations are non-standard.

This method replaces lengthy, narrative-style examples with dense, factual summaries that capture the same logical relationship.

• Example:
  • Inefficient: "The user, named John Doe who lives in London, asked for the weather. The assistant replied, 'It is currently sunny and 22 degrees in London.'"
  • Efficient: "User: Weather in London. Assistant: Sunny, 22°C."
• Mechanism: Removes syntactic filler and conversational markers, distilling the example to its core task schema (input intent → output fact).

Instead of full, concrete examples, this uses abstract templates with typed placeholders, teaching the model the required output format.

• Example: For a JSON generation task:
  • Full Example: {"input": "Translate 'hello' to Spanish", "output": {"translation": "hola"}}
  • Schema Placeholder: {"input": "<text>", "output": {"translation": "<translated_text>"}}
• Use Case: Highly effective for structured output generation tasks where the format, not the content, is the primary learning objective.

This approach merges the task instruction directly into the demonstration's format, eliminating redundant explanatory text.

• Example:
  • Standard: Instruction: "Extract the date." Example: "Text: 'Meeting on 2023-12-25' → Date: 2023-12-25"
  • Integrated: Extract Date: 'Meeting on 2023-12-25' → 2023-12-25
• Benefit: The model learns the instruction and the demonstration pattern simultaneously, a form of task-example alignment that reduces total prompt length.

This advanced technique shows only the difference between a correct and incorrect output, focusing the model on critical decision boundaries.

• Example: For a sentiment classification task:
  • Inefficient: Two full examples showing a positive and a negative review.
  • Efficient: Review: 'The product was okay, not great.' Sentiment: Neutral. [*Not Positive*]
• Mechanism: Uses concise meta-annotations (e.g., [*Not X*]) to highlight subtle distinctions, teaching the model to avoid common pitfalls with minimal tokens.

A single, densely constructed example is designed to demonstrate patterns for multiple related subtasks.

• Example: For a data extraction pipeline: Input: 'Invoice #501 from ACME on Jan 5 for $500.' → Output: {"id": 501, "vendor": "ACME", "date": "2024-01-05", "amount": 500}
• Impact: This one demonstration teaches entity recognition, date normalization, and numeric extraction simultaneously, serving as a multi-purpose pattern that reduces the need for separate examples for each subtask.

In-context learning is a prompting paradigm where a large language model performs a new task by conditioning its response on a few provided input-output examples, called demonstrations, without updating its internal parameters. It relies on the model's ability to infer patterns from the context.

• Core Mechanism: The model uses the demonstrations as a temporary, task-specific guide.
• Parameter-Free: No gradient updates are performed; learning is ephemeral and contained within the forward pass.
• Foundation: The underlying capability that makes few-shot prompting possible.

Few-shot prompting is the practical technique of providing a language model with a small number (K) of task-specific examples within its input context to guide its response for a new, similar query. It is the application of in-context learning.

• K-Shot: The number of examples provided (e.g., 2-shot, 5-shot).
• Static vs. Dynamic: Demonstrations can be fixed or retrieved on-the-fly.
• Primary Use Case: Rapid prototyping and task adaptation without model fine-tuning.

Demonstration selection is the strategic process of choosing which few-shot examples to include in a prompt to maximize a model's in-context learning performance on a target task. It moves beyond random selection.

• Key Criteria: Relevance (similarity to query) and Diversity (covering the problem space).
• Methods: Includes embedding-based similarity search, utility scoring, and heuristic rules.
• Impact: Poor selection can introduce bias or provide unhelpful context, degrading performance.

Demonstration ordering is the strategic arrangement of the sequence of few-shot examples within a prompt, which can significantly influence a model's in-context learning performance due to recency and priming effects.

• Common Strategies: Order by similarity to the query (closest first or last), by difficulty (easy to hard), or to maximize diversity.
• Empirical Finding: Order is not neutral; it acts as an implicit instruction.
• Optimization Target: Part of the hyperparameter search for optimal ICL setups.

Retrieval-augmented in-context learning is a technique that dynamically retrieves the most relevant few-shot examples from a corpus based on the input query, rather than using a static, pre-defined set of demonstrations.

• System Architecture: Combines a retriever (e.g., dense vector search) with the LLM.
• Advantage: Adapts context to the specific query, improving relevance and efficiency.
• Pipeline: Query -> Retrieval -> Demonstration Formatting -> Prompt Construction -> LLM Inference.

Context window optimization is the strategic management of the limited token budget within a model's context, balancing the inclusion of instructions, demonstrations, and the query for maximum ICL efficiency and performance.

• Trade-offs: More demonstrations may help but consume space needed for complex reasoning.
• Techniques: Includes token-efficient demonstrations, instruction compression, and strategic truncation.
• Direct Link: The primary driver for creating token-efficient demonstrations in the first place.