Demonstration Bias

Demonstration bias often manifests when the model learns incidental, non-causal patterns from the examples instead of the intended task logic. For instance, if all examples of a sentiment analysis task feature positive reviews about 'coffee' and negative reviews about 'tea', the model may incorrectly associate the product type with the sentiment, rather than the actual review content. This leads to poor generalization on queries that break this superficial pattern.

• Core Mechanism: The model performs pattern matching on surface-level features in the context.
• Result: The model's output becomes contingent on irrelevant variables present in the demonstrations.

If the few-shot examples contain societal or demographic biases, the model can amplify these stereotypes in its predictions. This is a critical failure mode for fairness. For example, in a occupation classification task, if demonstrations consistently pair certain professions with a specific gender, the model will replicate that association, propagating harmful bias.

• Source Bias: Bias originates from the human-curated or retrieved example set.
• In-Context Propagation: The model treats the biased correlation as a valid rule for the task at hand.

The model may latch onto the stylistic or formatting quirks of the demonstrations rather than the underlying task. This includes:

• Lexical Bias: Over-reliance on specific keywords or phrases present in the examples.
• Structural Bias: Mimicking exact output formats (e.g., bullet points, specific punctuation) even when the instruction requests a different style.
• Length Bias: Producing outputs of similar length to the examples, regardless of query complexity. This reduces robustness, as performance degrades when the query deviates from the demonstration 'template'.

Demonstration bias is often coupled with a strong primacy or recency effect, where the model's predictions are disproportionately influenced by the first or last few-shot example in the sequence. This ordering sensitivity means that the same set of examples can produce different model behaviors simply based on their arrangement, introducing non-determinism.

• Primacy Effect: The first example sets a strong anchor for the task 'frame'.
• Recency Effect: The final example has undue weight in the final reasoning step.
• Mitigation: Strategies like demonstration shuffling or scoring can help identify and reduce this bias.

The provided demonstrations can inadvertently redefine the task for the model, leading to task mis-specification. If the examples solve a simpler or different subtask than the one described in the instructions, the model will follow the demonstrated pattern. For instance, instructions may ask for detailed reasoning, but if all examples show concise answers, the model will default to concise outputs, ignoring the instruction.

• Instruction-Example Conflict: Demonstrations override or contradict the system prompt.
• Result: The actual executed task is defined by the examples, not the intended instructions.

If the selected few-shot examples contain errors—incorrect labels, factual inaccuracies, or misaligned input-output pairs—this label noise is directly learned by the model as ground truth for the in-context task. A single erroneous demonstration can significantly skew predictions, as the model has no external signal to correct it within the context window.

• Error Amplification: A small amount of noise in the context can lead to large performance drops.
• Contrast with Fine-Tuning: Unlike parameter updates, this 'learning' is transient but immediate for the given prompt.
• Detection: Requires rigorous demonstration scoring and validation pipelines.

This occurs when demonstrations consistently pair certain professions, traits, or outcomes with specific demographic groups. For example, providing examples where:

• All CEOs are described with male pronouns and names.
• All nurses are described with female pronouns and names.
• Successful outcomes are only associated with certain nationalities.

The model learns these correlations as a task rule, reproducing the stereotype in its generated responses for new queries, even when the input is neutral.

Bias arises when demonstrations over-emphasize a specific syntactic or structural pattern that is not essential to the core task. The model then prioritizes mimicking the format over understanding the intent.

Example: If all few-shot examples for a 'summarization' task use bullet points, the model may incorrectly assume bullet points are a required output format, even for queries where a prose summary is requested. This biases the model towards surface-level features rather than the underlying semantic task of condensation.

This bias happens when the selected demonstrations are unrepresentatively simple or complex compared to the expected distribution of real user queries.

• Oversimplification: Using only trivial examples can cause the model to fail on edge cases or nuanced inputs, as it hasn't learned to handle complexity.
• Overcomplication: Using only highly complex, edge-case examples can bias the model to over-engineer solutions for straightforward queries, reducing efficiency and clarity. This skew misaligns the model's learned 'task boundary' with actual user needs.

In classification tasks, bias is introduced when the label words (verbalizers) in the demonstrations are inconsistently phrased or carry unintended connotations. For instance:

• Using 'positive' and 'bad' as opposing sentiment labels injects a confound, as 'bad' is not the direct antonym of 'positive'.
• Using lengthy, descriptive phrases (e.g., 'this text expresses great joy') in some examples but single words (e.g., 'happy') in others creates ambiguity about the expected output format. The model learns to map inputs to these specific verbalizations, not the intended conceptual class.

Demonstrations can bias a model towards a specific reasoning strategy, even when multiple valid approaches exist. This is critical in Chain-of-Thought prompting.

Example: If all few-shot examples for math word problems solve using algebraic equations, the model may struggle with a problem best solved by arithmetic estimation or unit analysis. It has been biased to perceive the method as part of the task definition, limiting its ability to select the most efficient or correct solution path for a new problem.

Bias occurs when all demonstrations are drawn from a narrow sub-domain, causing the model to underperform on queries from related but distinct domains. The model's in-context learning becomes over-specialized.

Real-world case: Training a customer support bot with few-shot examples only from 'software billing inquiries' will bias its responses. When faced with a 'hardware troubleshooting' query, it may incorrectly apply billing-related logic or terminology, failing to generalize within the broader 'customer support' task. The demonstrations have anchored it to a specific context.

Demonstration selection is the strategic process of choosing which specific few-shot examples to include in a prompt to maximize a model's in-context learning performance. It directly combats demonstration bias by prioritizing high-quality examples.

• Key Methods: Include embedding-based selection for semantic relevance and diversity-based selection to cover the task's input space.
• Objective: To find examples that provide the clearest, most generalizable pattern for the model to infer, avoiding spurious correlations.

Demonstration ordering is the strategic arrangement of the sequence of few-shot examples within a prompt. The order can introduce bias or mitigate it, significantly influencing the model's learned pattern.

• Primacy & Recency Effects: Models can be disproportionately influenced by the first (primacy) or last (recency) examples in the context.
• Strategic Sequencing: Ordering by difficulty (easy to hard) or by clustering similar examples together are common techniques to steer learning.

Retrieval-augmented in-context learning is a technique that dynamically retrieves the most relevant few-shot examples from a corpus based on the input query, rather than using a static, potentially biased set.

• Dynamic Bias Mitigation: By fetching examples on-demand, RA-ICL reduces the risk of a fixed set perpetuating skewed patterns.
• Implementation: Uses a vector database to perform semantic search, embedding both the query and candidate demonstrations.

Task-example alignment is the property of a demonstration where its format, complexity, and domain closely match the target task. Misalignment is a core source of demonstration bias.

• Bias Source: An example with extraneous details (e.g., all inputs about a specific city) can cause the model to over-index on those details as a required part of the solution.
• Evaluation: Ensuring examples are prototypical and free of confounding variables is key to high alignment.

Demonstration contamination occurs when few-shot examples inadvertently leak information about the test query's expected answer, creating a severe form of measurement bias.

• Effect: Artificially inflates performance metrics, as the model is not performing genuine inference but is subtly "taught" the answer.
• Prevention: Requires rigorous separation of training/development data used for creating demonstrations from the final evaluation test set.

An ICL performance metric is a quantitative measure used to evaluate the effectiveness of an in-context learning setup, which must be carefully chosen to detect demonstration bias.

• Standard Metrics: Include accuracy, F1 score, and exact match.
• Bias Detection: Metrics should be analyzed across diverse slices of data (e.g., different demographics in a classification task) to uncover bias hidden in aggregate scores. A high overall accuracy with low performance on a specific slice indicates demonstration bias.