Stale Context vs New Information Arbitration Prompt

This prompt is for copilot and agent builders whose systems receive conflicting information from two sources: a stale or cached context (prior turns, old retrieval results, session summaries) and fresh input (the user's latest message, a live API response, a new document). The job-to-be-done is arbitration—deciding which source to trust, producing a resolution, and flagging cases where the system cannot decide confidently. The ideal user is an AI engineer integrating this prompt into a RAG pipeline, a dialogue state manager, or a tool-calling agent that must reconcile contradictions before acting or responding. You need the stale context, the new information, and any relevant metadata (timestamps, source types, confidence scores) as inputs.

Use this prompt when the cost of trusting stale context is high—incorrect answers, wrong tool calls, policy violations, or user-facing contradictions that erode trust. It is appropriate for workflows where the system has already detected a potential conflict (via a staleness detection trigger, a user correction, or a live-vs-cached discrepancy) and now needs a structured resolution. Do not use this prompt when the conflict is trivial, when the new information is clearly authoritative by system design (e.g., a user's explicit correction always wins), or when the latency budget cannot tolerate an extra model call. In those cases, hard-code the resolution rule. Also avoid this prompt when the arbitration itself requires domain expertise that the model lacks—legal, medical, or financial conflicts where an incorrect resolution causes harm should escalate to a human reviewer, not rely on a prompt-level arbitration.

Before wiring this prompt into production, define your evals: create a test set of intentionally ambiguous conflicts where the correct resolution is known, including cases where the stale context is actually correct and the new information is wrong (e.g., a user misremembering a fact). Measure whether the prompt correctly identifies these reversals. The most common failure mode is recency bias—the model defaults to trusting the newer information even when the older context is more authoritative. Your harness should explicitly test for this. After reading this section, copy the prompt template, adapt the placeholders to your source schemas, and run it against your conflict test cases before integrating it into your agent loop.

This prompt template is the core of the Stale Context vs New Information Arbitration workflow. It forces the model to explicitly compare two conflicting pieces of information—one from potentially stale session context or a prior retrieval, and one from fresh user input or a live data query—and produce a structured resolution. The template is designed to be copied directly into your prompt management system, with square-bracket placeholders that you replace at runtime. The output schema is strict: the model must select one source as authoritative, reject the other, and provide a reasoning chain that can be audited. This is not a prompt for casual use; it is a production arbitration step that should be followed by validation against ground truth or escalation to a human reviewer when confidence is low.

textCopy
You are an information arbitration system. Your task is to resolve a conflict between two pieces of information:

1. STALE_CONTEXT: Information from prior conversation turns, cached retrievals, or session state that may be outdated.
2. NEW_INFORMATION: Fresh user input, a live data query result, or a recent correction that contradicts the stale context.

## INPUTS

STALE_CONTEXT:
"""
[STALE_CONTEXT]
"""

NEW_INFORMATION:
"""
[NEW_INFORMATION]
"""

DOMAIN: [DOMAIN]
RISK_LEVEL: [RISK_LEVEL]

## ARBITRATION RULES

1. Prefer NEW_INFORMATION when it is specific, recent, and directly contradicts STALE_CONTEXT.
2. Retain STALE_CONTEXT only when NEW_INFORMATION is ambiguous, lower authority, or does not directly invalidate the stale fact.
3. If both sources could be partially correct, explain the overlap and the specific point of conflict.
4. If you cannot resolve the conflict with high confidence, set escalation_required to true.
5. For RISK_LEVEL = "high", any uncertainty must trigger escalation.
6. Do not invent facts to reconcile the conflict. Work only with the provided inputs.

## OUTPUT SCHEMA

Return a JSON object with exactly these fields:

{
  "selected_fact": "The fact chosen as authoritative, exactly as stated in the winning source.",
  "selected_source": "STALE_CONTEXT" | "NEW_INFORMATION",
  "rejected_fact": "The fact from the losing source.",
  "rejected_source": "STALE_CONTEXT" | "NEW_INFORMATION",
  "arbitration_rationale": "Step-by-step reasoning: why one source was trusted over the other.",
  "confidence": 0.0_to_1.0,
  "escalation_required": true | false,
  "escalation_reason": "If escalation_required is true, explain what additional information or human review is needed. Otherwise null.",
  "user_facing_caveat": "If confidence is below 0.8, provide a concise statement for the end user explaining the uncertainty. Otherwise null."
}

## CONSTRAINTS

[CONSTRAINTS]

## EXAMPLES

[EXAMPLES]

Adapting the template: Replace [STALE_CONTEXT] with the exact text from your session state, cached retrieval, or prior turn. Replace [NEW_INFORMATION] with the user's latest statement, a fresh API response, or a correction. The [DOMAIN] field helps the model calibrate its confidence—medical and financial domains should default to lower confidence thresholds. [RISK_LEVEL] should be set to "high" for regulated domains, financial decisions, or clinical contexts, which forces escalation on any uncertainty. Use [CONSTRAINTS] to inject domain-specific rules, such as "always prefer the live pricing API over cached prices" or "user corrections always override prior assistant assumptions unless the correction is ambiguous." The [EXAMPLES] placeholder should be filled with 2-3 few-shot examples showing correct arbitration in your domain, including at least one example where escalation is correctly triggered. After the model returns the JSON, validate that selected_fact and rejected_fact are not identical, that confidence is between 0.0 and 1.0, and that escalation_required is true whenever confidence is below your production threshold. For high-risk domains, route any output with escalation_required: true to a human review queue before the selected fact is used downstream.

The arbitration prompt is a decision point, not a final answer. It sits between context retrieval and response generation, receiving two conflicting claims—one from stale context and one from fresh input or live data—and producing a structured resolution. In a production harness, this prompt should be called as a dedicated step before the assistant commits to an answer. The harness must supply the stale claim, the fresh claim, the source metadata for each (origin, timestamp, retrieval method), and the user's original query. The prompt returns a JSON object containing the selected fact, the rejected alternative, the arbitration rationale, a confidence score, and an escalation flag. Do not pass this output directly to the user; it is a machine-readable signal for downstream logic.

Wire the prompt into a decision pipeline: (1) detect a conflict between a cached or prior-turn fact and a new user statement or live API result; (2) call the arbitration prompt with both claims and their provenance; (3) parse the JSON output and validate the schema—reject any response missing required fields like selected_fact, confidence, or escalate; (4) if escalate is true or confidence is below your threshold (e.g., <0.7), route to a human review queue with the full arbitration payload and source evidence; (5) if arbitration passes, use selected_fact as the ground truth for the current turn and log the rejected alternative for audit. Implement a retry with exponential backoff (max 2 retries) if the model returns malformed JSON or fails schema validation. Use a model with strong instruction-following and JSON mode enabled—GPT-4o, Claude 3.5 Sonnet, or equivalent. For high-stakes domains (healthcare, finance, legal), always require human review when escalate is true and log every arbitration decision with the full input context, model response, and reviewer action for audit trails.

The most common production failure is the model hedging by selecting both claims or producing an unparseable rationale. Mitigate this with strict output validation: check that selected_fact is exactly one of the two input claims, not a paraphrase or hybrid. If the model invents a third option, treat it as a validation failure and retry. A second failure mode is over-escalation—the model flags every conflict for human review, creating a bottleneck. Tune your confidence threshold and consider adding a cost-aware routing layer that auto-resolves low-stakes conflicts (e.g., product description discrepancies) while escalating safety-critical ones (e.g., dosage instructions). Log the arbitration_rationale field to an observability platform so you can review patterns: are certain source pairs consistently producing low-confidence arbitrations? That signals a need for better retrieval freshness or source authority metadata. Finally, never use the arbitration output to silently overwrite a knowledge base or cache—treat it as a session-scoped resolution unless a human reviewer explicitly approves the update.