Time-Aware Answer Confidence Scoring Prompt

This prompt is designed for evaluation engineers and AI quality teams who need to adjust an answer's initial confidence score based on the temporal alignment between the evidence used and the query's time sensitivity. It is a post-answer-generation calibration layer, not a primary answer synthesizer. Use it when your system already produces an initial confidence score, has extracted publication dates from retrieved evidence, and has classified the query's temporal sensitivity (e.g., real-time, recent, historical, time-agnostic). The core job-to-be-done is preventing a high semantic match from masking stale data—a 95% confidence score on a stock price answer is misleading if the supporting evidence is six months old for a query about today's price.

The ideal user has a RAG pipeline or agentic system where confidence scores are surfaced to end users or logged for monitoring and audit. You should have upstream components that handle evidence retrieval, date extraction, and query classification before invoking this prompt. The prompt expects structured inputs: an initial confidence score (0.0–1.0), a list of evidence objects with publication dates, and a query temporal classification with a recommended freshness window. It produces a calibrated confidence score and a detailed temporal adjustment explanation suitable for logging, user-facing transparency, or downstream evaluation dashboards. Wire this into your scoring pipeline after answer generation but before the confidence score is committed to your monitoring store or shown to users.

Do not use this prompt as a substitute for proper retrieval filtering. If your retrieval system can already exclude stale documents based on date ranges, do that first—this prompt is a safety net for cases where stale evidence slips through or where the temporal sensitivity of a query requires nuanced adjustment beyond binary filtering. Also avoid using this prompt for queries classified as 'time-agnostic' where temporal adjustment adds no value and only consumes latency and cost. For high-stakes domains like financial reporting or clinical decision support, always pair the calibrated output with human review checkpoints and log the full adjustment explanation for audit trails. The next section provides the copy-ready template you can adapt and deploy.

This template is the core instruction set for the Time-Aware Answer Confidence Scoring Prompt. It instructs the model to take a base confidence score, the evidence's publication date, and the query's temporal sensitivity, and output a calibrated score with a clear explanation of the adjustment. The prompt is designed to be stateless and can be called repeatedly within a batch evaluation pipeline. Before using it, ensure you have a dataset of known-correct answers to measure calibration error (ECE) and that you've defined your domain-specific time-decay parameters.

textCopy
You are a confidence calibration engine. Your task is to adjust an initial answer confidence score based on the temporal relevance of the evidence used to generate that answer.

**Input Data:**
- [QUERY]: The user's original question.
- [ANSWER]: The generated answer being evaluated.
- [BASE_CONFIDENCE_SCORE]: A float between 0.0 and 1.0 representing the initial confidence in the answer's correctness, ignoring temporal factors.
- [EVIDENCE_PUBLICATION_DATE]: The ISO 8601 date (YYYY-MM-DD) when the primary supporting evidence was published.
- [QUERY_REFERENCE_DATE]: The ISO 8601 date (YYYY-MM-DD) representing "now" for the query's context.
- [TEMPORAL_SENSITIVITY_CLASS]: One of "real-time" (seconds to hours), "recent" (days to weeks), "historical" (months to years), or "time-agnostic" (timeless facts).
- [DOMAIN_DECAY_RULES]: A description of how quickly information becomes stale in this domain (e.g., "Financial news is stale after 24 hours; earnings data is stale after one quarter.").

**Instructions:**
1. Calculate the age of the evidence in days: `age_days = (QUERY_REFERENCE_DATE - EVIDENCE_PUBLICATION_DATE)`.
2. Determine the maximum acceptable age (`max_age_days`) based on the [TEMPORAL_SENSITIVITY_CLASS] and [DOMAIN_DECAY_RULES].
3. Calculate a temporal relevance factor (`T`) between 0.0 and 1.0:
   - If `age_days <= max_age_days`, `T = 1.0`.
   - If `age_days > max_age_days`, `T = max(0.0, 1.0 - (age_days - max_age_days) / max_age_days)`. This applies a linear decay after the max age is exceeded.
   - If [TEMPORAL_SENSITIVITY_CLASS] is "time-agnostic", `T = 1.0`.
4. Calculate the adjusted confidence score: `ADJUSTED_CONFIDENCE = BASE_CONFIDENCE_SCORE * T`.
5. Generate a concise explanation for the adjustment.

**Output Format:**
You must respond with a single, valid JSON object conforming to this schema:
{
  "adjusted_confidence_score": <float, 0.0-1.0>,
  "temporal_relevance_factor": <float, 0.0-1.0>,
  "evidence_age_days": <integer>,
  "max_acceptable_age_days": <integer>,
  "adjustment_explanation": "<string explaining the calculation and reasoning>"
}

**Example:**
[QUERY]: "What is Acme Corp's latest stock price?"
[ANSWER]: "$150.25"
[BASE_CONFIDENCE_SCORE]: 0.95
[EVIDENCE_PUBLICATION_DATE]: "2024-05-20"
[QUERY_REFERENCE_DATE]: "2024-05-22"
[TEMPORAL_SENSITIVITY_CLASS]: "real-time"
[DOMAIN_DECAY_RULES]: "Stock prices are stale after 1 day."

Output:
{
  "adjusted_confidence_score": 0.0,
  "temporal_relevance_factor": 0.0,
  "evidence_age_days": 2,
  "max_acceptable_age_days": 1,
  "adjustment_explanation": "Query requires real-time data with a max age of 1 day. Evidence is 2 days old, resulting in a temporal relevance factor of 0.0. The base confidence of 0.95 is reduced to 0.0."
}

To adapt this template, you must replace the square-bracket placeholders with live data from your evaluation harness. The most critical step is defining [DOMAIN_DECAY_RULES] precisely, as this directly controls the max_age_days calculation. For a financial use case, you might hardcode rules like "Earnings reports are stale after 90 days; analyst ratings are stale after 30 days." For a news summarization system, you might set a blanket rule of 24 hours. The linear decay function provided is a sensible default, but you can replace it with an exponential decay function in the instructions if your offline calibration tests show it fits your data better. Always run this prompt against a golden dataset where you know the correct adjusted_confidence_score to measure Expected Calibration Error (ECE) before production deployment.

This prompt is designed to operate as a post-generation scoring step, not a user-facing response. In a production pipeline, the model generates an answer with citations and metadata, and this prompt receives that output alongside the evidence set and query timestamp. The harness should extract the original confidence score from the generation step, pass it through this temporal adjustment prompt, and return a calibrated score with an explanation. The output is consumed by downstream routing logic—such as escalating low-confidence answers for human review, suppressing responses below a threshold, or logging adjustments for audit.

Implement the harness as a structured validation layer around the LLM call. The prompt expects a JSON input with fields for [QUERY], [QUERY_TIMESTAMP], [ANSWER_TEXT], [ORIGINAL_CONFIDENCE_SCORE], [EVIDENCE_LIST] (each with publication_date, relevance_score, and source_authority), and [TEMPORAL_SENSITIVITY] (e.g., 'real-time', 'recent', 'historical', 'time-agnostic'). After the model returns the adjusted score and explanation, validate that the output conforms to the expected schema: a float between 0 and 1 for adjusted_confidence, a non-empty string for temporal_adjustment_explanation, and an array of evidence_freshness_assessments. If validation fails, retry once with the error message injected into the prompt context. Log every adjustment for calibration analysis—compare adjusted scores against known-correct answer datasets to detect systematic over- or under-adjustment. For high-risk domains like financial or medical intelligence, route all adjusted scores below 0.7 to a human review queue and attach the full temporal adjustment explanation.

Model choice matters here. Use a model with strong instruction-following and structured output capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). Avoid smaller models that may conflate confidence adjustment with answer rewriting. If you are processing batches for evaluation, run this prompt as an offline scoring pass over a golden dataset and measure calibration error (ECE) before and after temporal adjustment. Do not deploy the adjustment into a user-facing path until the calibration metrics show improvement. The next step after implementing this harness is to build a monitoring dashboard that tracks adjustment magnitude over time—if the average adjustment drifts upward, it may signal that your retrieval pipeline is serving increasingly stale evidence.