Context Relevance Decay Scoring Prompt

This prompt is designed for context window managers and AI engineers who need to programmatically estimate how quickly prior conversation turns lose relevance to the current task. The core job-to-be-done is assigning a quantitative decay score and a recommended retention window to each piece of context (turns, retrieved documents, tool outputs) before a context budget is exceeded. This is not a prompt for end-users; it is an infrastructure component that sits between a conversation state tracker and a context assembly step, deciding what to keep, compress, or drop.

The ideal user is a production AI engineer building a multi-turn assistant, copilot, or agent where context windows are a constrained resource. You should use this prompt when you have structured conversation history, a clear definition of the current user goal, and a need to make automated pruning decisions at scale. Required inputs include a structured list of context elements with timestamps or turn indices, a description of the current task or user intent, and a defined output schema for decay scores. Do not use this prompt for real-time, low-latency chat where the overhead of scoring exceeds the cost of simply retaining all context, or in single-turn workflows where there is no history to manage.

This prompt is a decision-support tool, not a final authority. Its output should feed into a deterministic pruning or summarization step in your application code. In high-stakes domains—such as healthcare, legal, or financial advice—a human must review the retention decisions before context is permanently dropped, because an incorrectly pruned fact can lead to harmful omissions. After reading this section, proceed to the prompt template to copy the exact instructions, then review the implementation harness for wiring this into a production context management pipeline.

This prompt template is the core of the Context Relevance Decay Scoring workflow. It instructs the model to act as a context window manager, analyzing each provided context element (a prior turn, a retrieved fact, a tool output) and assigning a decay score that estimates how quickly its relevance will diminish. The output is a structured list of decisions that can be consumed by your application's context pruning or retention logic. Before copying this template, ensure you have a clear definition of your conversation phases and a list of context elements you want to evaluate.

markdownCopy
You are a context window manager for an AI assistant. Your task is to analyze a list of context elements from an ongoing conversation and assign a relevance decay score to each one. This score will be used to decide which elements to retain, compress, or drop as the conversation progresses and the context budget tightens.

**Conversation Phase:** [CURRENT_PHASE]
**User's Latest Utterance:** [USER_INPUT]

**Context Elements to Score:**
[CONTEXT_ELEMENTS]

**Scoring Rubric:**
- **1 (Critical):** Essential for understanding the current turn or immediate next steps. Do not drop.
- **2 (High):** Directly relevant to the active topic but not the immediate focus. Retain if possible.
- **3 (Medium):** Related to the broader session but not the current topic. Candidate for compression.
- **4 (Low):** Resolved, completed, or from a clearly abandoned topic. Safe to drop.
- **5 (Expired):** Factually invalidated by new information or a user correction. Must be dropped or overwritten.

**Output Schema:**
Return a JSON object with a single key "scored_elements" containing an array of objects. Each object must have the following fields:
- "element_id": (string) The identifier of the context element from the input.
- "decay_score": (integer) A score from 1 to 5 based on the rubric.
- "rationale": (string) A brief, specific justification for the score, referencing the conversation phase and user input.
- "recommended_action": (string) One of: "retain", "compress", "drop", "overwrite".

**Constraints:**
- Score every element in the provided list.
- If the user's latest utterance signals a clear topic shift, scores for elements from the prior topic should trend toward 4 or 5.
- A user correction that contradicts a prior element must result in a score of 5 (Expired) for that element.
- Do not drop elements that contain unresolved questions or pending actions, even if the topic appears to have shifted.

To adapt this template, replace the square-bracket placeholders with your application's data. [CURRENT_PHASE] should be a string label from your conversation phase classifier. [USER_INPUT] is the raw text of the latest user message. [CONTEXT_ELEMENTS] is the most critical part: it should be a pre-formatted list of objects, each with an element_id and the content to be scored. The output schema is designed for direct parsing; your application should validate that every input element_id is present in the output and that the decay_score is an integer within the 1-5 range. For high-stakes applications, log the rationale field for human auditability.

This prompt is designed to be called as a background evaluation step, not as a synchronous user-facing interaction. The primary integration point is inside a context assembly pipeline that runs before each model call. After new user input arrives, the harness should collect all current context elements (prior turns, retrieved documents, tool outputs, session summaries) and pass them to this prompt to receive decay scores and retention recommendations. The output is consumed programmatically—users never see the raw scores. Instead, the scores drive automated decisions about which context to keep, compress, or drop before assembling the final prompt payload for the primary task model.

Validation and Schema Enforcement: The prompt output must be validated against a strict schema before any context is modified. Require a JSON object with a decay_scores array where each element contains context_id (string), decay_score (float 0-1), recommended_retention (enum: keep_full, summarize, drop), and rationale (string). Implement a post-processing validator that checks: (1) every input context element has exactly one corresponding output entry, (2) all scores are within 0-1, (3) retention recommendations are valid enum values, and (4) the total token count of keep_full items does not exceed your budgeted allocation. If validation fails, retry once with the validation errors appended to the prompt as feedback. If the retry also fails, fall back to a conservative policy: keep the last N turns and the most recent retrieval results, drop everything else, and log the failure for review.

Model Selection and Latency Budget: This scoring task benefits from models with strong instruction-following and structured output capabilities. Use a fast, cost-efficient model (e.g., GPT-4o-mini, Claude 3.5 Haiku, or a fine-tuned small model) rather than your primary high-capability model. The decay scoring step adds latency to every turn, so target sub-500ms response times. If your primary model call already has a 2-second budget, allocate no more than 500ms for decay scoring. Implement a timeout: if the scoring call exceeds 800ms, abort it and apply the default retention policy. Log timeouts separately from validation failures—they indicate a need for model downgrade or prompt compression, not a logic error.

Integration with Context Budgeting: The decay scores should feed directly into your context window budget allocator. A typical pattern: set a decay_threshold (e.g., 0.7) above which items are candidates for summarization, and a drop_threshold (e.g., 0.9) above which items are dropped entirely. Items scoring below 0.3 should be preserved in full. The summarization step itself can use a separate summarizer prompt or a lightweight model call, but only trigger it for items marked summarize—do not summarize everything preemptively. Store the decay scores alongside each context element in your session state so that subsequent turns can track decay trajectories and detect acceleration (e.g., a score jumping from 0.4 to 0.8 in one turn may signal a topic shift that requires broader context reset).

Observability and Evaluation: Log every decay scoring call with: input context count, output score distribution, validation pass/fail, latency, and the final retention decisions. Build a dashboard panel showing the average decay score per turn and the percentage of context dropped versus retained. For offline evaluation, create a golden dataset of conversation transcripts with human-annotated relevance judgments at each turn. Measure whether your decay scores correlate with human judgments and, more importantly, whether downstream task performance (answer quality, retrieval precision) degrades when context is pruned according to your scores versus a full-context baseline. If pruning causes a statistically significant quality drop, raise your retention thresholds or increase your context budget before shipping the optimization.

Failure Modes to Monitor: The most common production failure is premature context dropping—the scorer assigns high decay scores to turns that are actually relevant to a later user question, causing the primary model to answer without necessary history. Mitigate this by implementing a safety net: never drop the most recent 3 turns regardless of score, and never drop turns containing explicit user commitments or unresolved questions (detect these with a separate classifier if needed). A second failure mode is score inflation on long sessions, where the model assigns uniformly high decay scores to everything beyond a certain age, effectively reverting to a recency-only policy. Monitor for this by tracking the correlation between turn age and decay score—if the correlation exceeds 0.9, your prompt is not discriminating based on content relevance and needs adjustment. A third failure mode is context ID mismatch, where the model hallucinates context IDs that don't match your input. Your validator must catch this and trigger a retry or fallback.