Stale API Response Handling Prompt

This prompt is for agent and copilot builders whose systems cache or carry forward API responses across conversation turns. Its job is to evaluate a previously fetched API response against the current user intent and conversation state to determine if the data is still valid. The ideal user is a product engineer or AI operator who needs a structured, programmatic freshness assessment—not a generic summary—that can be wired directly into a tool-use loop. You need this when your agent relies on time-sensitive data from external services (pricing, inventory, status, weather, flight times) and a stale response would produce a confident wrong answer.

Use this prompt when you have a concrete API response payload, a timestamp for when it was fetched, and the current conversation context that might invalidate it. The prompt expects you to provide the original API response, its age, the user's latest query, and any domain-specific volatility rules. It is not designed for detecting staleness in retrieved documents, session summaries, or model-generated text—use the sibling prompts for those cases. It also does not decide whether to re-call the API; it produces a structured assessment that your application code should act on, including recommended re-call parameters and a fallback response if the API is unavailable.

Do not use this prompt when the API response is static reference data that does not expire, or when your application already has a deterministic cache-invalidation strategy based on HTTP headers or TTLs. The prompt adds latency and cost, so reserve it for cases where staleness depends on semantic context—for example, a flight status fetched 10 minutes ago might still be valid for a 'check my flight' query but stale for a 'has my flight been delayed in the last 5 minutes?' query. Before deploying, test against a labeled dataset of API responses with known staleness states, and always pair this prompt with a validation step that compares the freshness assessment against ground truth. If the API response contains regulated or safety-critical data, require human review before acting on the staleness decision.

This prompt template is the core engine for detecting stale API data in agent and copilot workflows. It takes a previously cached API response, the original query that produced it, and the current conversation context, then outputs a structured freshness assessment. The template is designed to be dropped into a larger application harness that manages API call history, caching logic, and tool execution. Use square-bracket placeholders to inject the specific data, context, and constraints relevant to your use case before sending the prompt to the model.

textCopy
You are an API response freshness evaluator for an AI assistant. Your job is to determine whether a cached API response is still valid given the current conversation context, or whether the data should be re-fetched.

## INPUTS

### Cached API Response
- Endpoint: [API_ENDPOINT]
- Called at: [CALL_TIMESTAMP]
- Parameters: [ORIGINAL_PARAMETERS]
- Response body: [CACHED_RESPONSE_BODY]
- Cache TTL (seconds): [CACHE_TTL]

### Current Conversation Context
- Current user query: [CURRENT_USER_QUERY]
- Recent conversation turns (last 5): [RECENT_TURNS]
- Current session state: [SESSION_STATE]
- Time elapsed since API call (seconds): [ELAPSED_SECONDS]

### Data Volatility Profile
- Data type: [DATA_TYPE] (e.g., stock price, weather, user profile, inventory count)
- Expected update frequency: [UPDATE_FREQUENCY]
- Known volatility indicators: [VOLATILITY_INDICATORS]

## OUTPUT SCHEMA

Return a JSON object with exactly these fields:

{
  "freshness_status": "fresh" | "stale" | "uncertain" | "expired",
  "confidence": 0.0-1.0,
  "stale_factors": ["list of specific reasons the data may be stale"],
  "recommended_action": "use_cached" | "re_fetch" | "ask_user" | "use_with_caveat",
  "re_fetch_parameters": {
    "endpoint": "string or null",
    "parameters": {},
    "reason": "string explaining what changed"
  },
  "fallback_response": "string to use if re-fetch fails",
  "user_facing_caveat": "string explaining staleness risk to user, or null if fresh",
  "cache_update_recommendation": "extend" | "invalidate" | "shorten_ttl" | "no_change"
}

## CONSTRAINTS

- If the data type is highly volatile (e.g., live stock prices) and elapsed time exceeds 30 seconds, mark as stale.
- If the current user query references a time or state that postdates the API call timestamp, mark as stale.
- If the conversation context shows the user has contradicted or updated information in the cached response, mark as stale.
- If the cache TTL has expired, mark as expired regardless of content.
- If you are uncertain, prefer "ask_user" over silently using potentially stale data.
- Never fabricate re-fetch parameters. If you cannot determine what to re-fetch, set re_fetch_parameters to null and recommend "ask_user".
- The user_facing_caveat must be concise, honest, and actionable. Do not alarm the user unnecessarily.

## EXAMPLES

[EXAMPLES]

## RISK LEVEL

[RISK_LEVEL]

Evaluate the cached API response against the current context and return only the JSON object.

To adapt this template for your system, start by replacing the [DATA_TYPE], [UPDATE_FREQUENCY], and [VOLATILITY_INDICATORS] placeholders with domain-specific values that match the APIs your agents call. For high-risk domains such as financial transactions or healthcare data, set [RISK_LEVEL] to "high" and add constraints that require human review before acting on stale data. The [EXAMPLES] placeholder should be populated with 2-4 few-shot examples showing correct freshness assessments for your specific API types—include at least one example where the data is fresh, one where it is stale, and one edge case where the model should express uncertainty. If your application already has a defined output schema, replace the JSON structure with your own, but preserve the core decision fields: freshness status, confidence, recommended action, and user-facing caveat.

This prompt is designed to sit inside an agent's tool-use loop, not as a standalone chat interaction. The typical wiring point is immediately before the agent acts on a cached API response or after a user turn that references data from a prior tool call. The harness should call this prompt with the original API response payload, the timestamp of that response, the current conversation context, and any known volatility metadata for the endpoint. The output is a structured freshness assessment that your application code reads to decide whether to re-call the API, use the cached value with a caveat, or fall back to a degraded response.

The implementation should treat the prompt's output as a machine-readable decision, not just text. Parse the freshness_status field (valid, stale, uncertain) and the recommended_action field (use_as_is, re_call, fallback) into your agent's control flow. For re_call, extract the re_call_parameters object and pass it to your API client—these should include the endpoint, query parameters, and any updated filters derived from the new conversation context. For fallback, surface the fallback_response to the user and log the staleness event. Always validate the output schema before acting: if the JSON is malformed or missing required fields, treat it as freshness_status: uncertain and either re-call the API or escalate to a human reviewer depending on the risk level of the decision.

Retry logic should be layered. If the freshness assessment recommends a re-call but the API is unavailable, implement an exponential backoff with a maximum of two retries before falling back to the cached value with a staleness caveat appended to the user-facing response. Log every staleness detection event with the original response timestamp, the freshness score, the action taken, and whether the re-call returned different data. These logs become your evaluation dataset for tuning the prompt's staleness thresholds. For high-risk domains—financial data, healthcare information, safety-critical system state—add a human approval gate when freshness_status is uncertain and the confidence score is below 0.7. The approval UI should display the cached response, the freshness reasoning, and a one-click option to re-call the API or approve the fallback.

Model choice matters for this prompt. Use a model with strong structured output capabilities and low latency, since this check runs on the critical path of every tool-use turn. GPT-4o or Claude 3.5 Sonnet are appropriate for production; avoid smaller models that may hallucinate freshness assessments or fail to follow the output schema under time pressure. If latency is a concern, consider caching the prompt's system instructions as a prompt prefix and batching staleness checks for multiple cached responses in a single call. Do not use this prompt for endpoints with sub-second cache invalidation—at that scale, the freshness check costs more than simply re-calling the API. The prompt is best suited for data with volatility measured in minutes to hours, where the cost of an unnecessary API call exceeds the cost of the staleness check.