Age of Information Filter Generation Prompt for RAG Freshness

This prompt is for RAG operators and search engineers who need to enforce information freshness constraints based on query intent. It generates staleness thresholds and recency filters so retrieval excludes outdated documents when the query implies a need for current information. Use this prompt when your corpus contains time-sensitive documents and user queries carry implicit or explicit recency expectations. This prompt belongs in the pre-retrieval stage of a RAG pipeline, after query analysis but before the search index is called. It does not handle date range extraction from explicit temporal expressions; pair it with a temporal normalization prompt when queries contain specific date references like 'last week' or 'Q3 2024'.

The ideal user is a search engineer or RAG developer who has already classified the user's intent and now needs to translate that intent into concrete freshness parameters. The prompt requires a query intent label as input—such as breaking_news, regulatory_compliance, historical_research, or product_availability—and produces a structured staleness policy with a maximum document age, a recency weight for hybrid ranking, and a fallback behavior when no documents meet the threshold. For example, a query classified as breaking_news might produce a 24-hour staleness window with a hard filter, while historical_research might produce no staleness constraint at all. The prompt also handles ambiguous intents by generating multiple candidate policies with confidence scores, allowing the retrieval orchestrator to choose or merge them.

Do not use this prompt when the query contains explicit date ranges or relative time expressions that require normalization first. Those should be resolved by a temporal normalization prompt before this freshness filter is applied. Also avoid this prompt when your corpus has no reliable publication or modification timestamps, or when freshness is irrelevant to the task—such as for literary analysis, static reference material, or code documentation where version tags matter more than recency. Finally, this prompt is not a substitute for document-level freshness metadata; if your index lacks timestamp fields, you must add them before this prompt can be useful. Next, review the prompt template below and adapt the intent taxonomy and staleness windows to match your domain's information lifecycle.

This prompt template generates a structured freshness filter from a natural language query. It analyzes the query's temporal intent, determines an appropriate staleness threshold (e.g., 'documents older than 6 months are stale'), and produces a metadata filter clause suitable for vector databases, search indexes, or retrieval APIs. The output includes the reasoning behind the threshold so operators can audit the decision before it affects retrieval results.

textCopy
You are a retrieval freshness analyst. Given a user query, determine whether the query implies a need for recent information. If it does, produce a staleness threshold and a metadata filter clause that excludes documents older than that threshold. If the query does not imply a freshness requirement, state that no filter is needed.

## INPUT
Query: [USER_QUERY]
Current date (ISO 8601): [REFERENCE_DATE]
Domain context (optional): [DOMAIN_CONTEXT]
Available metadata fields for filtering: [METADATA_FIELDS]
Filter syntax format: [FILTER_SYNTAX]

## TASK
1. Classify the query's freshness intent as one of: "high_recency", "moderate_recency", "no_recency_requirement", or "ambiguous".
2. If "high_recency" or "moderate_recency", determine a staleness threshold expressed as a duration (e.g., "7 days", "3 months", "1 year").
3. Generate a metadata filter clause using the provided [FILTER_SYNTAX] that excludes documents where [METADATA_FIELDS] indicates a date older than the threshold.
4. Provide a brief justification for the chosen threshold.

## CONSTRAINTS
- Do not invent metadata fields. Use only the fields listed in [METADATA_FIELDS].
- The filter clause must be syntactically valid for [FILTER_SYNTAX].
- If the query is ambiguous, default to a moderate threshold and note the assumption.
- If no recency requirement exists, explicitly state "no filter required" rather than applying a default.
- For queries about historical events, archival content, or explicitly dated references, do not apply a freshness filter.

## OUTPUT SCHEMA
Return a JSON object with these fields:
{
  "freshness_intent": "high_recency | moderate_recency | no_recency_requirement | ambiguous",
  "staleness_threshold": "duration string or null",
  "filter_clause": "filter expression string or null",
  "justification": "brief explanation of the threshold choice"
}

## EXAMPLES
Query: "What are the latest Kubernetes security patches?"
Current date: 2025-03-15
Domain: cloud infrastructure
Metadata fields: ["publication_date", "last_updated"]
Filter syntax: MongoDB query operators
Output:
{
  "freshness_intent": "high_recency",
  "staleness_threshold": "30 days",
  "filter_clause": "{ \"publication_date\": { \"$gte\": ISODate(\"2025-02-13\") } }",
  "justification": "Security patch information changes rapidly. A 30-day window captures recent patches while excluding outdated vulnerability data."
}

Query: "Explain the causes of World War I"
Current date: 2025-03-15
Domain: history
Metadata fields: ["publication_date"]
Filter syntax: MongoDB query operators
Output:
{
  "freshness_intent": "no_recency_requirement",
  "staleness_threshold": null,
  "filter_clause": null,
  "justification": "Historical analysis of World War I does not require recent publication. Archival and older scholarly sources remain relevant."
}

Adapt this template by replacing each square-bracket placeholder with your application's data. [USER_QUERY] receives the raw user input. [REFERENCE_DATE] should be the current date in ISO 8601 format, typically injected at runtime. [DOMAIN_CONTEXT] helps the model calibrate freshness expectations—'breaking news' implies hours, 'academic research' implies years. [METADATA_FIELDS] must match your actual document store schema exactly; mismatched field names produce broken filters. [FILTER_SYNTAX] tells the model which query language to generate, such as MongoDB query operators, Elasticsearch range filters, Pinecone metadata filters, or SQL WHERE clauses. Test the generated filter clause against your database before deploying to production—syntax errors here silently drop documents or return empty result sets.

This prompt is not a standalone chatbot interaction; it is a pre-retrieval filter generator that must sit inside a deterministic application harness. The harness is responsible for calling the model, parsing its structured output, validating the generated constraints against the target index's metadata schema, and applying the resulting filter clauses to the vector or hybrid search query. The model's job is to interpret the user's intent and the domain's freshness expectations. The harness's job is to enforce correctness, handle failures, and prevent impossible or contradictory filters from reaching the retrieval engine.

The implementation should follow a strict validate-then-execute pattern. After the model returns a JSON object with max_age_days, date_range, or staleness_threshold, the harness must run a validation layer before the filter touches any database query. This validation should check: (1) that all generated date ranges are logically consistent (start is before end, no future-dated boundaries unless explicitly allowed by the domain), (2) that the filter keys match the actual metadata fields in your document store (e.g., publication_date, effective_date, ingested_at), and (3) that the filter syntax is compatible with your vector database's query DSL (Pinecone, Weaviate, Qdrant, etc.). If validation fails, the harness should log the raw model output, the validation error, and the original query, then fall back to a safe default—typically a broader temporal window or no freshness filter at all—rather than returning zero results or crashing the pipeline. For high-stakes domains like financial filings or clinical guidelines, a failed validation should also trigger a human review flag or a clarification question back to the user.

Retry logic should be conservative and bounded. If the model returns malformed JSON or a filter that fails validation, a single retry with a more explicit error message in the prompt context is acceptable. For example, append: 'Your previous output failed validation: [VALIDATION_ERROR]. The metadata field must be publication_date and the range must be in ISO 8601 format. Generate a corrected filter.' Do not loop more than twice; after two failures, log the incident, fall back to a no-filter retrieval, and surface the issue in your observability dashboard. Model choice matters here: smaller, faster models (e.g., GPT-4o-mini, Claude Haiku) are sufficient for straightforward recency queries like 'last week' or 'recent news,' but ambiguous queries with domain-specific freshness expectations ('latest research on immunotherapy') may require a more capable model to reason about what 'latest' means in a biomedical context. Route accordingly based on query complexity or confidence scores.

Observability is non-negotiable. Every invocation of this prompt should emit structured logs containing: the original user query, the reference timestamp used as the temporal anchor, the raw model output, the validated filter that was applied, and the number of documents retrieved after filtering. This trace allows you to detect drift in the model's freshness judgments, identify queries where the filter was too aggressive (zero results), and tune the default staleness thresholds per domain. Set up a periodic eval that runs a golden set of queries with known expected date ranges and alerts if the model's outputs deviate beyond an acceptable tolerance. This prompt is a small but critical link in your retrieval chain; when it fails silently, your entire RAG system serves stale or irrelevant answers without anyone noticing until a user complains.