Document Publication Date Filter Generation Prompt

This prompt is designed for RAG developers and search engineers building retrieval systems over versioned document corpora where publication date is a critical relevance signal. Use it when your retrieval pipeline needs to convert natural language queries like 'show me research published after 2022' or 'find the latest version of the policy from last quarter' into structured filter clauses that your document store can execute. The prompt handles the extraction of publication date constraints, distinguishes between date-of-record and date-of-publication when both exist in your metadata schema, and generates filter syntax ready for injection into vector database queries, Elasticsearch bool queries, or SQL WHERE clauses.

The ideal user is a platform engineer or RAG architect who has already normalized relative temporal expressions to concrete date boundaries. This prompt sits downstream of temporal normalization steps like the Relative Date Normalization Prompt Template or the Temporal Expression to ISO 8601 Conversion Prompt. It assumes the incoming query contains explicit or already-resolved date references and focuses entirely on mapping those references to the correct metadata field and filter syntax. If your query still contains expressions like 'last quarter' or 'recently published,' you must resolve those before this prompt receives the input. The prompt also requires a defined metadata schema so it knows whether to target a publication_date field, a date_of_record field, or both.

Do not use this prompt for general temporal expression extraction or for resolving relative date expressions. Those tasks belong to dedicated normalization prompts earlier in the pipeline. Do not use it when your document store lacks publication date metadata or when the query's temporal constraints are too vague to map to concrete boundaries—in those cases, use the Temporal Vagueness Detection and Clarification Prompt first. This prompt also should not be used for temporal relevance scoring or recency boosting during ranking; that work belongs to the Temporal Relevance Scoring Prompt for Hybrid Search. When in doubt, trace your pipeline: if the query still contains unresolved relative time language, back up to normalization; if the query has concrete dates but you need to decide which metadata field to filter, this is the right tool.

This prompt template converts natural language queries about document publication dates into structured filter clauses that can be executed against document stores, search indexes, or vector databases. It handles explicit date references ('published in 2023'), relative expressions ('last month'), version-aware constraints ('latest version'), and ambiguous recency terms ('recent papers'). The output is a machine-readable filter specification that includes operator semantics, date boundaries, and versioning logic.

textCopy
SYSTEM: You are a retrieval filter generator for a document store. Your job is to extract publication date constraints from user queries and produce structured filter clauses. You must handle explicit dates, relative time expressions, version references, and recency terms. Always anchor relative expressions to the provided reference date. When the query is ambiguous about publication date vs. date-of-record, default to publication date unless the query explicitly references ingestion or indexing time.

INPUT:
- User Query: [USER_QUERY]
- Reference Date (ISO 8601): [REFERENCE_DATE]
- Document Store Schema Fields Available: [SCHEMA_FIELDS]
- Versioning Rules (if applicable): [VERSIONING_RULES]

OUTPUT_SCHEMA:
{
  "filters": [
    {
      "field": "string (e.g., publication_date, version, status)",
      "operator": "string (eq, gte, lte, gt, lt, between, in, is_not_null)",
      "value": "string or [string, string] for between",
      "confidence": "float 0.0-1.0",
      "rationale": "string explaining extraction logic"
    }
  ],
  "original_query": "string",
  "temporal_expressions_found": ["string"],
  "ambiguities": ["string"] | null,
  "requires_clarification": boolean,
  "clarification_question": "string" | null,
  "assumptions_made": ["string"]
}

CONSTRAINTS:
- Resolve all relative dates to absolute ISO 8601 dates using [REFERENCE_DATE] as the anchor.
- For 'latest', 'most recent', or 'current', apply versioning rules from [VERSIONING_RULES] to determine whether this means the newest publication_date or the highest version number.
- For 'recent' without a specific window, use domain-appropriate defaults: 90 days for fast-moving fields, 365 days for general academic content, 730 days for slow-moving domains. State your assumption.
- Distinguish between preprints, published versions, and updates when the schema supports it.
- If the query references 'date of record' or 'as of' explicitly, filter on date-of-record fields instead of publication_date.
- If temporal constraints are contradictory or impossible (e.g., 'before 2020 and after 2023'), flag in ambiguities and set requires_clarification to true.
- If no temporal constraint is detected, return an empty filters array with requires_clarification set to false.
- Confidence should reflect how clearly the temporal constraint maps to the available schema fields.

EXAMPLES:

Example 1:
User Query: 'Show me papers about transformers published in 2023'
Reference Date: 2025-01-15
Schema Fields: ["publication_date", "title", "status"]
Versioning Rules: status values are ["preprint", "published", "updated"]

Output:
{
  "filters": [
    {
      "field": "publication_date",
      "operator": "between",
      "value": ["2023-01-01", "2023-12-31"],
      "confidence": 0.98,
      "rationale": "Explicit year reference mapped to full calendar year range"
    }
  ],
  "original_query": "Show me papers about transformers published in 2023",
  "temporal_expressions_found": ["published in 2023"],
  "ambiguities": null,
  "requires_clarification": false,
  "clarification_question": null,
  "assumptions_made": ["Calendar year boundaries used for '2023'"]
}

Example 2:
User Query: 'Latest version of the COVID-19 treatment guidelines'
Reference Date: 2025-01-15
Schema Fields: ["publication_date", "version", "title", "status"]
Versioning Rules: version is integer, higher number = more recent version; status indicates "current" or "superseded"

Output:
{
  "filters": [
    {
      "field": "status",
      "operator": "eq",
      "value": "current",
      "confidence": 0.85,
      "rationale": "'Latest' interpreted as currently active version per versioning rules"
    }
  ],
  "original_query": "Latest version of the COVID-19 treatment guidelines",
  "temporal_expressions_found": ["latest version"],
  "ambiguities": null,
  "requires_clarification": false,
  "clarification_question": null,
  "assumptions_made": ["'Latest' means highest version number with status=current, not most recent publication_date"]
}

Example 3:
User Query: 'Recent studies on mRNA vaccine efficacy'
Reference Date: 2025-01-15
Schema Fields: ["publication_date", "title", "status"]
Versioning Rules: status values are ["preprint", "published", "updated"]

Output:
{
  "filters": [
    {
      "field": "publication_date",
      "operator": "gte",
      "value": "2024-10-17",
      "confidence": 0.6,
      "rationale": "'Recent' interpreted as last 90 days for biomedical domain"
    }
  ],
  "original_query": "Recent studies on mRNA vaccine efficacy",
  "temporal_expressions_found": ["recent"],
  "ambiguities": ["'Recent' is domain-dependent; 90-day window assumed for biomedical research"],
  "requires_clarification": false,
  "clarification_question": null,
  "assumptions_made": ["90-day recency window applied for biomedical domain", "Filtering on publication_date, not date-of-record"]
}

To adapt this template, replace each square-bracket placeholder before sending it to the model. The [SCHEMA_FIELDS] placeholder should list the exact field names available in your document store so the model maps temporal constraints to real columns. The [VERSIONING_RULES] placeholder is critical for corpora with preprints, updates, or versioned documents—without it, the model cannot distinguish between 'most recent by publication date' and 'most recent by version number.' The [REFERENCE_DATE] must be in ISO 8601 format and should reflect the current date or the session's temporal anchor. For production use, validate the output against your actual schema before executing the filter, and log any ambiguities or assumptions_made fields for auditability. When requires_clarification is true, route to a human or a clarification prompt rather than executing a potentially wrong filter.

This prompt is designed to sit in the pre-retrieval stage of a RAG pipeline, immediately after query intake and before any vector or keyword search is executed. Its sole job is to extract publication date constraints from a natural language query and produce a structured filter clause that your document store can enforce. The prompt should be called synchronously for every user query that touches a versioned or time-sensitive corpus. Do not batch this prompt or reuse its output across different queries—each query carries its own temporal intent, and caching a filter from 'last month' will produce stale results within hours.

Wiring the prompt into your application requires a thin orchestration layer. First, construct the prompt payload by injecting the user's raw query into the [USER_QUERY] placeholder and the current UTC timestamp into [REFERENCE_TIMESTAMP]. If your system maintains a session-level temporal anchor (e.g., the start of the conversation), pass that as [SESSION_ANCHOR]; otherwise, default to the current time. The [DOCUMENT_STORE_SCHEMA] placeholder must contain the exact metadata field names, types, and filter syntax your database accepts—for example, publication_date as an ISO 8601 string field with gte/lte operators in Elasticsearch, or a Pinecone metadata filter with $gte/$lte. The [OUTPUT_SCHEMA] should request a JSON object with filter_clause, date_range, confidence, and assumptions fields. After receiving the model response, validate the output before it touches your search index: confirm the filter_clause parses correctly against your store's query language, verify the date_range boundaries are logically consistent (start ≤ end, no future-only ranges unless explicitly requested), and check that confidence meets your threshold. If confidence is below 0.7, route the query to a clarification prompt or apply a conservative default window rather than executing a potentially wrong filter.

Model choice and latency considerations matter here. This is a structured extraction task with low reasoning depth, so a fast, cost-effective model like GPT-4o-mini, Claude Haiku, or a fine-tuned small open-weight model is usually sufficient. Latency is additive to your retrieval pipeline, so target sub-500ms response times. Implement a retry strategy for malformed outputs: if the JSON parse fails or the filter clause doesn't validate against your document store's syntax, retry once with the validation error message appended to the prompt as [PREVIOUS_ERROR]. If the second attempt also fails, fall back to a date-range-free retrieval and log the failure for prompt debugging. Logging should capture the original query, the generated filter clause, the confidence score, the model used, and whether validation passed or triggered a retry. This trace data is essential for tuning the prompt, identifying query patterns that produce low-confidence outputs, and debugging retrieval failures where date filters silently excluded relevant documents.

Human review is not required per query for this prompt under normal operation, but you should implement a periodic audit of low-confidence outputs and validation failures. If your corpus includes documents with complex versioning—preprints, updated versions, errata, or documents where date_of_record differs from date_of_publication—extend the [DOCUMENT_STORE_SCHEMA] to include these fields and add explicit handling instructions to the prompt's [CONSTRAINTS] section. For high-stakes domains like legal or financial compliance, consider adding a pre-execution guard that blocks filters with confidence below 0.5 and escalates to a human reviewer or a clarification dialogue. The next step after implementing this harness is to build a regression test suite using the eval criteria described in the Testing and QA section: run the prompt against queries with known date ranges, verify the generated filter clauses match expected boundaries, and measure how often low-confidence outputs reach your search index.