Intent-Driven Metadata Filter Extraction Prompt Template

The raw natural language question from the end user that needs metadata filters extracted.

Show me security incidents from production logs in the last 48 hours.

Must be a non-empty string. Check for minimum length of 3 characters. Reject or escalate if the query is entirely unrelated to the corpus domain.

The classified intent label for the user query, used to guide which metadata dimensions are relevant.

troubleshooting

Must match one of the predefined intent labels in your taxonomy. Validate against the allowed enum list. If intent confidence is below threshold, set to 'ambiguous' and trigger clarification workflow.

A JSON schema or structured description of the available metadata fields, their types, and allowed values in the search backend.

{ "source_type": ["incident_report", "runbook", "alert"], "severity": ["low", "medium", "high", "critical"], "timestamp": "datetime" }

Must be valid JSON. Validate that all field names match the actual search index mapping. Check that enum values are current and complete. Schema drift detection should flag missing fields.

A mapping from each intent label to the subset of metadata fields that are relevant for that intent, preventing the model from filtering on irrelevant dimensions.

{ "troubleshooting": ["source_type", "timestamp", "service_name"], "factual_lookup": ["source_type", "authority_level"] }

Must be valid JSON. Every key must match an allowed intent label. Every value array must contain only fields present in [CORPUS_SCHEMA]. Test that no intent maps to an empty array.

The current datetime in ISO 8601 format, used to resolve relative time expressions like 'last 48 hours' into absolute date ranges.

2025-03-21T14:30:00Z

Must be a valid ISO 8601 datetime string. Validate that the value is within a reasonable skew window of the actual current time. If null or missing, relative time expressions cannot be resolved and the prompt should return an error.

The expected JSON schema for the extracted filter object, including required fields, types, and constraints.

{ "type": "object", "properties": { "filters": { "type": "array" } }, "required": ["filters"] }

Must be valid JSON Schema. Validate that the schema is compatible with your search backend's filter DSL. Test that the model can produce output conforming to this schema with a representative sample of queries.

An integer cap on the number of filter clauses the model can return, preventing overly restrictive queries that return zero results.

5

Must be a positive integer. Validate that the value is reasonable for your search backend's performance characteristics. If set too high, retrieval latency degrades. If set too low, precision suffers. Recommended range: 3-7.

A float between 0.0 and 1.0. If the model's internal confidence in an extracted filter is below this threshold, the filter should be omitted or flagged for human review.

0.7

Must be a float in range [0.0, 1.0]. Validate that the threshold is calibrated against your eval dataset. Too low includes noisy filters. Too high drops useful constraints. Log all filters dropped due to confidence for observability.

Array of objects

Must be a JSON array. If no filters are extracted, return an empty array [].

String

Must match a field from the provided [AVAILABLE_METADATA_FIELDS] list exactly. Case-sensitive check required.

String (enum)

Must be one of: 'equals', 'not_equals', 'in', 'not_in', 'gte', 'lte', 'range'. Validate against this closed enum.

String, Number, or Array

Type must be compatible with the operator. 'in' and 'not_in' require an array. 'range' requires a two-element array [min, max]. 'gte'/'lte' require a number or ISO 8601 date string.

String or null

The exact substring from [USER_QUERY] that justifies this filter. If the filter is purely implied by [DETECTED_INTENT] and not explicit in the query, set to null.

Object

Must contain a 'rationale' string explaining how the extracted filters serve the [DETECTED_INTENT]. Required for auditability.

Array of strings or null

A list of metadata partitions or categories that were explicitly considered but rejected as irrelevant to the intent. Set to null if none were rejected.

Number

A float between 0.0 and 1.0. Represents the model's confidence in the filter set. A value below [CONFIDENCE_THRESHOLD] should trigger a human review or clarification step.

Extracted filters match the detected intent's scope requirements (e.g., factual lookup restricts to authoritative sources, summarization broadens to multiple facets)

Filters contradict the intent (e.g., narrow date range on a summarization request, missing authority constraint on a compliance query)

Run 20 intent-query pairs through the prompt; have a domain expert label whether filters are appropriate for each intent class

All explicit entities, date ranges, categories, and source types mentioned in [USER_QUERY] appear in the output filter object with correct mapping

Missing entity that was stated in query; wrong date range boundary; entity mapped to wrong filter field

Parse [USER_QUERY] for known entities and compare against output filter fields using exact-match and boundary comparison scripts

Prompt correctly infers filters implied by intent when user does not state them (e.g., 'recent' maps to a concrete date range based on [CURRENT_DATE])

Implied constraint missing entirely; vague term left unresolved; over-aggressive inference adds filters the intent does not support

Test with queries containing only vague temporal or authority terms; verify output contains concrete filter values and no hallucinated constraints

Output filter object excludes partitions that the intent makes irrelevant (e.g., no marketing content for a legal query, no deprecated API versions for a current how-to)

Filter object is empty when partitions should be excluded; filter includes broad catch-all categories that defeat partitioning

Check that filter object has at least one exclusion constraint when intent implies domain or source-type boundaries; verify excluded partitions are absent from retrieval results in integration test

Output is valid JSON matching the [OUTPUT_SCHEMA] exactly: all required fields present, no extra fields, correct types, enum values within allowed set

JSON parse failure; missing required field; field has wrong type; enum value outside allowed list; extra undeclared field present

Validate output against JSON Schema using a programmatic validator; reject any output that fails schema validation

When intent confidence is below [CONFIDENCE_THRESHOLD], output includes a low-confidence flag and omits filters that would be speculative

Low-confidence output still includes aggressive filters; high-confidence output missing filters that should be present; confidence flag missing or always true

Feed queries with known ambiguous intent through the prompt; verify that confidence flag is false and filter object is minimal or empty for ambiguous cases

Output filter object can be directly translated into the target search backend's filter syntax without manual transformation

Filter uses operators or field names not supported by the backend; nested structure exceeds backend depth limits; date format incompatible with backend

Write an adapter that converts output filter object to backend query DSL; run 50 outputs through adapter and verify zero translation errors

No filter values are invented that cannot be traced to [USER_QUERY], [INTENT], or [CURRENT_DATE]

Filter contains a source type, category, or entity not mentioned or implied by any input; date range has no grounding in query or current date

For each output, trace every filter value back to an input field; flag any value without a traceable source as a hallucination failure