Elasticsearch Filter DSL Generation Prompt Template

This prompt is for Elasticsearch operators and platform engineers building natural language search interfaces on top of Elasticsearch. It translates a user's unstructured question into a syntactically valid Elasticsearch bool query clause. Use it when you need to bridge the gap between a free-text search bar and a structured filter DSL that vector search alone cannot satisfy. The prompt assumes you have a defined field mapping and that the model receives the available field schema as context. It is not a replacement for a full query engine, but a focused DSL generation step inside a larger retrieval pipeline.

The ideal user is a search engineer who knows their Elasticsearch field mappings, data types, and query semantics but wants to offload the repetitive work of translating natural language constraints into must, filter, should, and must_not clauses. The prompt requires you to supply a [FIELD_SCHEMA] that describes available fields, their types, and any nested relationships. Without this context, the model will hallucinate field names or produce queries that fail against your index. You should also provide [EXAMPLES] of correct query translations to anchor the model's output style and operator choices.

Do not use this prompt when the user's question requires full-text relevance scoring, vector similarity search, or aggregation logic. This prompt generates filter clauses only—it does not produce match, multi_match, knn, or aggs sections. It is also unsuitable for queries that require cross-index joins, runtime field computation, or script-based filtering. If your retrieval pipeline needs a complete Elasticsearch query body including scoring and pagination, combine this prompt's output with a separate query construction step. For high-risk domains where a bad filter could expose restricted data, always validate the generated DSL against your field schema before execution and consider a human review step for queries that include must_not clauses or nested path traversals.

This template is the core instruction set you will copy into your prompt assembly layer. It is designed to accept a user's natural language question, a mapping of available index fields, and a set of constraints, and return a syntactically correct Elasticsearch Query DSL bool query. The prompt uses square-bracket placeholders for all dynamic inputs, ensuring you can wire it directly into a Python, TypeScript, or API-based prompt builder without manual string surgery.

textCopy
You are an expert Elasticsearch Query DSL generator. Your task is to translate a user's natural language question into a valid Elasticsearch `bool` query object.

## INPUT
User Question: [USER_QUESTION]

## CONTEXT
Available Index Fields and Their Types:
[INDEX_MAPPING]

## CONSTRAINTS
- Output must be a single, valid JSON object representing a `bool` query.
- Use only the fields provided in the index mapping. Do not invent fields.
- For full-text search, use a `match` query inside the `must` clause.
- For exact value filters (e.g., status, category, tags of type 'keyword'), use a `term` or `terms` query inside the `filter` clause.
- For numerical or date ranges, use a `range` query inside the `filter` clause.
- If the user's question implies excluding something, use a `must_not` clause.
- If the question contains multiple constraints that should all match, place them in `must` or `filter` as appropriate.
- If the question contains alternatives, use a `should` clause with `minimum_should_match` set appropriately.
- If the question is ambiguous or cannot be translated with the available fields, return a JSON object with a single key "error" and a descriptive message.
- Do not include any explanatory text outside the JSON object.

## OUTPUT_SCHEMA
{
  "query": {
    "bool": {
      "must": [],
      "filter": [],
      "should": [],
      "must_not": []
    }
  }
}

## EXAMPLES
User Question: "Find active users in the engineering department"
Index Mapping: { "status": { "type": "keyword" }, "department": { "type": "keyword" } }
Output:
{
  "query": {
    "bool": {
      "filter": [
        { "term": { "status": "active" } },
        { "term": { "department": "engineering" } }
      ]
    }
  }
}

User Question: "Show me documents about machine learning or AI from the last 30 days"
Index Mapping: { "content": { "type": "text" }, "created_date": { "type": "date" } }
Output:
{
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "should": [
              { "match": { "content": "machine learning" } },
              { "match": { "content": "AI" } }
            ],
            "minimum_should_match": 1
          }
        }
      ],
      "filter": [
        { "range": { "created_date": { "gte": "now-30d/d" } } }
      ]
    }
  }
}

To adapt this template, replace each square-bracket placeholder with data from your application context. [USER_QUESTION] receives the raw user input. [INDEX_MAPPING] should be a JSON-like summary of your index's field names and types, which is critical for preventing hallucinated fields. For production use, always validate the generated JSON against your actual index mapping and a DSL syntax checker before executing the query. If the model returns an error object, your harness should log it and fall back to a simpler multi_match query or surface the ambiguity to the user.

The Elasticsearch Filter DSL prompt is not a standalone chatbot. It is a component inside a retrieval pipeline that must produce syntactically valid, field-mapped, and safe-to-execute filter clauses. The application layer is responsible for receiving the model's output, validating it against the target index mapping, and deciding whether to execute, repair, or reject the generated DSL. Treat the LLM output as a draft that requires structural verification before it touches your Elasticsearch cluster.

Start by defining a strict output contract. The prompt template should include an [OUTPUT_SCHEMA] placeholder that specifies the expected JSON structure, such as a bool query object with must, filter, should, and must_not clauses. After the model responds, run a multi-stage validation harness: (1) JSON syntax parse with a library like json.loads() and reject on parse failure; (2) schema validation against a JSON Schema or Pydantic model that enforces the Elasticsearch DSL structure, including allowed clause types and operator values; (3) field mapping validation that checks every field name against the actual index mapping retrieved from GET /{index}/_mapping, rejecting unknown fields or suggesting corrections; (4) type coercion checks that verify string values match keyword fields, numeric values match integer/long/float fields, and date strings match the index's date format. If any stage fails, log the failure with the original query, the raw model output, and the specific validation error for debugging.

For retry logic, implement a bounded recovery loop. If validation fails, construct a retry prompt that includes the original user query, the failed DSL output, and the specific validation error message. Use a [RETRY_CONTEXT] placeholder in the prompt template to inject this feedback. Limit retries to a maximum of 2 attempts to avoid infinite loops and escalating latency. If the model cannot produce valid DSL after retries, fall back to a safe default: either execute the query without filters, return an empty filter clause ({"match_all": {}}), or surface the ambiguity to the user with a clarification request. Never execute partially validated or repaired DSL without explicit approval in high-risk contexts such as access-controlled indices or compliance-sensitive data.

Model choice matters for this task. Prefer models with strong JSON generation and instruction-following capabilities, such as GPT-4o, Claude 3.5 Sonnet, or fine-tuned open-weight models that have been trained on structured output tasks. Enable structured output modes or JSON mode where available to reduce syntax errors. For production observability, log every prompt invocation with: the user query, the generated DSL, validation results, retry count, latency, and whether the filter was executed or rejected. This trace data is essential for debugging field mapping drift, model regression after version upgrades, and query patterns that consistently produce invalid DSL.

Finally, implement a human review path for high-risk or novel filter patterns. If the generated DSL includes must_not clauses, nested queries, or script-based filters, consider flagging these for review before execution, especially in multi-tenant systems where a malformed exclusion filter could hide data the user is authorized to see. Pair this prompt with the Structured JSON Filter Output Prompt Template for schema enforcement and the Metadata Constraint Parsing Prompt Template for field-level extraction before DSL assembly. The goal is a defense-in-depth harness where the LLM proposes, the application validates, and the system executes only what passes every structural and semantic check.