Cross-Lingual Keyword Extraction Prompt for Sparse Retrieval

This prompt is for infrastructure teams running hybrid search systems that combine dense vector retrieval with sparse keyword indexes like BM25. When a user submits a query in one language but your document corpus contains relevant material in another, direct translation of the full query often fails. Literal translation can distort domain-specific terminology, break multi-word expressions, or produce grammatically correct but unsearchable phrases. This prompt extracts the core conceptual keywords from the source query and translates them into the target index language while preserving their domain significance. The output is a set of weighted, de-duplicated keywords optimized for sparse retrieval, not a fluent sentence.

Use this prompt as a pre-retrieval step in a pipeline where the extracted keywords are passed directly to a BM25 analyzer or similar term-matching engine. It belongs before retrieval execution and after any query cleaning or language detection steps. The ideal user is a search engineer or RAG developer who already has a language detection module in place and knows the target language of the sparse index. You must provide the source query text, the detected source language, and the target index language as inputs. The prompt is not designed for generating dense vector embeddings, handling code-switched queries, or producing human-readable translations. If your pipeline requires a fluent translated sentence for display or a dense vector query, use a dedicated translation prompt or a cross-lingual dense embedding model instead.

Do not use this prompt when the query contains ambiguous entities that require disambiguation before keyword extraction. If the source query says 'apple' and your corpus spans both technology and agriculture domains, resolve the entity intent upstream before passing it here. Similarly, avoid this prompt when the target sparse index uses a controlled vocabulary or taxonomy that requires canonical term mapping rather than conceptual keyword extraction. In those cases, pair this prompt with a domain-specific terminology mapping step. For high-risk domains like legal or medical retrieval, always validate the extracted keywords against the target corpus's term frequency and flag low-frequency terms for human review before retrieval execution.

This prompt template is designed to be dropped directly into your application code. It instructs the model to act as a keyword extraction and translation engine, focusing on terms that carry high retrieval significance in the target domain. The core instruction forces the model to prioritize domain meaning over literal translation, which is critical for matching against sparse indexes like BM25 where exact term matching drives recall. The output is constrained to a strict JSON schema, making it immediately parseable by a downstream retrieval system without post-processing.

textCopy
SYSTEM:
You are a cross-lingual keyword extraction engine for a sparse retrieval system. Your task is to extract the most significant domain-specific terms from a user query and translate them into the target language for keyword search. Prioritize domain significance and retrieval utility over literal translation. Do not translate common stop words or generic verbs unless they carry specific domain meaning. Return ONLY a valid JSON object with no additional text, markdown fences, or commentary.

USER QUERY:
[INPUT]

SOURCE LANGUAGE:
[SOURCE_LANGUAGE]

TARGET LANGUAGE:
[TARGET_LANGUAGE]

DOMAIN CONTEXT:
[DOMAIN_CONTEXT]

OUTPUT SCHEMA:
{
  "source_query": "string",
  "source_language": "string",
  "target_language": "string",
  "keywords": [
    {
      "source_term": "string",
      "translated_term": "string",
      "domain_significance": "high | medium | low",
      "translation_rationale": "string"
    }
  ],
  "retrieval_query_string": "string"
}

CONSTRAINTS:
- Extract between 3 and 7 keywords.
- The `retrieval_query_string` must be a space-separated concatenation of all `translated_term` values, suitable for direct input into a BM25 engine.
- If a source term has no meaningful domain-specific translation, set `translated_term` to an empty string and `domain_significance` to "low".
- The `translation_rationale` must briefly explain why the translation was chosen for retrieval purposes.

To adapt this template, replace the square-bracket placeholders with your application's runtime values. [INPUT] is the raw user query. [SOURCE_LANGUAGE] and [TARGET_LANGUAGE] should be ISO 639-1 codes or full language names, depending on what your model handles best. [DOMAIN_CONTEXT] is the most critical variable: provide a concise description of the knowledge base's domain (e.g., 'enterprise cloud infrastructure and DevOps', 'pharmaceutical clinical trial protocols', 'European Union competition law'). This context is what prevents the model from translating a generic term like 'container' literally when it means 'Docker container' in your index. For high-risk domains like legal or medical, add a [RISK_LEVEL] constraint to the system prompt that forces the model to flag low-confidence translations and requires human review of the keyword list before it hits the retrieval engine.

Before integrating this prompt, test it against a golden dataset of 20-30 queries with known relevant documents in your sparse index. Measure whether the generated retrieval_query_string retrieves those documents with higher precision and recall than a naive machine translation of the full query. Common failure modes include over-translation of proper nouns that should remain in the source language, extraction of terms that are too generic to be useful discriminators, and omission of critical compound terms. If you observe these failures, add few-shot examples to the prompt showing correct behavior for your domain, or implement a post-retrieval validation step that checks if the top-N results contain the expected document IDs.

This prompt is designed to sit directly in front of a sparse retrieval index (BM25, TF-IDF, or similar keyword-based engine). The application should call the LLM with the user's source-language query and the target index language, receive the structured keyword list, and pass those keywords as the query string to the sparse retriever. Do not use the original user query directly against the target-language index—the whole point is to extract and translate domain-significant terms, not to perform a literal translation of the full sentence. The prompt output is a JSON object with a keywords array and a domain_terms array, which should be joined into a single keyword string (space-separated or OR-joined depending on your engine) before retrieval.

Validation and retry logic: Parse the LLM output as JSON and validate that both keywords and domain_terms are non-empty arrays of strings. If the output is malformed, missing fields, or contains empty arrays, retry once with the same prompt and an added instruction: The previous output was invalid. Ensure you return a valid JSON object with non-empty 'keywords' and 'domain_terms' arrays. If the second attempt also fails, fall back to a direct translation of the original query using a simpler translation prompt or a machine translation API, and log the failure for review. For high-stakes domains (legal, medical, financial), flag the fallback retrieval results for human review before they reach the end user.

Model choice and latency: This extraction task benefits from models with strong multilingual alignment and instruction-following, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Smaller models (e.g., GPT-4o-mini, Claude Haiku) can work for common language pairs but should be tested for domain terminology preservation. Cache the extracted keywords per unique (query, target_language, domain) tuple to avoid redundant LLM calls for repeated queries. Set a timeout of 2–3 seconds for the LLM call; if exceeded, fall back to the direct translation path and log the timeout.

Observability and eval: Log every extraction call with the source query, target language, extracted keywords, and the retrieval hit count. Build an eval set of 50–100 queries with known relevant documents in the target language, and measure recall@10 with and without the keyword extraction step. Track the rate of JSON parse failures, empty keyword arrays, and fallback activations. If the fallback rate exceeds 5%, review the prompt's language coverage and consider adding few-shot examples for the failing language pairs. For ongoing quality monitoring, periodically sample extraction outputs and have a bilingual reviewer check whether domain-significant terms were preserved or lost in translation.