Domain-Specific Passage Filtering Prompt for Technical Manuals

This prompt is built for RAG pipeline engineers and technical documentation teams who need to filter retrieved passages from technical manuals before they reach the answer generation step. Generic relevance scoring fails on technical manuals because it ignores procedural intent, version compatibility, prerequisite chains, and applicability scopes. A passage might be semantically similar to a user's query but describe a procedure for the wrong product version, an incompatible operating system, or a configuration step that assumes prerequisites the user hasn't completed. When those passages slip through to the generator, the model produces answers that sound authoritative but are operationally dangerous—telling a user to run a command that doesn't exist in their version or skip a prerequisite that would brick their deployment. This prompt forces the model to assess each passage against domain-specific constraints: Is this a procedure or a concept? Does it apply to the user's stated product version and environment? Is it a prerequisite, the target procedure, or a postrequisite? The output is a structured keep/discard decision with technical reasoning, making filtering decisions auditable and debuggable rather than opaque relevance scores.

Deploy this prompt when your retrieval system pulls chunks from multi-version technical documentation, API references, configuration guides, or troubleshooting runbooks and you need to eliminate passages that are technically correct but contextually wrong for the current query. The prompt expects several inputs to function correctly: the user's original query, the retrieved passage to evaluate, and critical metadata about the user's environment—product name, version number, operating system, deployment type, and any stated prerequisites or constraints. Without this metadata, the prompt cannot perform version-aware or environment-aware filtering and will degrade to generic relevance scoring. You should wire this prompt as a filter stage between retrieval and answer generation, processing each retrieved passage individually or in small batches. The structured output format—a JSON object with a decision field, reasoning field, and optional constraint_violations array—makes it straightforward to implement programmatic filtering logic downstream. Passages marked discard are removed from the context window before generation; passages marked keep proceed to the next stage.

Do not use this prompt when your retrieval corpus is single-version, when user queries lack environmental metadata, or when the cost of per-passage LLM calls exceeds your latency or budget constraints. For high-throughput pipelines with hundreds of passages per query, consider a tiered approach: use fast embedding-based relevance scoring first, then apply this domain-aware filter only to the top-N candidates. Also avoid this prompt for non-technical content where versioning and procedural structure don't apply—a general knowledge base or FAQ system doesn't need prerequisite-chain awareness. The primary failure mode to watch for is over-filtering: the model may discard passages that are genuinely useful but don't match its internal template for what a procedure should look like. Mitigate this by running the prompt against a labeled validation set of known keep/discard pairs and measuring both precision and recall before deploying to production. When the prompt does discard a passage, log the reasoning for auditability—this is especially important in regulated environments where you may need to explain why certain information was excluded from an answer.

This prompt template is designed to be placed as a pre-generation filter in your RAG pipeline. It instructs the model to act as a technical documentation gatekeeper, evaluating each retrieved passage against strict domain-specific criteria before it enters the context window. The goal is to prevent irrelevant, outdated, or incompatible procedural content from reaching the answer-generation step, which is a primary cause of hallucination in technical support and engineering copilot systems.

textCopy
You are a technical documentation filter for [PRODUCT_NAME] version [TARGET_VERSION].
Your task is to evaluate each provided passage and determine if it is relevant, applicable, and safe to use as context for answering a user query about [PRODUCT_DOMAIN].

## Input
- User Query: [USER_QUERY]
- Target Environment: [TARGET_ENVIRONMENT] (e.g., OS, deployment type, hardware)
- Retrieved Passages: [PASSAGES]

## Evaluation Criteria
For each passage, assess the following and output a structured decision:
1. Procedural Relevance: Does the passage describe a procedure, configuration, or troubleshooting step directly related to the user query? (Score: 0-1)
2. Version Compatibility: Is the passage explicitly for [TARGET_VERSION] or a compatible version range? Flag if the version is missing or ambiguous. (Score: 0-1)
3. Environmental Applicability: Does the passage apply to the [TARGET_ENVIRONMENT]? Flag if it describes a different OS, deployment model, or hardware profile. (Score: 0-1)
4. Deprecation Risk: Does the passage reference deprecated features, removed commands, or end-of-life components? (Flag: True/False)
5. Safety and Precondition Check: Does the passage describe a destructive or irreversible action without clear warnings? (Flag: True/False)

## Output Schema
Return a JSON object with a "filtered_passages" array. Each element must conform to this schema:
{
  "passage_id": "string",
  "relevance_score": 0.0-1.0,
  "version_score": 0.0-1.0,
  "environment_score": 0.0-1.0,
  "is_deprecated": true/false,
  "has_safety_risk": true/false,
  "composite_score": 0.0-1.0,
  "decision": "INCLUDE" | "EXCLUDE" | "FLAG_FOR_REVIEW",
  "rationale": "Brief explanation of the decision."
}

## Decision Logic
- Set decision to "EXCLUDE" if composite_score < [MIN_SCORE_THRESHOLD], is_deprecated is true, or has_safety_risk is true and the query does not explicitly request a destructive action.
- Set decision to "FLAG_FOR_REVIEW" if version_score < 0.5 or environment_score < 0.5, indicating ambiguity that requires a human check.
- Otherwise, set decision to "INCLUDE".

## Constraints
- Do not modify the original passage text.
- If a passage is excluded, do not reference its content in any rationale.
- Prioritize precision over recall; it is better to exclude a marginally relevant passage than to include a misleading one.

To adapt this template, replace the square-bracket placeholders with your specific domain values. For [TARGET_VERSION], use a precise version string like 4.2.x or 2025.03 to anchor compatibility checks. For [TARGET_ENVIRONMENT], specify the deployment context such as Linux x86_64, Kubernetes 1.29. The [MIN_SCORE_THRESHOLD] should be tuned using your evaluation harness; start at 0.7 and adjust based on false-positive and false-negative rates observed in your golden dataset. The output schema is designed to be machine-readable so that your application code can programmatically filter the INCLUDE passages into the final context window and route FLAG_FOR_REVIEW items to a human review queue. If your use case involves high-risk operational commands, ensure the has_safety_risk flag is wired to a mandatory human-approval step before any downstream action is taken.

This prompt operates as a pre-generation filter in a RAG pipeline. It should execute after retrieval and before context assembly for answer generation. The filtered passage set becomes the input to your answer-generation prompt. Wire it as a separate LLM call with structured output parsing, not as inline instructions inside the answer prompt. This separation lets you log filtering decisions independently, tune filtering thresholds without touching answer quality, and run A/B tests on filter criteria.

Implement a validation layer that checks every output object against the output schema before passing passages downstream. If validation fails on any passage decision, retry the entire batch once. If the retry also fails, discard unvalidated passages and log the failure for pipeline debugging. For high-throughput systems, consider caching filter decisions keyed on passage_id and query embedding to avoid redundant LLM calls when the same passage appears across similar queries.

Always log the full filter decision array for observability. These logs become your primary debugging tool when answer quality degrades and you need to trace whether the problem is retrieval, filtering, or generation. For technical manual domains, pay special attention to version compatibility mismatches and procedural applicability errors, as these are the most common failure modes that downstream answer generation cannot recover from.