Evidence Ranking for Domain-Specific Retrieval Prompt Template

This prompt is for vertical AI product teams—legal, medical, financial, or technical engineering—who need to rank retrieved evidence passages using domain-specific authority signals, terminology weighting, and evidence standards. The job-to-be-done is converting a flat retrieval set into a scored, ordered list where ranking reflects domain rules (e.g., peer-reviewed journals over blog posts, recent case law over outdated precedent, or primary sources over tertiary summaries). Generic relevance ranking fails in these domains because a highly relevant passage from an unreliable source can cause more harm than a moderately relevant passage from an authoritative one. Use this prompt when your RAG pipeline serves professional users who will act on the ranked evidence and when downstream answer generation or citation selection depends on calibrated, domain-aware ordering.

The ideal user is an engineering lead or ML engineer building a retrieval-augmented system for a regulated or high-stakes domain. You should have already retrieved candidate passages and possess metadata such as publication date, source type, author credentials, and jurisdiction or domain classification. This prompt expects you to inject domain-context instructions—such as a taxonomy of acceptable source types, terminology glossaries, and evidence hierarchy rules—into the [DOMAIN_CONTEXT] placeholder. Do not use this prompt when you lack source metadata, when the domain rules are too ambiguous to encode, or when the retrieval set is so large that ranking latency exceeds your application budget. In those cases, pre-filter with a lighter relevance classifier first.

Before wiring this into production, define your evaluation criteria: ranking stability across similar queries, alignment with expert-annotated rankings, and failure modes like authority overfitting (where the prompt ignores relevant content from lesser-known but valid sources). Always pair this prompt with a validation harness that checks output schema compliance and score calibration. In regulated domains, route low-confidence or conflicting rankings to human review before they influence answers shown to users. The next section provides the copy-ready template you can adapt.

This template adapts evidence ranking for specialized domains such as legal, medical, scientific, or technical documentation. Generic relevance ranking fails when domain-specific authority, terminology precision, and evidence standards must take priority. The template injects domain context and configurable ranking criteria so that retrieved passages are ordered by signals that matter in the target domain, not just surface-level semantic similarity. Use this prompt when your RAG pipeline serves a vertical where standard embedding similarity produces rankings that miss critical domain signals.

Below is the copy-ready template. Replace each square-bracket placeholder with your domain-specific values before use. The [DOMAIN_CONTEXT] block defines what the domain values, such as regulatory authority, peer-reviewed sources, or specific evidence hierarchies. The [RANKING_CRITERIA] block specifies the weighted dimensions for scoring, such as source authority, terminology precision, recency, and directness of support. The [PASSAGES] block contains the retrieved evidence set to rank. The [QUERY] block holds the user question or claim being evaluated. The [OUTPUT_SCHEMA] block defines the expected JSON structure for ranked results, including fields for passage ID, rank position, score per criterion, overall score, and selection rationale. The [CONSTRAINTS] block enforces domain-specific rules, such as deprioritizing non-peer-reviewed sources or requiring explicit regulatory citations. The [EXAMPLES] block provides few-shot demonstrations of correct rankings for the domain.

textCopy
You are an evidence ranking specialist for [DOMAIN_CONTEXT].

Your task is to rank the provided evidence passages by their strength, relevance, and trustworthiness for answering the given query in this domain.

## Ranking Criteria
Apply the following weighted criteria when scoring each passage:
[RANKING_CRITERIA]

## Constraints
[CONSTRAINTS]

## Query
[QUERY]

## Evidence Passages
[PASSAGES]

## Output Format
Return a JSON object with the following structure:
[OUTPUT_SCHEMA]

## Examples
[EXAMPLES]

Now rank the evidence passages according to the criteria and constraints above. Return only the JSON output.

When adapting this template, start by defining [DOMAIN_CONTEXT] with enough specificity that the model understands what counts as authoritative. For legal domains, this might include court hierarchy, jurisdiction, and precedent age. For medical domains, it might specify evidence levels from systematic reviews down to case reports. The [RANKING_CRITERIA] should be a weighted list, not a flat set of considerations. Explicit weights prevent the model from over-indexing on one dimension. The [CONSTRAINTS] block is where you encode hard rules: passages older than N years are deprioritized, non-peer-reviewed sources cannot appear in the top K, or passages must contain specific terminology to be considered relevant. The [OUTPUT_SCHEMA] should require per-criterion scores and an overall score so that downstream systems can apply their own thresholds. Include at least two [EXAMPLES] that demonstrate correct ranking with rationale, covering both straightforward and edge cases such as conflicting evidence or borderline relevance. After generating rankings, validate the output against your schema, check that constraints were respected, and run an evaluation pass comparing the ranked order against a held-out set of human-annotated rankings before shipping to production.

This prompt is not a standalone artifact; it is a ranking stage inside a retrieval-augmented generation pipeline. The typical integration point is after retrieval and before answer generation. Your application retrieves N candidate passages, injects them into this prompt along with domain-specific ranking criteria, and receives a ranked, scored, and annotated list. The downstream answer generator then consumes only the top-K passages, reducing noise and grounding the final response in the strongest evidence. For domain-specific deployments, the harness must also inject domain context—such as regulatory standards, terminology taxonomies, or authority signals—at runtime, not hard-coded into the prompt template.

Wire the prompt into your application with a structured input contract. The [QUERY] placeholder receives the user's original question. [PASSAGES] receives a pre-formatted list of retrieved passages, each with a unique ID, source metadata, and full text. [DOMAIN_CONTEXT] injects domain-specific ranking instructions: for legal domains, this might include jurisdiction, court hierarchy, and precedent age; for medical domains, it might specify evidence levels (RCT, meta-analysis, guideline), recency thresholds, and institutional authority. [OUTPUT_SCHEMA] enforces a strict JSON structure with fields for passage_id, rank, relevance_score, strength_tier, domain_authority_flags, and selection_rationale. Implement a validation layer that checks the output against this schema immediately after generation. If validation fails—missing fields, non-numeric scores, duplicate ranks—trigger a retry with the validation error message appended to the prompt context. Log every ranking output with the model version, retrieval set hash, and domain context snapshot for auditability.

For high-stakes domains, add a gating step between ranking and answer generation. If no passage achieves a minimum strength tier (e.g., 'high' or 'medium'), or if the top-ranked passage's relevance score falls below a configured threshold, route the request to a human review queue or generate a refusal response instead of proceeding to answer synthesis. This prevents the system from fabricating answers from weak evidence. Also implement coverage verification: check whether the ranked set collectively addresses all required sub-topics of the query. If coverage gaps exist, trigger a re-retrieval with expanded queries before proceeding. For model selection, prefer models with strong instruction-following and structured output capabilities. Avoid models that exhibit position bias or length bias in ranking tasks without calibration. Run periodic eval checks using the companion Evidence Ranking Failure Mode Catalog prompt to detect drift in ranking quality over time.