Domain-Specific Evidence Ranking Prompt Template

This prompt is designed for vertical AI product teams who need evidence ranking that respects domain-specific authority rules, terminology, and evidence hierarchies. Generic relevance ranking fails in legal, medical, scientific, and financial domains because it ignores precedent weight, regulatory authority, study design quality, and tiered source credibility. Use this prompt when your RAG pipeline retrieves passages from domain-specific corpora and you need a ranked list that a domain expert would agree with. The prompt expects pre-retrieved evidence passages and a domain configuration that defines the ranking criteria. It produces a ranked list with per-item scores and structured explanations for each ranking decision.

Concrete scenarios where this prompt applies include: ranking legal precedents by court hierarchy and jurisdictional relevance rather than semantic similarity to a query; ordering medical studies by evidence level (RCT > cohort > case study) and sample size; prioritizing financial filings by regulatory authority (SEC filings over analyst reports); and sorting technical documentation by version currency and authoring team authority. In each case, the ranking logic must encode domain-specific rules that a general-purpose relevance model cannot infer. The domain configuration block in the prompt template lets you specify these rules explicitly—authority tiers, recency thresholds, terminology mappings, and evidence type hierarchies—so the model applies them consistently across ranking decisions.

Do not use this prompt for general web search results, consumer Q&A, or domains where recency alone determines relevance. If your evidence sources are homogeneous (all from the same authority tier), your ranking criteria are purely semantic, or your users do not require explainable ranking decisions, a simpler relevance scoring prompt will be faster and cheaper. This prompt adds token overhead and latency from structured explanations that only pay off when domain-specific ranking logic is required. Before adopting this prompt, confirm that your retrieval pipeline can supply the metadata fields the domain configuration references—such as source type, publication date, author credentials, or evidence tier—and that your evaluation framework includes domain experts who can judge whether the ranking respects domain norms.

This prompt template is designed to be copied directly into your AI system and adapted for domain-specific evidence ranking workflows. It accepts a query, a set of retrieved evidence passages, and domain configuration parameters that define what makes evidence authoritative in your vertical. The template uses square-bracket placeholders that you replace with your actual domain rules, evidence hierarchy definitions, terminology standards, and the specific passages to rank. The output is a structured, ranked list with explanations that reference your domain's criteria, making the ranking decisions auditable and defensible.

textCopy
You are an evidence ranking specialist operating in the [DOMAIN_NAME] domain. Your task is to rank the provided evidence passages according to domain-specific authority criteria, terminology standards, and evidence hierarchies.

## DOMAIN CONFIGURATION

**Authority Criteria (ordered by importance):**
[DOMAIN_AUTHORITY_CRITERIA]

**Evidence Hierarchy:**
[DOMAIN_EVIDENCE_HIERARCHY]

**Required Terminology Standards:**
[DOMAIN_TERMINOLOGY_STANDARDS]

**Recency Threshold:**
[DOMAIN_RECENCY_THRESHOLD]

**Disqualifying Factors:**
[DOMAIN_DISQUALIFYING_FACTORS]

## QUERY

[USER_QUERY]

## EVIDENCE PASSAGES

[EVIDENCE_PASSAGES]

## OUTPUT SCHEMA

Return a JSON object with this exact structure:
{
  "ranked_evidence": [
    {
      "rank": number,
      "passage_id": "string",
      "relevance_score": number (0.0 to 1.0),
      "authority_score": number (0.0 to 1.0),
      "composite_score": number (0.0 to 1.0),
      "ranking_rationale": "string explaining why this passage received its rank, referencing specific domain criteria",
      "key_terms_matched": ["string"],
      "hierarchy_level": "string from the evidence hierarchy",
      "limitations": ["string describing any weaknesses or caveats"]
    }
  ],
  "ranking_methodology_summary": "string describing the overall ranking approach",
  "evidence_gaps": ["string describing information missing from the evidence set"],
  "confidence_assessment": {
    "overall_confidence": "high|medium|low",
    "confidence_rationale": "string"
  }
}

## CONSTRAINTS

- Rank all passages, even if some are weak. Do not omit passages.
- If two passages have identical composite scores, break ties by authority_score, then by recency.
- Flag any passage that contains terminology inconsistent with domain standards.
- If a passage meets any disqualifying factor, still include it but set its composite_score to 0.0 and explain the disqualification in limitations.
- Do not invent evidence or facts not present in the provided passages.
- If the evidence set is insufficient to answer the query, reflect this in evidence_gaps and lower the confidence_assessment.

To adapt this template, replace each placeholder with your domain's actual configuration. The [DOMAIN_AUTHORITY_CRITERIA] placeholder should contain a ranked list of what makes a source authoritative in your field—for example, in medicine this might prioritize peer-reviewed RCTs over case reports, while in law it might prioritize binding precedent from higher courts over persuasive authority from other jurisdictions. The [DOMAIN_EVIDENCE_HIERARCHY] should define the levels of evidence quality specific to your domain, such as the Oxford Centre for Evidence-Based Medicine levels or a custom hierarchy your organization uses. The [DOMAIN_TERMINOLOGY_STANDARDS] placeholder should list the controlled vocabularies, taxonomies, or nomenclature systems your domain requires—SNOMED CT for clinical contexts, Bluebook citation format for legal, or GAAP terminology for financial domains. Before deploying, test this prompt against a golden dataset of ranked evidence pairs where domain experts have already established the correct ranking order, and measure whether the model's composite scores correlate with expert judgments. For high-stakes domains, route outputs where confidence_assessment is "low" or where disqualifying factors are triggered to a human reviewer before the ranking is used downstream.

The Domain-Specific Evidence Ranking Prompt Template is not a standalone artifact; it is a processing stage inside a retrieval-augmented generation pipeline. In production, this prompt sits between retrieval and answer generation. It receives a set of candidate passages, a query, and domain-specific ranking criteria, and it returns an ordered list with explanations. The harness must enforce that the output is a valid, complete ranking before any downstream component consumes it. A partial ranking, a hallucinated source ID, or a missing explanation can silently corrupt the final answer and break audit trails.

Wire the prompt into an application function that accepts query, retrieved_passages (with IDs, text, and metadata), and domain_config (authority criteria, terminology standards, evidence hierarchy rules). The function should call the model with the prompt template, parse the JSON output, and run a structural validator that checks: every passage ID in the output exists in the input set, no passage appears more than once, the ranking is a total order (no ties unless explicitly allowed), and every rank has a non-empty explanation. If validation fails, execute a single retry with the validation error appended as a [CORRECTION] block in the prompt. After one retry, if the output still fails validation, log the failure and route to a human review queue rather than silently falling back to an unranked list. This is especially critical in legal, medical, and financial domains where ranking errors can produce compliance violations.

For model choice, prefer a model with strong instruction-following and JSON mode support. In high-stakes domains, run a shadow evaluation using an LLM judge prompt that compares the ranked output against a domain-expert-curated gold ranking for a sample of queries. Log the judge's agreement score and flag queries where the model deprioritized a high-authority source. Store the full prompt, response, validation result, and any human override in an audit log with timestamps and model version. This log becomes the evidence trail for governance reviews. Do not use this prompt for real-time, user-facing answer generation without first validating the ranking in a staging environment with domain-representative retrieval sets. The next step is to integrate the validated ranking output into your answer generation prompt, passing the top-K passages and their rank explanations as prioritized context.