Evidence Ranking with Source Metadata Prompt Template

This prompt is designed for RAG pipeline engineers and search architects who have access to rich document metadata alongside retrieved passages. When your retrieval system returns passages with associated metadata such as publication dates, author credentials, document types, or domain authority scores, this prompt combines those trustworthiness signals with content relevance to produce a weighted, explainable ranking. Use this prompt when you need to prevent low-authority or outdated sources from outranking more credible evidence, and when downstream answer generation or citation selection depends on a calibrated evidence ordering rather than raw retrieval scores.

This prompt assumes you have already retrieved a candidate set of passages and their metadata. It does not perform retrieval itself, nor does it verify factual accuracy of passage content. It is a ranking and prioritization step that sits between retrieval and answer synthesis. You should provide the prompt with a query or claim, a list of passages with unique IDs, and structured metadata fields for each passage. The prompt expects metadata such as publication_date, author_credentials, document_type, domain_authority_score, and citation_count. If your retrieval pipeline does not produce these fields, you must enrich passages with metadata before invoking this prompt. The output is a ranked list with scores, tier assignments, and human-readable rationale for each ranking decision.

Do not use this prompt when your retrieval set lacks metadata, when passages are already pre-filtered by authority, or when you need strict factual verification rather than trust-weighted ranking. This prompt is also inappropriate for real-time streaming ranking where latency constraints preclude structured output generation. For high-stakes domains such as legal or clinical evidence ranking, always pair this prompt with human review of the ranked output before it feeds into answer generation. The ranking decisions are explainable but not infallible—metadata signals can be gamed, and authority is not a guarantee of accuracy. Wire this prompt into a pipeline that logs ranking decisions, tracks ranking stability across similar queries, and gates downstream synthesis on minimum confidence thresholds.

This prompt template is designed to be pasted directly into your system instructions for an evidence ranking step. It instructs the model to evaluate a set of retrieved passages against a user query, weighing both content relevance and source metadata—such as publication date, author authority, document type, and domain—to produce a weighted, explainable ranking. The goal is to move beyond simple vector similarity by explicitly incorporating trustworthiness signals that downstream answer generation and citation selection can depend on.

textCopy
You are an evidence ranking specialist. Your task is to rank a set of retrieved passages for a given query. You must balance content relevance with source trustworthiness using the provided metadata.

## INPUT
- Query: [QUERY]
- Passages: [PASSAGES_JSON]
  Each passage object includes:
  - id: string
  - text: string
  - metadata:
    - publication_date: ISO 8601 string or null
    - author_authority: "high" | "medium" | "low" | "unknown"
    - document_type: string (e.g., "research_paper", "technical_doc", "blog_post", "internal_wiki", "news_article")
    - domain: string (e.g., "medical", "legal", "engineering", "general")
- Ranking Criteria: [RANKING_CRITERIA]
  A list of weighted dimensions, e.g., [{"dimension": "relevance", "weight": 0.5}, {"dimension": "recency", "weight": 0.2}, {"dimension": "authority", "weight": 0.2}, {"dimension": "specificity", "weight": 0.1}]
- Output Schema: [OUTPUT_SCHEMA]
  Define the expected JSON structure for the ranked list.
- Constraints: [CONSTRAINTS]
  e.g., "Return exactly the top 5 passages.", "Exclude any passage with author_authority 'low'."

## INSTRUCTIONS
1. For each passage, assess its relevance to the query based on semantic overlap and specificity.
2. Adjust the relevance score using the metadata:
   - **Recency**: Prefer more recent publication dates for time-sensitive queries. Apply a decay factor for older documents.
   - **Authority**: Boost passages from high-authority authors or reputable document types (e.g., research papers over blog posts). Penalize low or unknown authority.
   - **Domain**: Increase weight if the document's domain matches the query's implied domain.
3. Calculate a final weighted score for each passage based on the provided [RANKING_CRITERIA].
4. Sort passages by their final score in descending order.
5. For each ranked passage, provide a concise rationale explaining how its content and metadata contributed to its score.
6. Output the result strictly according to the [OUTPUT_SCHEMA].

## OUTPUT
Return a valid JSON object with a "ranked_passages" array. Do not include any text outside the JSON.

To adapt this template, replace the square-bracket placeholders with your actual data before sending it to the model. The [PASSAGES_JSON] should be a serialized JSON array of your retrieved documents with their metadata. The [RANKING_CRITERIA] array lets you tune the balance between relevance and trustworthiness for your specific use case—a legal research tool might weight authority at 0.4, while a news summarizer might push recency to 0.5. The [OUTPUT_SCHEMA] placeholder should be replaced with your exact expected JSON structure, including field names like rank, passage_id, final_score, and rationale. The [CONSTRAINTS] field is your lever for enforcing business rules like top-K limits or authority floors. After generating the ranking, always validate the output against your schema and run evaluation checks for common failure modes like position bias or authority overfitting before passing the ranked list to downstream answer generation.

This prompt is designed to sit between your retrieval step and your answer generation step in a RAG pipeline. It consumes a set of retrieved passages with their associated metadata and produces a ranked list. The implementation harness must treat this prompt as a deterministic component: same inputs should produce a stable ranking, and any deviation should be logged and flagged. The harness is responsible for assembling the [PASSAGES] array with consistent metadata fields, invoking the model, validating the output schema, and passing only the top-K ranked passages to the downstream answer generation prompt. Do not skip validation here—a malformed ranking silently degrades answer quality and citation accuracy downstream.

Start by defining a strict output contract. The prompt expects a JSON array of ranked passages, each with passage_id, rank, composite_score, and ranking_rationale. Your harness should validate this schema immediately after model response. Use a JSON Schema validator (e.g., ajv for Node.js, pydantic for Python) to check required fields, types, and value ranges. If validation fails, implement a retry loop with a maximum of two additional attempts, appending the validation error message to the prompt context so the model can self-correct. After three total failures, log the full input, output, and errors to your observability platform, then fall back to a simpler ranking heuristic (e.g., recency-only or vector similarity score) to keep the pipeline running. For high-stakes domains like legal or medical, route failed rankings to a human review queue instead of falling back to a heuristic.

Observability is non-negotiable. Log every ranking request with: a unique trace_id, the query, the number of input passages, the model used, the latency, the output ranking, and a hash of the ranked passage IDs. This lets you detect ranking drift when you change models or update the prompt. Add a lightweight eval check that compares the top-ranked passage against a baseline (e.g., the passage with the highest vector similarity score). If the top-ranked passage changes by more than a configurable threshold, emit a warning metric. For cost control, cache rankings for identical [QUERY] + [PASSAGES] combinations with a short TTL, but invalidate the cache when any passage metadata changes. Finally, wire the ranked output into your answer generation prompt by injecting only the top-K passages (where K is determined by your context budget) along with their composite_score and ranking_rationale, so the downstream model can weight evidence appropriately.