Source Credibility Weighted Synthesis Prompt

This prompt is designed for AI platform teams that have already assigned authority or credibility scores to their retrieved sources and need the synthesis step to respect that ranking. The primary job-to-be-done is generating an answer that prioritizes high-credibility sources, downgrades or excludes low-credibility sources, and transparently explains its weighting decisions. The ideal user is a RAG pipeline architect or an AI engineer who controls the retrieval and scoring stages and now needs the generation stage to be evidence-weighting-aware. The required context is a pre-retrieved, pre-scored set of passages, each with a credibility score and source identifier, plus a user query that the answer must address.

You should use this prompt when your retrieval pipeline already produces a credibility score per passage—for example, a 0.0–1.0 authority rating from a domain-specific classifier, a recency-weighted trust score, or a human-curated source tier. The prompt assumes you have already completed retrieval and scoring; it does not perform search or assign credibility scores from scratch. It is particularly valuable in regulated or high-stakes domains (legal, financial, medical) where citing a low-authority source can cause compliance or trust failures. The prompt includes explicit instructions to explain weighting decisions, making it suitable for applications that require audit trails or user-facing transparency about why certain sources were relied upon more heavily.

Do not use this prompt if you lack source-level credibility metadata, if your retrieval set has not been pre-filtered for relevance, or if you need the model to perform the initial credibility assessment. This prompt is not a replacement for a retrieval step or a scoring model—it is a synthesis harness that consumes scored evidence. Avoid using it when all sources are equally credible, as the weighting instructions will add unnecessary complexity without improving output quality. For high-risk applications, always pair this prompt with a post-generation faithfulness verification step and human review for critical decisions. If your credibility scores are unreliable or uncalibrated, the prompt will faithfully amplify those errors, so validate your scoring pipeline before relying on weighted synthesis in production.

This template is designed to be pasted into your system or user message. It instructs the model to generate a final answer that prioritizes information from high-credibility sources, downgrades or excludes low-credibility sources, and explains its weighting decisions. Replace the square-bracket placeholders with your specific inputs, output schema, and constraints before wiring it into your application.

textCopy
You are an evidence synthesis engine. Your task is to produce a final answer to a user's question by combining information from multiple provided sources. Each source has a pre-assigned credibility score from 0.0 (untrustworthy) to 1.0 (fully authoritative).

# INPUT
User Question:
[USER_QUESTION]

Sources with Credibility Scores:
[SOURCES_JSON_ARRAY]

# INSTRUCTIONS
1. **Prioritize High-Credibility Sources:** Base your answer primarily on sources with a credibility score of [HIGH_CREDIBILITY_THRESHOLD] or higher. Use these sources as the foundation of your response.
2. **Downgrade or Exclude Low-Credibility Sources:** Information from sources with a score below [LOW_CREDIBILITY_THRESHOLD] should be treated as unreliable. Do not use it to form the core of your answer. If you must mention it, clearly flag it as low-credibility.
3. **Handle Conflicts:** If high-credibility sources disagree, present the conflicting viewpoints without fabricating a consensus. If a high-credibility source contradicts a low-credibility one, default to the high-credibility source.
4. **Explain Weighting:** In your reasoning, briefly note how the credibility scores influenced your synthesis. If you downgraded or excluded a source, explain why.
5. **Synthesize, Don't List:** Do not summarize each source individually. Weave the information into a single, coherent answer.

# OUTPUT FORMAT
You must respond with a single JSON object conforming to this schema:
[OUTPUT_SCHEMA]

# CONSTRAINTS
- Do not introduce any external knowledge not present in the provided sources.
- If no source meets the high-credibility threshold, state that a reliable answer cannot be provided and explain why.
- If the provided sources are insufficient to answer the question, set the `answerable` field to `false` and explain what is missing.

To adapt this template, replace [USER_QUESTION] with the user's query, and [SOURCES_JSON_ARRAY] with a JSON array of objects, each containing source_id, text, and credibility_score. Define [HIGH_CREDIBILITY_THRESHOLD] and [LOW_CREDIBILITY_THRESHOLD] based on your risk tolerance (e.g., 0.8 and 0.4). The [OUTPUT_SCHEMA] placeholder must be replaced with your exact JSON schema, including fields for the answer, weighting_rationale, and an answerable boolean. For high-stakes domains, ensure your application layer validates the output against this schema and logs any instances where the model fails to provide a weighting rationale or incorrectly marks a question as unanswerable.

This prompt is designed to sit downstream of a retrieval step and an authority-scoring step. Before calling the model, assemble the [SOURCES] array where each object contains the passage text, a pre-computed authority_score (e.g., 0.0–1.0), and a source_id. The prompt expects these scores to already exist—do not ask the model to compute credibility on the fly, as that introduces latency and inconsistency. The model's job is to weight its synthesis according to the scores you provide, not to re-evaluate source trustworthiness.

Wire the prompt into an application harness that validates the output schema before the response reaches the user. The expected output is a JSON object with answer, weighting_explanation, and excluded_sources fields. Implement a strict JSON schema validator in your application layer. If parsing fails, retry once with a repair prompt that includes the raw output and the schema. If the retry also fails, log the failure and fall back to a safe response that surfaces the raw sources without synthesis. For high-stakes domains, route outputs where excluded_sources contains high-authority sources to a human review queue—this indicates a possible scoring error upstream.

Log every invocation with the prompt_version, source_ids, authority_scores, output, and validator_result. This audit trail is essential for debugging credibility-weighting failures. Common failure modes include the model ignoring low-authority sources entirely when they contain critical context, or over-weighting a single high-authority source and producing a one-sided answer. Use eval assertions to check that the weighting_explanation references specific scores and that excluded sources are genuinely low-authority. Run a periodic eval sweep with synthetic source sets where the correct weighting is known to detect drift in model behavior.