Source Authority Weighting Prompt for Claim Scoring

This prompt is designed for verification engineers who need to dynamically weight evidence sources based on their authority for a specific claim domain. It is not a general source credibility check. Use it inside a claim-scoring pipeline when you have a discrete claim and a set of candidate sources, and you need a structured, justifiable authority score for each source before using it to support or contradict the claim. The ideal user is a developer or AI architect building an automated fact-checking or claim verification system where evidence quality must be assessed programmatically.

This prompt assumes you have already retrieved or identified the sources. It does not perform retrieval, fact-checking, or final claim verification. It outputs a per-source authority assessment with domain relevance, recency, and credential-based justification. The required inputs are a specific claim and a list of candidate sources with their metadata. The output is a structured JSON object containing an authority score, a domain relevance assessment, a recency evaluation, and a credential-based justification for each source. This allows downstream scoring logic to weight evidence appropriately.

Do not use this prompt for general web page quality assessment, SEO evaluation, or social media credibility scoring. It is not a replacement for a full source vetting process in high-stakes domains like legal or medical verification without human review. Avoid using it when the claim domain is too broad to define meaningful authority criteria, as the model will default to generic credibility heuristics. For production deployments, always pair this prompt with eval checks for authority inflation and domain mismatch, and route low-confidence authority assessments to human reviewers when the claim risk level is high.

This template is the core instruction set for the Source Authority Weighting prompt. It is designed to be copied directly into your prompt management system, IDE, or orchestration layer. The prompt instructs the model to act as a verification engineer, dynamically weighting a source's authority based on its relevance to a specific claim domain, its recency, and its formal credentials. The output is a structured JSON object containing a score, a detailed justification, and flags for common failure modes like domain mismatch.

textCopy
You are a verification engineer assessing the authority of a source for a specific claim domain. Your task is to produce a dynamic authority score that reflects how credible and relevant a given source is for verifying a particular type of claim.

INPUT:
- Source Description: [SOURCE_DESCRIPTION]
- Claim Domain: [CLAIM_DOMAIN]
- Claim Statement (for context): [CLAIM_STATEMENT]

OUTPUT_SCHEMA:
{
  "authority_score": <float between 0.0 and 1.0>,
  "score_justification": {
    "domain_relevance": "<explanation of how the source's expertise aligns with the claim domain>",
    "recency_assessment": "<explanation of the source's timeliness relative to the claim>",
    "credential_basis": "<explanation of the formal credentials, reputation, or institutional weight of the source>"
  },
  "flags": {
    "domain_mismatch_risk": <boolean>,
    "authority_inflation_risk": <boolean>,
    "requires_human_review": <boolean>
  }
}

CONSTRAINTS:
- Do not assume a source is authoritative for all domains. A Nobel laureate in physics is not inherently an authority on medieval history.
- Penalize sources that are self-referential or lack external validation.
- If the source's recency is critical to the claim (e.g., a breaking news event), heavily weight the `recency_assessment`.
- Set `requires_human_review` to `true` if the source is anonymous, has a known extreme bias, or if the domain relevance is ambiguous.
- Set `authority_inflation_risk` to `true` if the source is a well-known brand or individual that might be overvalued by a general-purpose model.

To adapt this template, replace the [SOURCE_DESCRIPTION], [CLAIM_DOMAIN], and [CLAIM_STATEMENT] placeholders with your data. The OUTPUT_SCHEMA is critical for integration; ensure your application's parser expects this exact JSON structure. The CONSTRAINTS section is where you should add domain-specific rules, such as a list of banned sources or a minimum recency threshold for time-sensitive claims. For high-stakes verification pipelines, always route outputs where requires_human_review is true to a review queue before accepting the score.

Integrating the Source Authority Weighting prompt into an application requires treating it as a deterministic scoring function within a broader claim verification pipeline. The prompt expects a claim, a source identifier, and the claim's domain as inputs, and it returns a structured authority score with justification. In production, this prompt should be called once per source per claim, making it a high-volume component. Therefore, the implementation harness must focus on input validation, output schema enforcement, cost control, and consistent logging for downstream auditability. Do not call this prompt in isolation; its output should feed directly into an evidence sufficiency or confidence calibration scorer that combines authority weights with other signals like corroboration count and recency.

The recommended integration pattern is a serverless function or microservice that wraps the model call with strict guards. Before calling the model, validate that the [CLAIM] text is non-empty and under a token limit, the [SOURCE_DESCRIPTOR] contains enough information for authority assessment (e.g., publication name, author credentials, or domain), and the [DOMAIN] maps to a controlled vocabulary of claim categories your system recognizes. After receiving the model response, parse the JSON output and validate it against a defined schema: authority_score must be a float between 0.0 and 1.0, domain_relevance must be a float in the same range, and justification must be a non-empty string. If parsing or schema validation fails, implement a single retry with the error message appended to the prompt as a correction instruction. If the retry also fails, log the failure, assign a default low authority score (e.g., 0.1), and flag the source for human review. For model choice, a fast, cost-effective model like GPT-4o-mini or Claude Haiku is sufficient for this structured classification task, but switch to a more capable model if your domain vocabulary includes highly specialized scientific or legal fields where nuanced credential assessment is critical.

Observability is non-negotiable for this prompt because authority scores directly influence which evidence is trusted downstream. Log every call with a trace ID, the claim hash, the source identifier, the raw model response, the validated score, and the model version. This audit trail enables you to replay decisions, debug authority inflation, and demonstrate to stakeholders why one source was weighted higher than another. Implement a periodic eval harness that runs a golden set of source-claim pairs with known authority expectations through the prompt and compares scores against human-labeled baselines. Watch for two specific failure modes: authority inflation, where well-known but domain-irrelevant sources receive high scores, and domain mismatch, where a source's general reputation bleeds into a specialized domain where it lacks expertise. If your eval detects drift, recalibrate by adjusting the prompt's domain relevance instructions or adding few-shot examples that penalize off-domain authority. Finally, never expose raw authority scores to end users without the accompanying justification text; the explanation is the primary defense against disputes and the foundation of an auditable verification product.