Domain Authority Assessment Prompt for Retrieved Documents

This prompt is designed for RAG pipeline architects and trust-and-safety engineers who need to assign structured authority scores to source domains before evidence is used for answer generation. It evaluates institutional affiliation, expertise signals, and citation patterns to produce a tiered authority rating with explainability fields. Use this prompt when your retrieval system returns documents from mixed-quality domains and you need a repeatable, auditable method to weight or filter sources before they enter the evidence ranking stage.

The ideal user is building or maintaining a production RAG system where source quality directly impacts answer reliability. You should have access to source metadata including the domain, publisher name, author credentials when available, and the document's citation context. This prompt works best as a pre-ranking filter: run it after retrieval but before evidence synthesis, so low-authority domains can be deprioritized or excluded before they influence the final answer. The output is structured JSON suitable for downstream ranking logic, audit logging, or human review queues.

This is not a general credibility classifier. It focuses specifically on domain-level authority assessment, not page-level fact-checking or claim verification. Do not use this prompt when you need to verify specific factual claims, detect hallucination in generated text, or assess the relevance of a passage to a query. It also should not be the sole gatekeeper for high-stakes decisions in regulated domains—always pair it with human review, evidence grounding checks, and recency validation. If your retrieval pipeline already filters by an allowlist of trusted domains, this prompt adds little value. Use it when you operate in open-domain retrieval with heterogeneous source quality.

This template is designed to be the core instruction set for a language model tasked with evaluating the authority of a domain. It moves beyond simple binary checks, asking the model to consider institutional affiliation, expertise signals, and citation patterns to assign a structured authority tier. The output is designed for direct consumption by a downstream ranking system in a RAG pipeline, providing both a score and human-readable explanations for auditability. Copy the block below, replace the placeholders with your specific inputs, and integrate it into your prompt assembly process.

textCopy
You are an expert source authority analyst. Your task is to assess the authority tier of a given source domain based on institutional affiliation, expertise signals, and citation patterns. You will output a structured JSON object with a score and detailed reasoning.

**INPUT DATA**
- Source Domain: [SOURCE_DOMAIN]
- Source URL: [SOURCE_URL]
- Document Title: [DOCUMENT_TITLE]
- Document Snippet: [DOCUMENT_SNIPPET]
- Query Context: [USER_QUERY]

**OUTPUT SCHEMA**
You must respond with a single, valid JSON object conforming to this structure:
{
  "authority_tier": "high" | "medium" | "low" | "unverifiable",
  "authority_score": 0.0-1.0,
  "signals": {
    "institutional_affiliation": {
      "detected": true | false,
      "type": "government" | "academic" | "commercial" | "non-profit" | "personal" | "unknown",
      "evidence": "Brief quote or description from the snippet."
    },
    "expertise_signals": {
      "detected": true | false,
      "indicators": ["List of strings like 'author credentials', 'peer-reviewed citation', 'official documentation'", "..."],
      "evidence": "Brief quote or description from the snippet."
    },
    "citation_patterns": {
      "detected": true | false,
      "type": "self-citation" | "external-authoritative" | "external-unverifiable" | "none",
      "evidence": "Brief quote or description from the snippet."
    }
  },
  "explanation": "A concise, 2-3 sentence summary of the overall assessment, explicitly linking the detected signals to the assigned tier.",
  "confidence": 0.0-1.0
}

**CONSTRAINTS**
- Base your assessment *only* on the provided INPUT DATA. Do not use external knowledge about the domain.
- If you cannot determine a signal, set its `detected` field to `false` and provide an empty string for `evidence`.
- An `authority_tier` of "unverifiable" must be used if the provided snippet contains no usable signals. The score should be 0.0.
- A "high" tier requires strong, positive signals in at least two of the three signal categories.
- The `confidence` score must reflect how much evidence was available, not your certainty in a guess. Low evidence should result in low confidence, even if the tier is "high" based on a single strong signal.

To adapt this template, replace the [SOURCE_DOMAIN], [SOURCE_URL], [DOCUMENT_TITLE], [DOCUMENT_SNIPPET], and [USER_QUERY] placeholders with data from your retrieval system. The OUTPUT_SCHEMA is a strict contract; ensure your post-processing code validates the JSON structure and enum values before using the authority_score for downstream ranking. For high-stakes domains like legal or medical research, this prompt's output should be treated as a preliminary signal, not a final verdict, and must be paired with a human review step or a more authoritative ground-truth database.

This prompt is designed to sit between retrieval and evidence ranking in a RAG pipeline. After your search system returns a set of candidate documents with their source domains, you call this prompt to assign an authority tier and structured score to each unique domain. The output feeds directly into your downstream evidence weighting logic, where authority scores combine with relevance, recency, and specificity signals to produce a final ranking. Do not call this prompt on every retrieved passage individually—batch by unique domain to control latency and cost, then join the authority scores back to individual passages before ranking.

Wire the prompt into your application as a preprocessing step with a domain deduplication cache. Before calling the model, extract all unique domains from the retrieval set and check a local cache (Redis, in-memory dictionary, or database table) for recent authority assessments. Only send uncached domains to the model. This prevents redundant API calls for frequently retrieved domains like arxiv.org or cdc.gov. Set a cache TTL appropriate to your domain—24 hours for most applications, shorter for fast-moving news domains. The model response should be validated against a strict schema: require domain, authority_tier (one of an enum you define, such as high, medium, low, unverifiable), score (float 0.0–1.0), and explanation fields. Reject any response missing these fields and retry once with a repair prompt that includes the validation error.

For production reliability, implement a fallback authority tier when the model call fails after retries. A safe default is unverifiable with a score of 0.0, which forces downstream ranking to rely on other signals. Log every authority assessment with the domain, score, tier, model version, and timestamp for auditability. If your use case involves regulated content (legal, medical, financial), add a human review queue for domains scored below a configurable threshold or flagged with conflicting signals in the explanation field. Start with a threshold of 0.4 and adjust based on observed false-positive and false-negative rates in your eval set. Model choice matters: use a model with strong reasoning capabilities (GPT-4o, Claude 3.5 Sonnet) rather than a lightweight classifier, because the explainability field requires nuanced institutional knowledge that smaller models often hallucinate.