Publisher Reputation Analysis Prompt

This prompt is designed for AI platform teams and trust-and-safety engineers who need to vet content sources at scale before those sources enter a retrieval-augmented generation (RAG) pipeline, evidence ranking system, or any workflow where grounding quality depends on source authority. The job-to-be-done is not casual browsing; it is a structured, auditable assessment of a publisher's credibility based on editorial standards, retraction history, and industry standing. The ideal user is an engineer or operator integrating this prompt into an automated evidence ingestion or filtering pipeline, where the output—a reputation tier with evidence citations—feeds directly into downstream decisions about whether to trust, weight, or discard content from that publisher.

Use this prompt when you need a repeatable, evidence-backed reputation signal that can be logged for audit trails. It is appropriate for vetting news outlets, research publishers, technical documentation sources, and institutional publications. The prompt requires structured input: a publisher name, domain, and any available metadata such as a sample of recent headlines, a Wikipedia summary, or a known retraction record. It is not designed for real-time breaking-news assessment where publication history is unavailable, nor for evaluating individual authors in isolation. Do not use this prompt as a standalone decision-maker for content removal; its output should inform a broader trust-and-safety policy, not replace human judgment in high-stakes moderation.

Before deploying this prompt, ensure your pipeline can supply the required context fields. If publisher metadata is sparse, the prompt will correctly flag low-confidence assessments rather than fabricating a reputation score. The output includes a reputation tier (e.g., 'High Authority,' 'Standard,' 'Low Credibility,' 'Unverifiable'), a confidence score, and specific evidence citations that justify the tier. This structure supports both automated filtering and human-in-the-loop review. Avoid using this prompt for sources where editorial standards are not a meaningful concept—such as personal blogs with no institutional backing—unless you explicitly configure the prompt to handle those cases with an appropriate 'Unverifiable' tier.

This prompt template evaluates the reputation of a publisher or content source by analyzing editorial standards, retraction history, industry standing, and known bias patterns. It is designed for AI platform teams who need to vet content sources at scale before allowing retrieved documents into evidence pipelines. The template uses square-bracket placeholders so you can swap in your own source metadata, evaluation criteria, and output schema without rewriting the core logic.

textCopy
You are a publisher reputation analyst. Your task is to evaluate the credibility of a publisher or content source using the provided metadata and evidence. You must produce a structured reputation assessment with specific citations for every claim you make.

## INPUT
Publisher Name: [PUBLISHER_NAME]
Publisher Domain: [PUBLISHER_DOMAIN]
Source Metadata: [SOURCE_METADATA]
Additional Context: [CONTEXT]

## EVALUATION CRITERIA
Assess the publisher across these dimensions. For each dimension, cite specific evidence from the metadata or context. If evidence is insufficient, state that explicitly.

1. **Editorial Standards**: Does the publisher have documented editorial policies, fact-checking processes, corrections procedures, and named editorial leadership? Look for transparency about how content is produced and reviewed.

2. **Retraction and Correction History**: Has the publisher issued retractions or corrections? How did they handle them? A publisher that promptly corrects errors with visible notices demonstrates stronger credibility than one that silently edits or ignores errors.

3. **Industry Standing**: Is the publisher cited by authoritative institutions, peer-reviewed journals, government bodies, or established industry organizations? Check for membership in recognized press associations, academic indexing, or regulatory recognition.

4. **Funding and Ownership Transparency**: Is the publisher's ownership structure, funding sources, and organizational affiliation clearly disclosed? Undisclosed ownership or funding from parties with vested interests in covered topics is a negative signal.

5. **Content Originality and Attribution**: Does the publisher produce original reporting or analysis, or does it primarily aggregate and republish content from other sources? Original work with clear attribution practices is a positive signal.

6. **Known Bias or Controversy Patterns**: Are there documented patterns of misleading content, partisan distortion, or controversy from independent media watchdogs, fact-checking organizations, or academic studies?

## OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "publisher_name": "string",
  "publisher_domain": "string",
  "assessment_date": "string (ISO 8601)",
  "reputation_tier": "HIGH_AUTHORITY | MEDIUM_AUTHORITY | LOW_AUTHORITY | UNRELIABLE | INSUFFICIENT_EVIDENCE",
  "dimension_scores": {
    "editorial_standards": {
      "score": "STRONG | ADEQUATE | WEAK | NO_EVIDENCE",
      "evidence": ["string (specific citation from metadata or context)"],
      "explanation": "string"
    },
    "retraction_history": {
      "score": "STRONG | ADEQUATE | WEAK | NO_EVIDENCE",
      "evidence": ["string"],
      "explanation": "string"
    },
    "industry_standing": {
      "score": "STRONG | ADEQUATE | WEAK | NO_EVIDENCE",
      "evidence": ["string"],
      "explanation": "string"
    },
    "ownership_transparency": {
      "score": "STRONG | ADEQUATE | WEAK | NO_EVIDENCE",
      "evidence": ["string"],
      "explanation": "string"
    },
    "content_originality": {
      "score": "STRONG | ADEQUATE | WEAK | NO_EVIDENCE",
      "evidence": ["string"],
      "explanation": "string"
    },
    "bias_patterns": {
      "score": "LOW_BIAS_INDICATORS | SOME_BIAS_INDICATORS | HIGH_BIAS_INDICATORS | NO_EVIDENCE",
      "evidence": ["string"],
      "explanation": "string"
    }
  },
  "overall_assessment": "string (2-4 sentence summary of findings and tier justification)",
  "confidence_level": "HIGH | MEDIUM | LOW",
  "evidence_gaps": ["string (dimensions where evidence was insufficient)"],
  "recommended_action": "TRUST_FOR_EVIDENCE | USE_WITH_CAUTION | DO_NOT_USE | NEEDS_HUMAN_REVIEW"
}

## CONSTRAINTS
- Every score must be supported by at least one specific evidence citation from the provided metadata or context.
- If you cannot find evidence for a dimension, set the score to NO_EVIDENCE and explain what you looked for.
- Do not infer reputation from domain name alone. Use only the provided metadata and context.
- If the publisher is unknown and metadata is sparse, set reputation_tier to INSUFFICIENT_EVIDENCE and recommended_action to NEEDS_HUMAN_REVIEW.
- Do not make claims about the publisher's intent, motives, or character. Stick to observable signals.
- If the publisher has a history of retractions that were handled transparently, that is a positive signal for retraction_history, not a negative one.

## RISK LEVEL
[RISK_LEVEL]

## EXAMPLES
[EXAMPLES]

To adapt this template, replace the placeholders with your actual data and configuration. [PUBLISHER_NAME] and [PUBLISHER_DOMAIN] are straightforward identifiers. [SOURCE_METADATA] should contain structured fields such as ownership records, editorial policy URLs, retraction database entries, press council memberships, and fact-checker assessments. [CONTEXT] is optional additional information such as user notes, domain-specific requirements, or prior assessments. [RISK_LEVEL] controls how conservative the assessment should be: set to HIGH for regulated domains where false positives are costly, or STANDARD for general content vetting. [EXAMPLES] should contain 1-3 few-shot examples showing the expected output format for known publishers, which dramatically improves consistency. After generating the output, validate the JSON structure, check that every score has at least one evidence citation, and flag any INSUFFICIENT_EVIDENCE results for human review before the reputation tier is used in downstream evidence ranking.

This prompt is designed to operate as a synchronous evaluation step inside a broader evidence ingestion or retrieval pipeline. It should not be called directly from a user-facing chat interface. Instead, wire it as a pre-retrieval filter or a post-retrieval re-ranker that assigns a reputation tier to each publisher before their content enters your grounding or answer-generation flow. The prompt expects structured publisher metadata as input—domain, editorial policy excerpts, retraction history, and industry standing—and returns a tiered reputation score with cited evidence. Plan for a single publisher per call to keep the evaluation focused and auditable.

Integration pattern: Deploy this prompt behind an internal API endpoint that accepts a publisher identifier and returns a JSON reputation assessment. The application layer should first resolve the publisher identifier to the required metadata fields by querying your source registry, web crawl metadata, or a publisher database. If any required field is missing, the application should either abort the call or populate the field with "NOT_AVAILABLE" and flag the result for human review. After receiving the model's response, validate the output against the expected schema: confirm the reputation_tier is one of the allowed enum values, verify that each evidence_citation references a concrete source, and check that the confidence_score falls within the 0.0–1.0 range. Reject and retry once if validation fails; escalate to a human reviewer if the second attempt also fails or if the confidence score falls below your operational threshold (we recommend 0.6 as a starting floor for automated decisions).

Model choice and latency: Use a model with strong reasoning and instruction-following capabilities, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Avoid smaller or older models that may collapse the structured output or hallucinate evidence citations. Expect 2–5 seconds of latency per publisher evaluation. For batch processing of large publisher lists, implement a concurrent worker pool with a configurable concurrency limit (start at 10) and a retry queue with exponential backoff. Log every evaluation—including the input metadata, raw model response, validated output, and any validation failures—to an append-only audit table. This audit trail is essential for downstream trust decisions and for defending your source filtering choices to compliance reviewers.

When to add product code instead of prompt-tuning: If you find yourself repeatedly adding the same publisher reputation data to the prompt (e.g., a known list of high-tier or blocked domains), move that data into a deterministic pre-check layer in your application code. A hardcoded allowlist or blocklist is faster, cheaper, and more reliable than asking the model to rediscover known facts. Reserve the prompt for publishers that fall outside your known lists. Similarly, if you need real-time retraction database lookups, integrate a tool-calling pattern where the model can request a live API call rather than relying on its training data cutoff. This keeps the prompt focused on analysis and keeps your evidence current.