Model Output Trustworthiness Assessment Prompt

This prompt is designed for RAG and factuality teams who need an automated, structured assessment of whether a model-generated output can be trusted without manual review. It acts as a programmatic gate between your generation step and downstream consumers, producing a trustworthiness score, hallucination indicators, and source fidelity checks. The ideal user is an ML engineer or platform developer responsible for output quality in a production system where incorrect or fabricated answers carry high cost—customer-facing search, regulated document Q&A, clinical summarization, or audit-facing reporting. You should have access to the original source context used during generation, because the prompt's core value comes from comparing the output against that evidence.

Use this prompt when you need a repeatable, low-latency gating decision that can run at inference time without blocking on human review. It is particularly effective when your generation pipeline already retrieves source documents and you can pass both the output and the retrieved context into the assessment step. The prompt produces structured output—a score, a set of hallucination flags, and a fidelity breakdown—that your application can act on: route to a human queue, trigger a retry, log for audit, or block the response entirely. Do not use this prompt for creative writing, open-ended brainstorming, summarization of fictional content, or tasks where factual grounding is not the primary quality dimension. It is also a poor fit when source context is unavailable or when the cost of an incorrect gate decision (false positive or false negative) hasn't been calibrated against your product's risk tolerance.

Before deploying this prompt, you must define what 'trustworthy' means for your use case and set explicit thresholds. A score of 0.8 might be acceptable for internal tooling but dangerous for patient-facing clinical answers. Plan to run this prompt against a labeled evaluation set—ideally the same factuality benchmarks you use for your generator—so you can measure gate accuracy, false-positive rates, and false-negative impact. If your domain is regulated or high-risk, this prompt should feed a human review queue, not make final autonomous decisions. The next sections give you the template, harness wiring, and failure modes to make that operational.

This prompt template provides a structured framework for evaluating whether a model's output can be trusted without manual review. It is designed for RAG and factuality teams who need to produce a trustworthiness score alongside detailed evidence grounding assessments, hallucination indicators, and source fidelity checks. The template uses square-bracket placeholders that you must replace with your specific inputs, context, and evaluation criteria before running it in your evaluation harness.

textCopy
You are an expert output trustworthiness evaluator. Your task is to assess the trustworthiness of a model-generated output against provided source evidence and produce a structured assessment.

## INPUT

### Model Output to Evaluate
[OUTPUT]

### Source Evidence
[SOURCE_EVIDENCE]

### Context
[CONTEXT]

## EVALUATION CRITERIA

Assess the output across the following dimensions:

1. **Evidence Grounding**: Does each claim in the output have supporting evidence in the source material? Identify unsupported claims.
2. **Hallucination Indicators**: Are there fabricated facts, entities, citations, or values not present in the source evidence?
3. **Source Fidelity**: Does the output accurately represent the source material without distortion, omission of critical context, or misleading paraphrasing?
4. **Completeness**: Does the output address all required aspects without gaps that could mislead?
5. **Uncertainty Expression**: Does the output appropriately express uncertainty where evidence is ambiguous or incomplete?

## CONSTRAINTS

[CONSTRAINTS]

## OUTPUT SCHEMA

Return a JSON object with the following structure:

```json
{
  "trustworthiness_score": <float between 0.0 and 1.0>,
  "overall_assessment": "<TRUSTWORTHY|NEEDS_REVIEW|UNTRUSTWORTHY>",
  "dimension_scores": {
    "evidence_grounding": <float 0.0-1.0>,
    "hallucination_indicators": <float 0.0-1.0 where 1.0 means no hallucinations>,
    "source_fidelity": <float 0.0-1.0>,
    "completeness": <float 0.0-1.0>,
    "uncertainty_expression": <float 0.0-1.0>
  },
  "unsupported_claims": [
    {
      "claim": "<exact claim text>",
      "reason": "<why it is unsupported>"
    }
  ],
  "hallucinated_content": [
    {
      "content": "<fabricated text>",
      "type": "<entity|fact|citation|value|other>",
      "explanation": "<why this appears fabricated>"
    }
  ],
  "source_fidelity_issues": [
    {
      "issue": "<description of distortion or omission>",
      "source_reference": "<relevant source passage>",
      "severity": "<minor|moderate|major>"
    }
  ],
  "recommendation": "<clear guidance on whether to accept, review, or reject the output>",
  "review_priority": "<low|medium|high|critical>"
}

EXAMPLES

[EXAMPLES]

RISK LEVEL

[RISK_LEVEL]

INSTRUCTIONS

1. Compare every factual claim in the output against the source evidence.
2. Flag any content that cannot be traced to the source material.
3. Note where the output adds interpretation beyond what the evidence supports.
4. Assess whether omissions could mislead a reader.
5. Produce the assessment in the exact JSON schema specified above.
6. If the risk level is HIGH or CRITICAL, err on the side of flagging for human review.

To adapt this template for your use case, replace each square-bracket placeholder with your specific content. The [OUTPUT] placeholder should contain the model-generated text you are evaluating. [SOURCE_EVIDENCE] must include the retrieved passages, documents, or ground-truth data against which you are checking. [CONTEXT] provides any additional framing, such as the original user query or task description. [CONSTRAINTS] allows you to specify domain-specific rules, such as regulatory requirements or acceptable deviation thresholds. [EXAMPLES] should include few-shot demonstrations of correct assessments to calibrate the evaluator. [RISK_LEVEL] sets the evaluation strictness—use LOW for exploratory outputs, MEDIUM for internal tools, HIGH for customer-facing content, and CRITICAL for regulated domains. After adapting the template, validate the output JSON against your schema before trusting the assessment. For high-risk workflows, always route NEEDS_REVIEW and UNTRUSTWORTHY assessments to human reviewers and log all assessments for audit trails.

The Model Output Trustworthiness Assessment Prompt is not a standalone evaluator—it is a gating component that must sit inside a broader application harness. The prompt produces a structured trustworthiness score, evidence grounding assessment, and hallucination indicators, but the harness is responsible for deciding what happens next: accept the output, route it for human review, retry with a different model, or discard it entirely. This section covers the integration points, validation layers, and operational controls needed to make the assessment reliable in production.

Begin by wrapping the prompt call in a validation layer that checks the structural integrity of the returned JSON before trusting the scores. The expected output schema includes fields like trustworthiness_score (a float between 0.0 and 1.0), grounding_assessment (an object with source_fidelity and evidence_coverage fields), hallucination_indicators (an array of flagged spans with claim, severity, and evidence_status), and a recommended_action enum of accept, review, or reject. Validate that all required fields are present, types are correct, scores fall within expected ranges, and the recommended_action is consistent with the numeric score. If validation fails, retry the prompt once with the validation error message appended as context—but cap retries at two attempts to avoid infinite loops. Log every validation failure for later analysis; repeated schema violations often indicate prompt drift or model version changes.

Routing and gating logic should live in application code, not in the prompt. After validation passes, read the recommended_action field and route accordingly: accept outputs proceed to downstream systems, review outputs enter a human review queue with the full assessment payload attached, and reject outputs are discarded with a logged reason. For high-stakes domains such as healthcare, legal, or finance, consider overriding accept decisions with mandatory human review when the trustworthiness_score falls below a conservative threshold like 0.85, even if the prompt returned accept. This defense-in-depth approach prevents borderline outputs from reaching users without inspection. Store the full assessment payload alongside the original output in your logging or observability system so that routing decisions are auditable.

Model selection matters. This prompt performs best with models that have strong instruction-following and structured output capabilities. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are suitable defaults. Avoid using smaller or older models (e.g., GPT-3.5, Claude Haiku) for trustworthiness assessment because they are more likely to hallucinate within the assessment itself—producing confidence scores that are themselves untrustworthy. If cost or latency constraints require a smaller model, run a calibration evaluation comparing the small model's assessments against a larger judge model on a held-out dataset before deploying. Track score distribution drift across model versions; a sudden shift in average trustworthiness scores after a model update often signals changed behavior rather than changed output quality.

Human review integration requires careful context packaging. When an output is routed for review, the reviewer needs more than the raw assessment JSON. Include the original user input or query, the retrieved context or source documents (for RAG workflows), the full model output under assessment, and the structured assessment payload with highlighted hallucination indicators. Pre-format this into a review interface that lets the reviewer quickly accept, override, or amend the assessment. Track reviewer decisions against the automated assessment to measure false-positive and false-negative rates over time—this data is essential for tuning thresholds and detecting assessment drift.

What to avoid: Do not use this prompt as a real-time synchronous gate on the critical path of a user-facing chat application unless latency is carefully managed—the assessment call adds a second model inference that can double response time. For latency-sensitive applications, run the assessment asynchronously after the output is delivered, flagging issues for retrospective correction or user notification. Do not treat the trustworthiness score as a calibrated probability without running your own calibration evaluation on in-distribution data. Finally, never skip logging: every assessment decision, including the raw prompt request, model response, validation result, and routing action, should be recorded for debugging, audit, and continuous improvement of your quality gates.