Production Output Gating Rubric Prompt Template

This prompt is for engineering leads and ML platform teams who need a consistent, automated way to decide whether an AI-generated output is good enough to pass to a downstream system or user. Instead of ad-hoc spot checks, this prompt produces a weighted, multi-criteria rubric that scores outputs against your specific quality dimensions. Use it when you need a repeatable gating decision with an audit trail, not a simple pass/fail flag. It assumes you already have a validated output format and are now defining the acceptance criteria that sit between generation and delivery.

The ideal workflow starts after you have a stable output schema and a set of example outputs that represent both good and bad cases. You provide the prompt with your quality dimensions (e.g., factuality, completeness, safety, format compliance), their relative weights, and concrete threshold definitions. The prompt returns a structured rubric you can embed directly into an evaluation harness. This is not a prompt for initial schema design or for fixing malformed JSON—those belong to the Output Repair and Validation pillar. It is also not a replacement for human review in high-risk domains; the rubric should include an escalation path for outputs that fall into a review band.

Do not use this prompt when you lack clear quality definitions or when your output format is still unstable. If you cannot articulate what 'good' looks like in measurable terms, the rubric will be too vague to gate reliably. Also avoid this prompt for real-time streaming outputs where latency constraints prevent running a full evaluation pass before delivery—in those cases, consider a lightweight confidence check or post-hoc sampling instead. Finally, this prompt is not a substitute for regression testing against a golden dataset; use it to define the criteria, then validate the rubric's stability across your output distribution using the eval checks described in the implementation harness.

This prompt template acts as a programmable quality gate. You provide the original task, the output to evaluate, and a set of weighted quality dimensions. The prompt instructs the model to act as a strict, consistent evaluator, scoring each dimension on a 1-4 scale and applying configurable gating rules. The output is a single JSON object containing the final decision, a total weighted score, and detailed justifications for every criterion. This is designed to be wired directly into an application's post-generation validation step, not used for one-off manual reviews.

textCopy
Act as a production quality gate engineer. Your task is to evaluate the provided AI-generated output against a structured rubric and produce a final gating decision. You must be strict, consistent, and provide a detailed justification for every score.

## Evaluation Criteria
Apply the following weighted criteria to the [OUTPUT_TO_EVALUATE]. The total weight must sum to 100.

[QUALITY_DIMENSIONS]

## Scoring Rubric
For each criterion, assign a score from 1 to 4 based on these definitions:
1 (Unacceptable): The output completely fails this criterion. Critical errors present.
2 (Needs Improvement): The output partially meets the criterion but has significant flaws.
3 (Meets Expectations): The output adequately meets the criterion with minor or no issues.
4 (Exceeds Expectations): The output is exemplary for this criterion.

## Context
Original Task/User Prompt: [ORIGINAL_PROMPT]

## Gating Rules
- If any criterion with a weight >= [CRITICAL_WEIGHT_THRESHOLD] scores a 1, the output must be REJECTED.
- If the total weighted score is below [PASS_THRESHOLD], the output must be REJECTED.
- If the output is REJECTED, provide a specific, actionable reason for each failing criterion.
- If the output is ACCEPTED, note any minor issues that should be monitored.

## Output Format
Return a single valid JSON object with the following structure:
{
  "decision": "ACCEPTED" | "REJECTED",
  "total_weighted_score": <number>,
  "criteria_scores": [
    {
      "criterion_name": "<string>",
      "score": <1-4>,
      "weight": <number>,
      "justification": "<string>"
    }
  ],
  "rejection_reasons": ["<string>"],
  "minor_issues": ["<string>"]
}

To adapt this template, replace the square-bracket placeholders with your specific context. [QUALITY_DIMENSIONS] should be a markdown table or list defining each criterion (e.g., 'Factual Accuracy', 'Tone Compliance') and its weight. [CRITICAL_WEIGHT_THRESHOLD] and [PASS_THRESHOLD] are integers (e.g., 30 and 70) that define the gating logic. The [ORIGINAL_PROMPT] provides the evaluator with the necessary context for what the AI was asked to do. For high-risk domains like healthcare or finance, always set [CRITICAL_WEIGHT_THRESHOLD] to a low value and route REJECTED outputs for mandatory human review before any automated retry or fallback logic is triggered.

This prompt is designed to be called as a secondary model inference after your primary generation step. In your application code, construct the prompt by injecting the variables, then call a fast, cost-effective model like GPT-4o-mini or Claude Haiku. Parse the JSON response and use the decision field to route the output. If REJECTED, log the full payload, including rejection_reasons, to your observability platform and either retry the generation step with more context, fall back to a safe default, or escalate to a human review queue. Implement a circuit breaker: if the rejection rate for a specific prompt or model exceeds a threshold (e.g., 20% over 5 minutes), automatically trigger an alert and pause the pipeline. Never expose the raw rubric scores to end users; they are for internal gating and debugging.

Validation must happen at two levels. First, validate the JSON structure of the rubric response itself—ensure decision is one of the allowed enum values (ACCEPT, REJECT, REVIEW), overall_score is a float between 0.0 and 1.0, and criteria_scores is a dictionary with the expected keys matching your rubric dimensions. Second, validate that the decision is consistent with the scores: if overall_score falls below your configured threshold but the decision is ACCEPT, flag this as a schema-logic mismatch and route to REVIEW. Store the raw rubric payload alongside the original output in your evaluation database for offline calibration analysis. This dual validation prevents silent gating failures where the model produces valid JSON but logically inconsistent decisions.

For high-stakes domains—healthcare, legal, finance, safety-critical systems—never allow an ACCEPT decision to bypass human review without additional safeguards. Configure the harness to force REVIEW for any output touching regulated data, regardless of the rubric score. Log every gating decision with a unique gate_id, the model version, the prompt template version, and a timestamp. This audit trail is essential for compliance reviews and for debugging threshold drift over time. When routing to human review, include the original output, the full rubric breakdown, and a pre-formatted review interface payload so reviewers can act quickly without reconstructing context. Measure reviewer agreement rates against the automated rubric to continuously tune your thresholds and criteria weights.