Multi-Constraint Priority and Trade-Off Evaluation Prompt

This prompt is for evaluation leads, AI engineers, and QA teams who need to grade model outputs when a set of prioritized constraints cannot all be satisfied simultaneously. The core job-to-be-done is producing a structured, defensible report on which constraints were sacrificed, whether the model's trade-off was appropriate given the defined precedence rules, and an overall judgment of the output's quality. You should use this prompt when you have a list of ranked requirements, a model output that had to navigate conflicts between them, and a need for an audit trail explaining the model's prioritization behavior.

This is not a prompt for simple pass/fail checks on independent constraints. If your constraints are non-conflicting and can be evaluated in isolation, use the Multi-Constraint Instruction Adherence Grading Prompt instead. This prompt assumes you have already defined constraint precedence rules—either as an explicit ranking, a weighting scheme, or a policy document—and are now evaluating whether the model respected those rules when forced to choose. The prompt requires you to provide the original instructions given to the model, the prioritized constraint list with precedence rules, the model's output, and any context about the task's risk tolerance. Without clear precedence rules, the evaluation becomes subjective and the prompt's output will be unreliable.

Before using this prompt, ensure your constraint precedence is documented and agreed upon by stakeholders. The prompt will not invent priorities for you; it will only judge whether the model's trade-offs align with the rules you provide. For high-risk domains such as healthcare, legal, or finance, always include a human review step after the automated evaluation, as trade-off judgments can have downstream consequences that require domain expertise to validate.

This template is designed to be copied directly into your evaluation harness. It instructs a judge model to assess a candidate output against multiple constraints that may conflict, requiring the judge to identify which constraints were prioritized, which were sacrificed, and whether the trade-off was appropriate given the precedence rules you define. Replace every square-bracket placeholder with your specific constraints, precedence rules, and evaluation context before sending this to the judge model.

textCopy
You are an expert evaluation judge. Your task is to assess a candidate model output against multiple constraints that may conflict with each other. You must determine which constraints were satisfied, which were sacrificed, and whether the trade-off decisions were appropriate given the defined precedence rules.

## CONSTRAINTS AND PRECEDENCE RULES

Below are the constraints the model was required to follow, listed in priority order from highest to lowest. When constraints conflict, higher-priority constraints must be satisfied at the expense of lower-priority ones.

[CONSTRAINT_LIST_WITH_PRIORITIES]

Example format:
1. [HIGHEST_PRIORITY_CONSTRAINT] - Must be satisfied even if it breaks lower-priority constraints.
2. [MEDIUM_PRIORITY_CONSTRAINT] - Should be satisfied unless it conflicts with constraint 1.
3. [LOWEST_PRIORITY_CONSTRAINT] - Satisfy if possible without breaking higher-priority constraints.

## CANDIDATE OUTPUT TO EVALUATE

[CANDIDATE_OUTPUT]

codeCopy

## INPUT CONTEXT (if applicable)

[INPUT_CONTEXT]

## EVALUATION INSTRUCTIONS

1. For each constraint, determine whether it was SATISFIED, PARTIALLY SATISFIED, or VIOLATED.
2. For any constraint that was VIOLATED or PARTIALLY SATISFIED, identify whether the violation was:
   - JUSTIFIED: The constraint was sacrificed to satisfy a higher-priority constraint.
   - UNJUSTIFIED: The constraint was violated without a valid higher-priority reason.
   - AVOIDABLE: The constraint could have been satisfied without breaking higher-priority constraints.
3. Assess the overall trade-off quality:
   - APPROPRIATE: All trade-offs correctly followed the precedence rules.
   - MOSTLY APPROPRIATE: Minor deviations from ideal trade-offs, but overall priority order was respected.
   - INAPPROPRIATE: Higher-priority constraints were sacrificed for lower-priority ones, or unnecessary violations occurred.
4. Provide a final judgment: PASS or FAIL.
   - PASS: All highest-priority constraints are satisfied, and any violations are justified by the precedence rules.
   - FAIL: A highest-priority constraint was violated, or unjustified/avoidable violations occurred.

## OUTPUT FORMAT

Return a JSON object with this exact structure:

{
  "overall_judgment": "PASS" | "FAIL",
  "trade_off_quality": "APPROPRIATE" | "MOSTLY_APPROPRIATE" | "INAPPROPRIATE",
  "constraint_results": [
    {
      "constraint_id": "string",
      "priority_level": number,
      "description": "string",
      "status": "SATISFIED" | "PARTIALLY_SATISFIED" | "VIOLATED",
      "violation_type": "JUSTIFIED" | "UNJUSTIFIED" | "AVOIDABLE" | null,
      "explanation": "string explaining the assessment with specific evidence from the output"
    }
  ],
  "trade_off_analysis": "string summarizing which trade-offs were made and whether they respected the precedence rules",
  "failure_mode": "string describing the primary failure if judgment is FAIL, or null if PASS"
}

## ADDITIONAL GUIDANCE

- If the candidate output found a creative way to satisfy conflicting constraints simultaneously, note this in the trade-off analysis and treat it as a positive outcome.
- If the candidate output ignored all constraints entirely, mark all as VIOLATED with violation_type UNJUSTIFIED and judgment FAIL.
- If the precedence rules themselves contain ambiguity, note this in the trade-off analysis but still apply your best interpretation.
- Base your assessment strictly on the output provided. Do not speculate about what the model might have intended.

To adapt this template, replace [CONSTRAINT_LIST_WITH_PRIORITIES] with your actual constraints ordered by importance. Each constraint should be a clear, verifiable statement—avoid vague instructions like "be helpful" in favor of testable conditions like "response must include a citation for every factual claim." Replace [CANDIDATE_OUTPUT] with the text you are evaluating, and [INPUT_CONTEXT] with any user query, retrieved documents, or system context that the model received. If your use case requires different output fields, modify the JSON schema in the OUTPUT FORMAT section, but preserve the per-constraint status tracking and violation-type classification—these are essential for diagnosing whether failures were acceptable trade-offs or genuine errors.

Before deploying this prompt into an evaluation pipeline, test it against at least three hand-labeled examples: one where the model correctly prioritized constraints, one where it made an unjustified trade-off, and one edge case where constraints are in direct tension. Verify that the judge's PASS/FAIL decisions and violation-type classifications match your human judgments. If the judge consistently disagrees with human raters on borderline cases, refine the constraint descriptions or add examples of acceptable versus unacceptable trade-offs to the ADDITIONAL GUIDANCE section. For high-stakes evaluation contexts—such as regulated outputs, safety-critical decisions, or customer-facing content—always maintain a human review step for FAIL judgments before taking automated action.

This prompt is designed to run as a secondary evaluation step after a primary model generates its output. It should be wired into an evaluation pipeline, not a real-time user-facing flow. The typical harness pattern is: primary generation → output capture → constraint evaluation → compliance report → gating decision. The evaluation prompt receives the original instructions, the constraint precedence rules, and the generated output, then returns a structured compliance report that your application can parse and act on.

For implementation, wrap the prompt in a function that accepts primary_output, constraints (an ordered list with id, description, priority, and precedence_rule fields), and trade_off_policy (a string describing acceptable sacrifice rules). Validate inputs before calling the model: ensure constraints are non-empty, priorities are numeric and unique, and the primary output is a non-empty string. On the model response, parse the JSON output and validate the schema—check that overall_judgment is one of PASS, FAIL, or CONDITIONAL_PASS, that each constraint has a compliance field with FULL, PARTIAL, or FAILED, and that any PARTIAL or FAILED constraint includes a trade_off_rationale. If parsing fails, retry once with a repair prompt that includes the raw response and the expected schema. Log every evaluation result with the prompt version, model used, timestamp, and constraint set for auditability.

Model choice matters here. Use a model with strong instruction-following and structured output capabilities—GPT-4o, Claude 3.5 Sonnet, or equivalent. Avoid smaller models that may conflate priority levels or produce inconsistent JSON. For high-stakes domains like healthcare or finance, route CONDITIONAL_PASS and FAIL results to a human review queue before blocking deployment. Store evaluation results alongside the primary output in your observability system so you can track constraint compliance drift over time. Never use this prompt as the sole gate for safety-critical constraints without human-in-the-loop confirmation.