Budget-Conscious Multi-Model Consensus Prompt

This prompt is for infrastructure engineers and AI platform architects who need to make a data-driven decision about consensus strategy before committing to production routing logic. The core job-to-be-done is answering the question: 'Given my candidate request, available models with per-token pricing, historical agreement rates, and error correlation data, should I route to a single model, run dual-model verification, or invoke a full multi-model consensus pattern?' The prompt forces a structured tradeoff analysis rather than relying on intuition or blanket rules like 'always use the best model.' It is designed for offline analysis and routing strategy configuration, not for inline per-request decisioning under tight latency budgets.

Use this prompt when you have a representative sample of requests, a catalog of candidate models with accurate per-token pricing (input and output), and historical data on how often those models agree or disagree on the target task. You also need error correlation data—do the models tend to fail on the same inputs or different ones? Low error correlation makes consensus more valuable because models catch each other's mistakes. High error correlation means you're paying for redundant failures. The prompt expects these inputs as structured variables: [MODEL_CATALOG] with pricing, [AGREEMENT_MATRIX] showing pairwise agreement rates, [ERROR_CORRELATION] data, and [ACCURACY_THRESHOLD] defining the minimum acceptable accuracy for the use case. Without these inputs, the prompt will produce speculative output that shouldn't drive production routing decisions.

Do not use this prompt for real-time per-request routing where the consensus analysis itself adds latency that violates an SLA. The prompt is designed for pre-routing analysis: you run it once per request class, workload pattern, or routing rule, then bake the resulting strategy into your routing middleware. Do not use it when you lack historical agreement data—the output quality degrades to guesswork. Do not use it for latency-critical paths where the overhead of calling multiple models in parallel or sequence exceeds your p95 latency target. For those cases, use the Latency-SLA Classification Prompt or the Cost-Budget-Aware Router System Prompt instead. After running this analysis, the next step is to encode the chosen strategy into your routing configuration, set up cost monitoring dashboards, and establish eval gates that verify the accuracy improvement actually materializes in production.

This prompt template is designed to be invoked at the routing layer before committing to a multi-model consensus strategy. It takes a request profile, model catalog with pricing, historical agreement data, and accuracy improvement thresholds as inputs, then returns a structured decision on whether to use a single model or pay for consensus. The output includes a clear recommendation, cost delta, expected accuracy gain, and the reasoning chain so the decision is auditable.

textCopy
You are a cost-aware routing decision engine for an AI infrastructure platform. Your job is to determine whether invoking multiple models for consensus on a given request is justified by the expected accuracy improvement relative to the additional cost.

## INPUTS

### Request Profile
- Request Type: [REQUEST_TYPE]
- Input Token Count (estimated): [INPUT_TOKENS]
- Expected Output Token Count: [OUTPUT_TOKENS]
- Risk Level: [RISK_LEVEL] // one of: low, medium, high, critical
- Latency Budget (ms): [LATENCY_BUDGET_MS]

### Model Catalog
[MODEL_CATALOG]
// Array of available models with fields: model_id, cost_per_1k_input_tokens, cost_per_1k_output_tokens, p50_latency_ms, p95_latency_ms, quality_score_0_to_1

### Consensus Configuration
- Primary Model: [PRIMARY_MODEL_ID]
- Consensus Models: [CONSENSUS_MODEL_IDS] // array of model IDs to use for voting
- Consensus Strategy: [CONSENSUS_STRATEGY] // one of: majority_vote, unanimous, weighted_vote, primary_with_override
- Agreement Rate (historical): [AGREEMENT_RATE] // 0.0 to 1.0, how often these models agree on this request type
- Error Correlation (historical): [ERROR_CORRELATION] // 0.0 to 1.0, how often models make the same mistake

### Accuracy Thresholds
- Minimum Accuracy Improvement Required (%): [MIN_ACCURACY_IMPROVEMENT_PCT]
- Single Model Baseline Accuracy (%): [BASELINE_ACCURACY_PCT]
- Consensus Expected Accuracy (%): [CONSENSUS_ACCURACY_PCT]

### Budget Constraints
- Per-Request Cost Cap ($): [PER_REQUEST_COST_CAP]
- Daily Spend Remaining ($): [DAILY_SPEND_REMAINING]
- Cost of Error (estimated $): [COST_OF_ERROR] // what a wrong answer costs the business

## OUTPUT SCHEMA

Return a JSON object with the following structure:
{
  "decision": "use_consensus" | "use_single_model" | "conditional",
  "recommended_strategy": string,
  "cost_analysis": {
    "single_model_cost": number,
    "consensus_cost": number,
    "cost_delta": number,
    "cost_delta_pct": number,
    "within_per_request_cap": boolean,
    "daily_budget_impact_pct": number
  },
  "accuracy_analysis": {
    "baseline_accuracy_pct": number,
    "consensus_expected_accuracy_pct": number,
    "accuracy_improvement_pct": number,
    "meets_minimum_threshold": boolean
  },
  "latency_analysis": {
    "single_model_p95_ms": number,
    "consensus_p95_ms": number,
    "within_latency_budget": boolean
  },
  "expected_value": {
    "error_cost_savings": number,
    "net_benefit": number,
    "roi_ratio": number
  },
  "risk_factors": [string],
  "conditions": [string],
  "reasoning": string
}

## DECISION RULES

1. If accuracy_improvement_pct < MIN_ACCURACY_IMPROVEMENT_PCT, recommend single_model unless risk_level is critical.
2. If consensus_cost > PER_REQUEST_COST_CAP, recommend single_model regardless of accuracy gain.
3. If daily_spend_remaining < (consensus_cost * 10), recommend single_model to preserve budget.
4. If error_correlation > 0.7, discount consensus accuracy gain by 50% — correlated errors reduce voting benefit.
5. If consensus_p95_ms > LATENCY_BUDGET_MS, recommend single_model or a faster consensus subset.
6. If risk_level is critical and net_benefit is positive, recommend consensus even if accuracy improvement is marginal.
7. If agreement_rate > 0.95, consensus adds cost with minimal benefit — recommend single_model.

## CONSTRAINTS
- Use only the models listed in MODEL_CATALOG.
- Calculate costs using the provided per-token pricing and estimated token counts.
- If any required input is missing, flag it in risk_factors and return decision: "conditional" with conditions specifying what's needed.
- Round all monetary values to 4 decimal places.
- Round all percentages to 1 decimal place.
- Include specific numbers in reasoning, not vague statements.

To adapt this prompt for your environment, replace each square-bracket placeholder with runtime data from your routing middleware. The model catalog should be populated from your model registry or configuration store. Historical agreement rates and error correlations should come from your observability pipeline — do not hardcode these. Wire the output JSON into a decision switch in your router: use_consensus triggers parallel model calls with your voting logic, use_single_model sends the request to the primary model only, and conditional should pause routing and surface the conditions to an operator or fallback policy. Before deploying, run this prompt against a golden dataset of known routing decisions and measure whether the cost-benefit calculations match your offline analysis within a 5% tolerance. If the prompt consistently overestimates consensus value, adjust the error correlation discount rule or tighten the minimum accuracy improvement threshold.

This prompt is designed to be called by a routing middleware or a decision service, not directly by an end user. It receives structured data about a pending request—including model costs, agreement rates, error correlation data, and accuracy improvement thresholds—and returns a structured consensus strategy decision. The calling application is responsible for assembling the input context from live telemetry, cost databases, and historical evaluation results. The prompt itself should never have direct access to billing systems or production databases; instead, the harness should query those systems, format the relevant data into the [MODEL_COST_TABLE], [AGREEMENT_MATRIX], and [ERROR_CORRELATION_DATA] placeholders, and then invoke the prompt.

The harness must enforce a strict validation contract on the model's output. Parse the response against the expected [OUTPUT_SCHEMA]—which should require fields like consensus_strategy (e.g., single_model, majority_vote, unanimous, weighted_vote), selected_models, estimated_cost, expected_accuracy_gain, and justification. If the output fails schema validation, retry once with a repair prompt that includes the validation error. If the retry also fails, fall back to a safe default strategy defined in the harness (e.g., route to the single cheapest model that meets the minimum accuracy threshold). Log every decision, including the input parameters, the model's raw output, validation results, and the final routed strategy, for audit and cost attribution. For high-stakes domains, add a human approval gate before executing any consensus strategy that exceeds a configurable cost threshold.

Cost tracking is the critical non-functional requirement here. The harness must calculate the actual cost of executing the chosen consensus strategy after the models run and compare it against the prompt's estimated_cost. If actual cost exceeds the estimate by more than a configurable margin (e.g., 20%), flag the decision for review and update the cost estimation parameters. Similarly, track whether the expected_accuracy_gain materializes by comparing outcomes against a held-out evaluation set or production metrics. Use these observations to periodically recalibrate the cost and accuracy data fed into the prompt. Avoid wiring this prompt into a synchronous user-facing request path without a tight timeout; multi-model consensus calls can add significant latency, so the harness should enforce a deadline and fall back to the single-best-model strategy if the consensus workflow cannot complete in time.