Multi-Signal Release Confidence Score Prompt

This prompt is for platform engineering teams building automated release dashboards and decision-support systems. The job-to-be-done is aggregating heterogeneous release signals—test pass rates, code review sentiment, static analysis findings, performance benchmark deltas, dependency vulnerabilities, and deployment history—into a single, calibrated confidence score with a transparent breakdown of contributing factors. The ideal user is a release manager or SRE who needs a dashboard-ready summary that answers 'how confident should I be in this release candidate?' without manually synthesizing a dozen disjointed reports.

Use this prompt when you have structured signal data available and need a weighted, evidence-backed confidence score that can be consumed by a dashboard, a go/no-go gate, or an audit trail. The prompt expects pre-processed inputs: it does not run tests, query databases, or call external APIs itself. You must supply [TEST_RESULTS], [STATIC_ANALYSIS_FINDINGS], [PERFORMANCE_BENCHMARKS], [DEPENDENCY_HEALTH], [CODE_REVIEW_SUMMARY], and [HISTORICAL_OUTCOME_DATA] as structured inputs. The output is a JSON object containing a confidence_score (0-100), a recommendation enum, a factor_breakdown array with per-signal scores and weights, and an evidence_summary for drill-down. This prompt is not a replacement for human judgment in high-risk deployments; it is a synthesis tool that makes the evidence legible and the decision faster.

Do not use this prompt when signals are missing, unvalidated, or when the release decision requires real-time production metrics that the prompt cannot access. It is also unsuitable for the final go/no-go call in regulated environments without a human approval step—the prompt produces a recommendation, not an authorization. Before wiring this into a release gate, calibrate the scoring against at least 10 historical releases where the outcome is known, and validate that the confidence score correlates with successful deployments. If the calibration shows poor correlation, revisit the signal weights or add missing signals before relying on the output in production.

This prompt template is designed to be dropped into an automated release dashboard or CI/CD pipeline stage. It expects pre-processed signal summaries as input and produces a structured JSON output suitable for dashboard rendering, automated gating, or human review. The square-bracket placeholders represent the dynamic data your system must inject before calling the model. Do not pass raw logs or unsummarized data; the prompt works best when each signal is already reduced to a structured summary with counts, severities, and trends.

textCopy
You are a release confidence scoring engine. Your job is to aggregate multiple deployment readiness signals into a single confidence score (0-100) with contributing factor weights and drill-down evidence. You must be conservative: when signals conflict or data is missing, reduce confidence and flag the uncertainty.

## INPUT SIGNALS
[TEST_RESULTS_SUMMARY]
[STATIC_ANALYSIS_SUMMARY]
[CODE_REVIEW_SUMMARY]
[PERFORMANCE_BENCHMARK_SUMMARY]
[DEPENDENCY_HEALTH_SUMMARY]
[CONFIGURATION_DRIFT_SUMMARY]
[HISTORICAL_RELEASE_OUTCOMES]

## OUTPUT SCHEMA
Return valid JSON matching this structure exactly:
{
  "confidence_score": <integer 0-100>,
  "recommendation": "<GO | CONDITIONAL_GO | NO_GO | INSUFFICIENT_DATA>",
  "factor_weights": {
    "test_results": <float 0.0-1.0>,
    "static_analysis": <float 0.0-1.0>,
    "code_review": <float 0.0-1.0>,
    "performance_benchmarks": <float 0.0-1.0>,
    "dependency_health": <float 0.0-1.0>,
    "configuration_drift": <float 0.0-1.0>,
    "historical_patterns": <float 0.0-1.0>
  },
  "blocking_findings": [
    {
      "signal_source": "<string>",
      "finding": "<string>",
      "severity": "<BLOCKER | HIGH | MEDIUM | LOW>",
      "recommended_action": "<string>"
    }
  ],
  "uncertainty_flags": [
    {
      "signal": "<string>",
      "issue": "<missing_data | conflicting_signals | stale_data | below_threshold>",
      "detail": "<string>"
    }
  ],
  "calibration_notes": "<string explaining how historical outcomes informed this score, or 'No historical data available'>",
  "drill_down_summary": "<string: 2-4 sentence executive summary of the strongest signals and biggest risks>"
}

## CONSTRAINTS
- If any signal has a BLOCKER finding, recommendation must be NO_GO or CONDITIONAL_GO with explicit conditions.
- If more than 2 signals are missing or stale, recommendation must be INSUFFICIENT_DATA.
- Factor weights must sum to 1.0. Distribute weight based on signal quality, recency, and historical predictive power.
- Confidence score must reflect both signal strength and signal coverage. Missing signals reduce the ceiling.
- Do not invent data. If a signal summary is empty or marked 'UNAVAILABLE', set its weight to 0.0 and flag it in uncertainty_flags.
- Calibration notes must reference specific patterns from [HISTORICAL_RELEASE_OUTCOMES] when available, not generic statements.

## RISK LEVEL
[RISK_LEVEL: HIGH | MEDIUM | LOW]
- If HIGH: require explicit human review before any automated action. Flag all MEDIUM findings for attention.
- If MEDIUM: automated gating is permitted but CONDITIONAL_GO requires human acknowledgment.
- If LOW: automated gating is permitted for GO and CONDITIONAL_GO recommendations.

## EXAMPLES
[FEW_SHOT_EXAMPLES]

Adaptation guidance: Replace each bracketed placeholder with structured data from your pipeline. [TEST_RESULTS_SUMMARY] should contain pass/fail counts, coverage percentages, and any flaky test annotations—not raw JUnit XML. [HISTORICAL_RELEASE_OUTCOMES] should include the last 5-10 releases with their signal profiles and whether they succeeded or were rolled back. The [FEW_SHOT_EXAMPLES] placeholder is critical for calibration: include 2-3 examples showing how specific signal combinations map to confidence scores, drawn from your team's actual release history. If you lack historical data, remove the calibration requirement and note that the model will operate without historical grounding. For high-risk deployments, always set [RISK_LEVEL] to HIGH and route the output through a human approval step before any automated gate acts on the recommendation.

This prompt is designed to be the final aggregation step in a release dashboard, not a standalone chatbot. It expects pre-processed, structured signals from upstream systems—test suites, static analysis tools, performance benchmarks, dependency scanners, and code review platforms. The implementation harness must collect, normalize, and format these signals into the [SIGNALS] placeholder before the prompt is invoked. Do not pass raw logs or unstructured tool output directly to the model; the prompt's value depends on receiving clean, comparable signal objects with consistent severity levels, timestamps, and coverage metadata.

Wire the prompt as a synchronous API call within a release gate service. The service should: (1) fetch signal data from upstream sources via their APIs or a centralized release event bus, (2) transform each signal into the required JSON schema with fields for signal_type, severity, confidence, source, timestamp, and evidence_summary, (3) inject the assembled [SIGNALS] array and the optional [HISTORICAL_OUTCOMES] calibration data into the prompt template, (4) call the model with response_format set to JSON and a strict schema matching the expected confidence_score, factor_weights, risk_breakdown, and drill_down_evidence output structure, (5) validate the response against that schema and check that all confidence scores fall within 0.0–1.0 and all weights sum to approximately 1.0, and (6) log the full prompt, response, and validation result for auditability. On schema validation failure, retry once with a repair prompt that includes the validation error; if the retry also fails, escalate to a human release manager and emit a confidence_score: null with an error field in the dashboard payload.

Model choice matters here. Use a model with strong JSON mode and reasoning capabilities, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Avoid smaller or older models that may hallucinate weights or fail to maintain arithmetic consistency across factor contributions. If you have historical release outcome data (deployments with known success/failure labels and their corresponding signal profiles), include it in [HISTORICAL_OUTCOMES] to improve calibration. Without historical data, the model will rely on its pre-trained risk heuristics, which may not match your organization's risk tolerance. Plan to collect this data over time and periodically re-evaluate the prompt's calibration against actual outcomes. The output should feed directly into a dashboard UI that displays the overall confidence score, a weighted factor breakdown as a bar or radar chart, and expandable drill-down evidence for each signal. Never use the raw model output to automatically block or approve a release without human review; the confidence score is a decision-support artifact, not an automated gate actuator.