Regression Risk Assessment Prompt for Multi-File Edits

This prompt is designed for engineering leads and platform teams who need a pre-merge safety signal for proposed multi-file changes. The core job-to-be-done is to produce a structured risk assessment that identifies test coverage gaps, untested code paths, and files with high historical churn that intersect with the change. You should use this prompt when you have a concrete diff or a list of changed files and access to repository context, including test files, coverage data, and version control history. The output is a risk-scored report that helps reviewers decide whether a change needs additional testing, a staged rollout, or a deeper manual review. It is not a replacement for running the actual test suite or for human judgment on architectural risk; it augments existing CI checks by providing a focused, evidence-backed safety signal.

To use this prompt effectively, you must provide it with a structured [DIFF] or [CHANGED_FILES] list, along with [REPOSITORY_CONTEXT] that includes a mapping of test files to source files, recent coverage reports, and git log output showing historical churn for the affected paths. The model will cross-reference the changed lines with existing test coverage to flag untested code paths and will weigh the historical volatility of each file to escalate risk. For example, a one-line change in a file with 90% test coverage and low churn will score differently than a multi-function refactor in a file with 40% coverage and a history of frequent post-release patches. The prompt's constraints should enforce that every risk flag is tied to a specific file and line range, preventing vague or unactionable warnings.

Do not use this prompt for changes where the primary risk is architectural or design-level, such as introducing a new anti-pattern or violating a subsystem boundary. It is also unsuitable for greenfield code in new files with no historical churn data or for runtime configuration changes where the risk is environmental rather than code-level. After receiving the assessment, your next step should be to correlate the high-risk findings with your team's manual review checklist and to gate the merge until the flagged gaps are addressed, either through additional tests or a documented acceptance of the risk. Avoid treating the model's risk score as a binary pass/fail; it is a prioritization tool for human reviewers.

This prompt template is designed to be copied directly into your AI harness, IDE, or agent workflow. It accepts a proposed change description, a set of modified file paths, and repository context to produce a structured risk report. Every placeholder in square brackets must be replaced with real data from your repository before sending the prompt to the model. The prompt is structured to force the model to reason about test gaps, historical churn, and untested code paths rather than producing a generic confidence score.

textCopy
You are a regression risk assessor for a production codebase. Your job is to evaluate a proposed multi-file change and produce a structured risk report. Do not approve or reject the change. Identify specific risks, test gaps, and files that require additional scrutiny.

## INPUT

**Change Description:**
[CHANGE_DESCRIPTION]

**Files Modified (with diff summaries):**
[MODIFIED_FILES_WITH_DIFFS]

**Repository Context:**
- Dependency graph for affected modules: [DEPENDENCY_GRAPH]
- Test coverage report for modified files: [TEST_COVERAGE_REPORT]
- Historical churn data (commits per file, last 90 days): [HISTORICAL_CHURN_DATA]
- Recent regression incidents related to these modules: [RECENT_REGRESSION_INCIDENTS]
- Static analysis findings for modified files: [STATIC_ANALYSIS_FINDINGS]

## CONSTRAINTS

- Do not speculate about code you cannot see. If evidence is missing, flag it as an unknown risk.
- Distinguish between high-confidence findings (directly observable in diffs or data) and low-confidence inferences.
- If a file has high historical churn and is touched by this change, flag it regardless of apparent change size.
- Identify specific untested code paths by referencing function names, conditionals, or error handlers visible in the diff.
- Do not produce a single numeric risk score without explaining its components.

## OUTPUT SCHEMA

Return a JSON object with this exact structure:

{
  "overall_risk_level": "low" | "medium" | "high" | "critical",
  "risk_summary": "string (2-4 sentence executive summary)",
  "findings": [
    {
      "severity": "critical" | "high" | "medium" | "low",
      "category": "untested_code_path" | "high_churn_file" | "breaking_contract_change" | "ordering_dependency" | "missing_rollback_path" | "static_analysis_flag" | "regression_pattern_match" | "unknown_risk",
      "file": "string (relative path)",
      "location": "string (function name, line range, or symbol)",
      "description": "string (specific, actionable finding)",
      "evidence": "string (what in the diff, coverage data, or churn data supports this finding)",
      "suggested_mitigation": "string (concrete action: add test, add rollback check, split commit, manual review, etc.)"
    }
  ],
  "test_gaps": [
    {
      "file": "string",
      "untested_path": "string (specific code path or condition not covered by existing tests)",
      "risk_if_untested": "string (what could break if this path is wrong)"
    }
  ],
  "high_churn_intersections": [
    {
      "file": "string",
      "recent_commits": "number",
      "reason_for_concern": "string (why high churn plus this change is risky)"
    }
  ],
  "unknowns": [
    {
      "description": "string (what we cannot determine from available evidence)",
      "impact_if_wrong": "string (what happens if this unknown turns out to be a problem)"
    }
  ],
  "recommended_actions": [
    {
      "priority": 1,
      "action": "string (concrete, ordered step)",
      "owner_hint": "string (suggested role: author, reviewer, QA, SRE)"
    }
  ]
}

## EXAMPLES

**Example Finding (high confidence):**
{
  "severity": "high",
  "category": "untested_code_path",
  "file": "src/payment/refund_handler.py",
  "location": "RefundHandler.process_partial_refund(), lines 142-158",
  "description": "New partial refund logic adds a branch for negative tax amounts. No test covers this branch.",
  "evidence": "Diff adds `if tax_amount < 0:` at line 142. Coverage report shows 0% branch coverage for process_partial_refund.",
  "suggested_mitigation": "Add unit test for negative tax amount scenario before merging."
}

**Example Unknown:**
{
  "description": "Cannot determine if downstream service `inventory-svc` consumes the changed field `refund_reason_code`.",
  "impact_if_wrong": "If inventory-svc expects the old enum values, partial refunds will fail silently in production."
}

To adapt this prompt for your workflow, replace each placeholder with data from your repository tooling. The [DEPENDENCY_GRAPH] can come from tools like madge, dependency-cruiser, or your build system. The [TEST_COVERAGE_REPORT] should be a machine-readable coverage export (Istanbul, coverage.py, JaCoCo). The [HISTORICAL_CHURN_DATA] can be generated from git log --format=format: --name-only aggregated over 90 days. The [RECENT_REGRESSION_INCIDENTS] should reference your incident tracker or postmortem docs. If any data source is unavailable, replace the placeholder with "Not available" so the model can flag it as an unknown risk rather than hallucinating evidence. For high-risk repositories, always route findings with severity: "critical" to a human reviewer before allowing an automated merge decision.

This prompt is designed to operate as a pre-merge safety gate within a CI/CD pipeline or an AI-assisted code review system. The implementation harness must treat the model's output as a structured risk signal—not a final decision. The primary integration points are: (1) a diff ingestion step that collects the proposed multi-file change, (2) a context assembly step that gathers historical churn data, test coverage reports, and recent incident logs, (3) the model call itself, and (4) a post-processing step that validates the output schema, compares risk scores against configurable thresholds, and routes high-risk assessments for human review. The harness should never block a merge solely on a model-generated risk score without a human-in-the-loop confirmation step for high-severity findings.

Input assembly requires three data sources wired into the prompt's [CHANGE_DIFF], [FILE_CHURN_HISTORY], and [TEST_COVERAGE_REPORT] placeholders. The diff should be a unified diff of all files in the proposed change, truncated per file if the total exceeds the model's context window—prioritize files with the highest historical churn first. Churn history should be extracted from git log --format='%h %an %ad %s' --follow for each changed file over a configurable lookback window (default: 90 days). Test coverage data should come from your coverage tool's JSON or XML report, filtered to only the files in the change set. If coverage data is unavailable, the [TEST_COVERAGE_REPORT] placeholder must be replaced with an explicit NO_COVERAGE_DATA_AVAILABLE marker so the model can flag this as an elevated risk factor rather than hallucinating coverage gaps. Output validation must enforce the JSON schema defined in the prompt: check that risk_score is an integer between 0–100, that risk_factors is a non-empty array of strings, that untested_code_paths contains valid file paths from the diff, and that recommended_tests items include both a file and test_description field. Reject and retry any response that fails schema validation, up to a maximum of 2 retries before escalating to a human reviewer with the raw model output attached.

Model choice and latency budgeting are critical for CI integration. Use a model with strong structured output capabilities and low latency for this synchronous gate—GPT-4o or Claude 3.5 Sonnet are appropriate defaults. Set a total timeout of 30 seconds for the model call plus validation. If the model call times out or exhausts retries, the harness should log the failure, emit a REG_RISK_ASSESSMENT_FAILED signal, and fall back to a conservative gate behavior (e.g., require human approval for all changes touching files with above-median churn). Logging and observability must capture: the prompt version hash, the truncated diff size in tokens, the raw model response, the validated risk score, any schema validation failures, and the final gate decision (auto-approve, warn, block-for-review). These logs feed into the eval pipeline described in the Evaluation section, enabling comparison of predicted risk scores against actual regression incidents discovered post-merge. Human approval routing should trigger when risk_score >= 70 or when any risk_factor contains a CRITICAL severity tag. The approval UI must display the model's risk report alongside a diff view and a one-click option to override, request more tests, or approve with a required justification comment that is stored in the audit log.