Codebase Audit Diff and Trend Analysis Prompt

This prompt is designed for engineering managers, tech leads, and platform engineers who run recurring, automated codebase audits and need to track health trends over time. The core job-to-be-done is producing a normalized, reviewable diff report that compares a current audit run against a historical baseline. It flags regressions, highlights improvements, and calculates a net trend direction for each audited dimension, such as complexity, duplication, or security posture. The ideal user already has tooling that emits structured, machine-readable findings with stable identifiers and wants a consistent narrative layer before presenting results to leadership or filing remediation tickets.

To use this prompt effectively, you must provide two structured audit payloads: a [CURRENT_AUDIT] and a [BASELINE_AUDIT]. Each payload must contain a list of findings with stable, cross-run identifiers (e.g., a rule ID plus a file path hash) that allow the model to match the same logical issue across runs. Without stable identifiers, the model cannot reliably distinguish a new finding from a pre-existing one, leading to false regressions. The prompt also requires a [DIMENSION_MAP] that groups finding types into the health dimensions you care about, such as security, performance, or maintainability. The output is a structured trend report with a per-dimension summary, a net trend indicator (improving, degrading, or stable), and a detailed list of new, resolved, and changed findings. This structure is designed to be consumed by a downstream dashboard or a human reviewer, not as a raw chat response.

Do not use this prompt for a first-time audit where no baseline exists; the diff operation requires two complete runs. It is also unsuitable for real-time CI/CD gating because the analysis is designed for periodic, batch comparison rather than per-commit latency. Avoid using this prompt when audit findings lack stable identifiers, as the normalization step will produce unreliable matches and misleading trends. Finally, this prompt is not a replacement for the audit tooling itself—it assumes findings are already generated and focuses solely on comparison, normalization, and trend interpretation. For high-risk codebases where a false negative could lead to a production incident, always route the final trend report through a human review step before accepting remediation decisions.

This prompt template is designed to be the core instruction set for an AI coding agent performing a diff-and-trend analysis on a codebase audit. It expects structured input representing a current audit run and a prior baseline audit run. The prompt instructs the model to act as a principal engineer, normalizing findings across runs, identifying statistically significant shifts, and producing a structured trend report. The placeholders within the template must be populated by your audit pipeline before the prompt is sent to the model.

markdownCopy
You are a principal engineer analyzing the health trajectory of a codebase. Your task is to compare a [CURRENT_AUDIT_REPORT] against a [BASELINE_AUDIT_REPORT] and produce a structured trend analysis.

**Input Data:**
- Current Audit Report: [CURRENT_AUDIT_REPORT]
- Baseline Audit Report: [BASELINE_AUDIT_REPORT]
- Comparison Dimensions: [COMPARISON_DIMENSIONS]
- Significance Threshold: [SIGNIFICANCE_THRESHOLD]

**Instructions:**
1.  **Normalize Findings:** Map findings from both reports to the specified [COMPARISON_DIMENSIONS]. If a finding in one report has no clear counterpart in the other, note it as a new or resolved issue.
2.  **Calculate Deltas:** For each dimension, calculate the change in the number of findings, their aggregated severity, and any other relevant metrics provided in the reports.
3.  **Identify Trends:** Flag dimensions where the change exceeds the [SIGNIFICANCE_THRESHOLD] as a significant regression or improvement. Ignore minor fluctuations.
4.  **Synthesize Narrative:** Write a concise executive summary that explains the overall health trajectory, the most critical regressions, and the most impactful improvements.
5.  **Generate Recommendations:** Based on the regressions, propose 2-3 concrete, prioritized remediation actions.

**Output Format:**
Return a single JSON object conforming to this schema:
{
  "executive_summary": "string",
  "trend_verdict": "IMPROVING" | "STABLE" | "DEGRADING",
  "significant_changes": [
    {
      "dimension": "string",
      "direction": "REGRESSION" | "IMPROVEMENT",
      "delta_description": "string",
      "current_value": "number",
      "baseline_value": "number"
    }
  ],
  "new_issues": ["string"],
  "resolved_issues": ["string"],
  "recommended_actions": [
    {
      "priority": "P0" | "P1" | "P2",
      "action": "string",
      "rationale": "string"
    }
  ]
}

**Constraints:**
- Do not hallucinate findings not present in the input reports.
- If the data is insufficient for a dimension, set its value to null and note it in the executive summary.
- Base all recommendations strictly on the regressions identified in the `significant_changes` array.

To adapt this template, start by defining the schema for your [CURRENT_AUDIT_REPORT] and [BASELINE_AUDIT_REPORT]. These should be structured objects, not free-text, containing arrays of findings tagged by dimension and severity. The [COMPARISON_DIMENSIONS] placeholder should be replaced with a JSON array of strings representing the categories you track, such as ["security", "complexity", "duplication", "deprecated_api_usage"]. The [SIGNIFICANCE_THRESHOLD] is a critical control; set it as a percentage (e.g., 20) to prevent noisy, low-impact changes from cluttering the trend report. Before integrating this into a CI/CD pipeline or a scheduled audit job, run the prompt against a golden dataset of paired audit reports where you know the expected trend verdict. Validate that the output JSON strictly adheres to the schema and that the trend_verdict matches your manual assessment. For high-risk codebases, always route the generated trend report to a team lead or architect for human review before publishing it or triggering automated remediation tasks.

The primary implementation challenge for diff and trend analysis is normalization across audit runs. The prompt compares a current audit report against a prior baseline, but if the two reports were generated with different schemas, severity scales, or finding taxonomies, the comparison produces noise instead of signal. The harness must enforce a strict, versioned output schema for all audit runs and validate that the current run conforms to the same schema as the baseline before invoking the diff prompt. Store the schema version in the audit report metadata and reject comparisons where versions differ. This is not a prompt-level concern—it is a pre-flight check in the application layer.

Wire the prompt into a scheduled or event-driven workflow that triggers after a new audit report is generated and validated. The harness should: (1) load the current audit report from a structured store (JSON, database, or artifact repository); (2) retrieve the most recent prior baseline report with a matching schema version and repository identifier; (3) run a structural diff to confirm schema compatibility; (4) assemble the prompt with both reports injected into the [CURRENT_AUDIT] and [BASELINE_AUDIT] placeholders; (5) call the model with a low temperature (0.0–0.2) to maximize consistency; (6) validate the output against the expected trend report schema; and (7) store the result with a timestamp, model version, and input report hashes for auditability. If the model call fails validation, retry once with an explicit error message injected into the retry prompt before escalating for human review.

Model choice matters for numerical consistency. The prompt asks the model to count regressions, improvements, and unchanged findings, then compute trend percentages. Smaller or older models frequently miscount or produce arithmetic errors when comparing long finding lists. Use a model with strong structured output and counting capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). Add a post-generation validation step that independently counts findings from the input reports and cross-checks the model's counts. If counts don't match within a tolerance of zero, flag the output for human review rather than silently accepting incorrect trend data. For high-stakes reporting to engineering leadership, consider running the diff prompt twice with different random seeds and comparing outputs for consistency.

Logging and observability are critical because trend reports feed into engineering health dashboards and remediation planning. Log the full prompt, model response, validation results, and any count mismatches. Tag each trend report with the repository, audit run IDs, schema version, model identifier, and generation timestamp. If a trend report shows a sudden regression spike, teams will need to trace back to the specific audit findings that caused it. The harness should make this traceable by linking each trend line item back to its source findings in both the current and baseline reports. Avoid treating the trend report as a black-box summary—preserve the evidence chain.

Human review gates should be triggered when: (a) the count validation fails, (b) the model reports a regression severity increase of more than one level for any finding category, (c) the total number of findings changes by more than 30% between runs (which may indicate a scanner configuration change rather than a real codebase shift), or (d) the model's confidence markers in the output indicate uncertainty. The review workflow should present the diff output alongside the raw input reports and highlight the specific findings that changed. Do not auto-publish trend reports to dashboards or stakeholder communications without these gates in place. The prompt is a useful accelerator, but trend analysis that drives resource allocation decisions requires human judgment on the final output.