Test Debt Quantification and Prioritization Prompt Template

This prompt is built for engineering managers and tech leads who need to move from anecdotal frustration about test suite quality to a defensible, data-backed remediation plan. It is designed to be used by a coding agent or analysis tool that has access to repository context, CI/CD history, and test infrastructure metadata. The prompt ingests raw signals—flakiness logs, coverage reports, execution duration data, assertion quality audits, and fixture staleness checks—and produces a quantified, prioritized backlog that maps each debt item to risk reduction and estimated effort. The output is not a single test fix or a generic coverage recommendation; it is a structured remediation backlog suitable for sprint planning, resource allocation, and engineering leadership review.

Use this prompt when you have access to concrete test suite data and need a prioritization framework that accounts for multiple debt dimensions simultaneously. The ideal input includes flakiness frequency and cost (time wasted on reruns), coverage gaps mapped to critical code paths, slow tests with duration trends, assertion weakness audit results, and fixture freshness reports. The prompt expects these inputs to be provided as structured context—either inline or via tool-retrieved data—so the model can reason across dimensions rather than generating plausible-sounding but ungrounded advice. Do not use this prompt when you lack quantitative data about your test suite, when you need a single test repair, or when you are looking for general testing philosophy. It is also inappropriate for teams that cannot act on a prioritized backlog due to organizational constraints; the output assumes an engineering team ready to execute remediation work.

Before running this prompt, ensure your coding agent or analysis harness has collected and normalized the required input signals. The prompt includes placeholders for [FLAKINESS_DATA], [COVERAGE_REPORT], [DURATION_DATA], [ASSERTION_AUDIT], and [FIXTURE_FRESHNESS_REPORT]. If any of these inputs are unavailable, the prompt will still produce output, but the prioritization will be skewed toward the available dimensions. For high-risk domains—such as financial systems, healthcare software, or safety-critical infrastructure—the output should be treated as a starting point for human review, not a final remediation plan. The prompt includes a [RISK_LEVEL] parameter to adjust the caution and evidence requirements in the output. After generating the backlog, validate that each debt item is traceable to a specific input signal and that effort estimates are consistent with your team's historical velocity before committing to a remediation sprint.

This prompt template is designed to be wired into an automated test health pipeline or used as part of a manual engineering review. It expects structured inputs describing the current state of the test suite, including flakiness data, coverage reports, execution timing, and known issues. The model is instructed to reason across these dimensions, apply a consistent scoring rubric, and output a strictly typed JSON payload that can be ingested by dashboarding tools, ticketing systems, or CI/CD quality gates. Before using this prompt in production, ensure that all input data is recent (e.g., from the last 30 days of CI runs) and that the output schema is validated against your downstream consumer.

textCopy
You are a test health analyst. Your task is to quantify test debt across a software test suite and produce a prioritized remediation backlog.

## INPUT DATA
- Flakiness Report: [FLAKINESS_REPORT]
- Coverage Report: [COVERAGE_REPORT]
- Test Execution Timings: [EXECUTION_TIMINGS]
- Known Issues & Stale Fixtures: [KNOWN_ISSUES]
- Team Capacity & Velocity: [TEAM_CAPACITY]

## SCORING DIMENSIONS
For each test or test group, assess debt across these dimensions on a scale of 0 (no debt) to 10 (critical debt):
1. Flakiness Cost: Failure rate * CI rerun cost * developer investigation time.
2. Coverage Gap: Critical code paths with zero or inadequate test coverage.
3. Execution Slowness: Tests exceeding [SLOW_THRESHOLD_MS]ms, multiplied by frequency of execution.
4. Assertion Weakness: Tests with over-broad matches, missing negative cases, or unverified side effects.
5. Fixture & Environment Staleness: Tests relying on outdated schemas, unrealistic data, or contaminated shared state.

## OUTPUT INSTRUCTIONS
Generate a JSON object with the following structure. Do not include any text outside the JSON.

{
  "summary": {
    "total_tests_analyzed": <int>,
    "overall_debt_score": <float 0-10>,
    "critical_issues_count": <int>,
    "estimated_remediation_weeks": <float>,
    "projected_risk_reduction_pct": <float>
  },
  "top_remediation_items": [
    {
      "rank": <int>,
      "test_or_group_name": "<string>",
      "debt_dimension": "flakiness|coverage_gap|slowness|assertion_weakness|fixture_staleness",
      "severity_score": <float 0-10>,
      "evidence": "<specific data point from input reports>",
      "suggested_fix": "<concrete, actionable remediation step>",
      "effort_estimate_hours": <float>,
      "risk_reduction_if_fixed_pct": <float>
    }
  ],
  "dimension_breakdown": {
    "flakiness_cost": { "total_items": <int>, "avg_severity": <float>, "total_effort_hours": <float> },
    "coverage_gap": { "total_items": <int>, "avg_severity": <float>, "total_effort_hours": <float> },
    "execution_slowness": { "total_items": <int>, "avg_severity": <float>, "total_effort_hours": <float> },
    "assertion_weakness": { "total_items": <int>, "avg_severity": <float>, "total_effort_hours": <float> },
    "fixture_staleness": { "total_items": <int>, "avg_severity": <float>, "total_effort_hours": <float> }
  },
  "excluded_items": [
    {
      "test_or_group_name": "<string>",
      "reason_for_exclusion": "<why this item was deprioritized or excluded>"
    }
  ]
}

## CONSTRAINTS
- Base all severity scores and evidence strictly on the provided input data. Do not invent or assume data.
- If input data is missing for a dimension, set its severity to 0 and note the absence in `excluded_items`.
- Prioritize items that block CI pipelines or cause developer toil over purely cosmetic improvements.
- For each remediation item, the `risk_reduction_if_fixed_pct` must be a reasonable estimate relative to the overall debt score.
- If team capacity data is provided, ensure `estimated_remediation_weeks` accounts for it.

To adapt this template for your environment, replace each square-bracket placeholder with structured data from your test infrastructure. The [FLAKINESS_REPORT] should include per-test failure rates and CI rerun costs. The [COVERAGE_REPORT] should map uncovered code paths to specific tests or modules. Set [SLOW_THRESHOLD_MS] to your team's agreed-upon threshold (e.g., 5000ms for integration tests, 200ms for unit tests). If you lack data for a dimension, pass an explicit "NOT_AVAILABLE" string rather than omitting the placeholder, so the model can correctly populate the excluded_items array. After receiving the output, validate the JSON schema, check that all severity scores are grounded in the provided evidence, and route items with severity_score above 8 to your on-call or test-infrastructure team for immediate review.

This prompt is designed to be called from a CI/CD pipeline, a code quality dashboard, or a scheduled analysis job—not as a one-off chat interaction. The implementation harness must collect the required inputs (test suite metadata, flakiness history, coverage reports, execution timing data, and repository context), assemble them into the prompt's [INPUT] block, and parse the structured JSON output for downstream consumption. Because the output includes effort estimates and risk reduction projections that may influence team roadmaps, the harness should treat this as a high-stakes analysis workflow with mandatory validation and human review before any remediation backlog is committed to a sprint.

Input Assembly: Build a data collection step that gathers: (1) test execution history from your CI system (at least 30 days of runs with pass/fail/duration per test), (2) code coverage reports in a machine-readable format (e.g., LCOV, JaCoCo XML, or Istanbul JSON), (3) test file paths and sizes, (4) assertion counts per test if available, (5) fixture or mock file modification dates, and (6) a module-to-team ownership map. Serialize this into a structured JSON block that becomes the [INPUT] placeholder. For large codebases, pre-filter to the top N modules by churn or failure rate to stay within context windows. Model Choice: Use a model with strong structured output capabilities and a context window large enough for your assembled input—Claude 3.5 Sonnet, GPT-4o, or Gemini 1.5 Pro are appropriate. For very large repositories, consider a two-pass approach: first pass identifies high-debt areas, second pass deep-dives on those areas with full context. Output Validation: Parse the model's JSON response and validate against a schema that requires: dimensions (array of debt categories with name, severity, affected_tests, cost_estimate_hours), prioritized_backlog (array of remediation items with rank, description, effort_days, risk_reduction_score, blocked_by), and summary (object with total_estimated_debt_hours, highest_risk_dimension, quick_wins). Reject outputs missing required fields, containing circular dependencies in blocked_by, or with risk_reduction_score values outside 0-1 range. On validation failure, retry once with the validation errors appended to the prompt as [CONSTRAINTS] feedback.

Human Review Gate: The validated output must route to a review queue—typically a GitHub issue, Linear ticket, or internal dashboard—where an engineering manager or tech lead can accept, reject, or adjust each backlog item before it becomes actionable work. The harness should render the output as a readable summary with expandable detail per dimension, not just raw JSON. Logging and Traceability: Log the full prompt input, model response, validation results, and reviewer decisions. This creates an audit trail for comparing debt assessments over time and measuring whether remediation efforts actually reduced the quantified debt. What to Avoid: Do not auto-create tickets or assign work from this prompt's output without human review—effort estimates and risk scores are model judgments that need engineering calibration. Do not run this on every commit; schedule it weekly or per-release to avoid noise and token costs. If your test suite has fewer than 100 tests or less than 14 days of execution history, the signal-to-noise ratio will be too low for meaningful quantification—use a simpler health check prompt instead.