Test Suite Health Scoring Rubric Prompt

This prompt is designed for test architects and Senior Development Engineers in Test (SDETs) who need a repeatable, quantitative baseline of a test suite's overall health. The primary job-to-be-done is to generate a structured, scored assessment across multiple dimensions—such as redundancy, coverage efficiency, flakiness, and maintenance burden—before initiating a major consolidation sprint, conducting a quarterly engineering health review, or building an evidence-based business case to justify refactoring investment. It transforms a subjective feeling that 'the suite is bloated' into a defensible, dimension-specific score that can be tracked over time.

To use this prompt effectively, you must provide a representative sample of the test suite's structure and recent execution data. The required context includes a list of test case names with their functional descriptions, recent pass/fail histories to calculate flakiness, and ideally a mapping of tests to requirements or code coverage areas. The prompt is calibrated for suites with at least 50 test cases where manual review would be time-prohibitive. The output is not just a single score but a dimension breakdown with prioritized, actionable improvement recommendations. You should calibrate the scoring thresholds against your team's historical baselines before treating any absolute score as a definitive gate.

Do not use this prompt for real-time CI/CD gating or as a blocking check in a deployment pipeline; its strength is in strategic analysis, not millisecond-level pass/fail decisions. It is also unsuitable for small test suites with fewer than 50 cases, where a manual review by a senior engineer is faster and more contextually accurate. Avoid using this prompt on suites that lack any execution history, as the flakiness and maintenance burden dimensions require empirical pass/fail data to produce meaningful scores rather than speculative guesses.

This prompt template produces a structured health score for a test suite by evaluating it across multiple dimensions: redundancy, coverage efficiency, flakiness, and maintenance burden. Replace the square-bracket placeholders with your suite's metadata, historical data, and calibration baselines. The output is a JSON rubric you can feed directly into a dashboard, a CI quality gate, or a stakeholder review packet.

textCopy
You are a test suite health analyst. Evaluate the provided test suite data and produce a scored health assessment.

## INPUT DATA
[TEST_SUITE_METADATA]
[TEST_EXECUTION_LOGS]
[FLAKINESS_HISTORY]
[COVERAGE_REPORTS]
[MAINTENANCE_TICKETS]

## SCORING DIMENSIONS
Score each dimension from 0 (critical) to 100 (excellent). Use the team's calibration baselines where provided.

1. Redundancy: Degree of functionally duplicate or near-duplicate test cases.
2. Coverage Efficiency: Ratio of unique risk coverage to total test count.
3. Flakiness: Rate of non-deterministic failures over the last [TIME_PERIOD].
4. Maintenance Burden: Effort spent on test upkeep relative to value delivered.

## CALIBRATION BASELINES
[CALIBRATION_BASELINES]

## OUTPUT SCHEMA
Return a single JSON object with this exact structure:
{
  "overall_health_score": number (0-100),
  "dimension_scores": {
    "redundancy": { "score": number, "rationale": "string" },
    "coverage_efficiency": { "score": number, "rationale": "string" },
    "flakiness": { "score": number, "rationale": "string" },
    "maintenance_burden": { "score": number, "rationale": "string" }
  },
  "top_improvement_recommendations": [
    { "action": "string", "dimension": "string", "expected_impact": "string", "effort_estimate": "low|medium|high" }
  ],
  "risk_notes": "string summarizing any data gaps or uncertainty in the assessment"
}

## CONSTRAINTS
- Do not invent data. If a dimension cannot be scored due to missing input, set the score to null and explain in the rationale.
- Weight flakiness and coverage efficiency higher than redundancy and maintenance burden unless calibration baselines specify otherwise.
- Recommendations must be specific and actionable, not generic advice.
- If the overall health score is below [CRITICAL_THRESHOLD], include an explicit escalation flag in risk_notes.

Adaptation notes: Replace [TEST_SUITE_METADATA] with a structured summary of test count, types, and ownership. [TEST_EXECUTION_LOGS] should include pass/fail rates and durations over a representative window. [FLAKINESS_HISTORY] needs per-test flakiness counts or a flakiness dashboard export. [COVERAGE_REPORTS] should map tests to requirements, code paths, or risk items. [MAINTENANCE_TICKETS] can be a count or summary of test-related bug tickets and refactoring effort. [CALIBRATION_BASELINES] are your team's historical thresholds—for example, "flakiness above 5% is poor" or "redundancy below 10% is excellent." [TIME_PERIOD] should match your team's review cadence, such as "the last 4 weeks." [CRITICAL_THRESHOLD] is the score below which the suite needs immediate attention, typically 40-50 depending on your risk tolerance.

Before integrating this prompt into a CI pipeline or dashboard, run it against a known baseline suite where you already agree on the health assessment. Compare the model's scores and rationales to your team's manual evaluation. Calibrate the dimension weights and thresholds until the prompt produces assessments you would defend in a release readiness meeting. For regulated environments, always retain the full prompt input and output as an audit artifact, and require human sign-off on any retirement or consolidation actions recommended by the scoring rubric.

The Test Suite Health Scoring Rubric Prompt is designed to be integrated into a CI/CD pipeline or a scheduled analysis job, not run as a one-off chat. The prompt expects a structured [TEST_SUITE_METADATA] input containing raw metrics—execution times, pass/fail rates, flakiness scores, coverage data, and maintenance history—that your test infrastructure already collects. The model's job is to apply a consistent scoring rubric across dimensions like redundancy, coverage efficiency, flakiness, and maintenance burden, producing a scored assessment with dimension breakdowns and improvement recommendations. This means your implementation harness must gather, validate, and format the input data before calling the model, then validate and store the structured output afterward.

Wire the prompt into an application by building a pre-processing step that queries your test management system (e.g., TestRail, Zephyr) and CI logs (e.g., Jenkins, GitHub Actions) to populate the [TEST_SUITE_METADATA] fields. The harness should enforce schema validation on the input side—if required fields like flakiness_rate or coverage_percentage are missing or out of range, the job should fail fast with a clear error rather than sending incomplete data to the model. On the output side, parse the model's JSON response and validate it against the expected [OUTPUT_SCHEMA]: a top-level health_score (0-100), a dimension_breakdown array with per-dimension scores and rationales, and an improvement_recommendations array with prioritized actions. If the model returns malformed JSON or scores outside the 0-100 range, implement a retry loop with a maximum of two retries using a repair prompt that includes the validation error message. After two failures, log the raw output and escalate to a human reviewer rather than silently accepting a broken score.

For model choice, use a model with strong structured output capabilities and a context window large enough to hold the full [TEST_SUITE_METADATA] plus the rubric instructions. GPT-4o or Claude 3.5 Sonnet are appropriate defaults. Enable structured output mode if the provider supports it (e.g., OpenAI's response_format with a JSON schema) to reduce parsing failures. Log every scoring run—input metadata, raw model response, parsed scores, and any validation errors—to an observability system for trend analysis and prompt debugging. Avoid using this prompt for real-time gating decisions (e.g., blocking a deployment based on a single health score) without first calibrating the rubric against your team's historical baselines over multiple scoring cycles. The score is a diagnostic signal, not a release authority.