Test Quality and Anti-Pattern Review Prompt

This prompt is designed for QA engineers and developers who maintain test suites and need to systematically identify anti-patterns that reduce test reliability, increase maintenance cost, and slow down CI pipelines. The job-to-be-done is a structured code review of test files—not a runtime analysis—to catch structural problems like assertion roulette, eager tests, mystery guests, test dependencies, conditional test logic, and slow test patterns before they erode suite trustworthiness. The ideal user is someone who can read test code fluently and is responsible for test suite health, whether during a pull request review, a pre-merge check, or a periodic test audit.

Use this prompt when you have complete test file content or a focused diff of test changes as input. It expects the raw test code, not test execution logs or coverage reports. The prompt is most effective when the input includes enough context to distinguish between intentional test design choices and genuine anti-patterns—for example, a helper method that looks like a mystery guest might be a deliberate shared fixture. Do not use this prompt as a replacement for running tests, measuring actual coverage, or profiling test execution time. It analyzes structure and patterns, not runtime behavior. It is also not a general code review prompt; it is tuned specifically for test code anti-patterns and will miss non-test logic issues.

The output is a structured refactoring recommendation list, not a pass/fail verdict. Each finding should include the anti-pattern type, the specific code location, a severity rating, and a concrete refactoring suggestion. The prompt includes eval checks that force the model to justify why a pattern is flagged and to identify cases where the pattern is intentional. Before using this prompt in an automated CI gate, calibrate it against your team's test style guide and a set of known-clean and known-problematic test files. Human review remains essential for architectural decisions about test strategy, but this prompt scales the initial detection pass so reviewers can focus on judgment rather than pattern scanning.

The following prompt template is designed to be copied directly into your AI harness, test pipeline, or IDE. It instructs the model to act as a test quality reviewer, analyzing provided test code for a specific set of known anti-patterns. The prompt is structured to produce a consistent, machine-readable JSON output that can be parsed by downstream automation, such as a CI bot that comments on merge requests or a dashboard tracking test suite health.

markdownCopy
You are a senior test quality engineer reviewing a test suite for maintainability and reliability anti-patterns. Your task is to analyze the provided test code and identify specific instances of the following anti-patterns:
- **Assertion Roulette:** Multiple assertions without explanatory messages, making failure diagnosis difficult.
- **Eager Test:** A single test method that verifies multiple unrelated behaviors.
- **Mystery Guest:** A test that depends on external data, files, or database state not visible within the test itself.
- **Test Dependency:** Tests that must run in a specific order or rely on the side effects of other tests.
- **Conditional Test Logic:** Tests containing `if`, `switch`, or loop logic that alters the verification path.
- **Slow Test:** Tests with explicit waits, large data setups, or real I/O calls that are not mocked.

**Input Test Code:**

[TEST_CODE]

codeCopy

**Instructions:**
1.  For each anti-pattern found, create a finding object in the `findings` array.
2.  If no anti-patterns are found, return an empty `findings` array.
3.  Provide a `refactoring_summary` that synthesizes the top 3 most critical issues into a concise, actionable paragraph. If there are no findings, state that the test code is clean.

**Output Schema:**
You must respond with a single valid JSON object conforming to this structure:
{
  "findings": [
    {
      "anti_pattern": "Assertion Roulette",
      "location": "test_user_update_success",
      "severity": "high",
      "evidence": "Lines 12-15 contain 4 assertions without failure messages.",
      "refactoring_suggestion": "Add a unique assertion message to each assertEqual call, or split into individual test methods."
    }
  ],
  "refactoring_summary": "The test suite has critical reliability issues. The 'test_user_update_success' method suffers from Assertion Roulette, making CI failures hard to diagnose. The 'test_full_order_flow' is an Eager Test that should be decomposed into separate unit tests for validation, payment, and notification. Finally, 'test_report_generation' is a Slow Test due to its un-mocked database calls, which will degrade pipeline performance."
}

To adapt this template, replace the [TEST_CODE] placeholder with the actual test file content or a specific test method. The anti-pattern list and output schema are designed to be stable, but you can extend the anti_pattern enum values in the instructions and schema if your team has custom categories, such as 'Sensitive Data Leak' or 'Over-Mocking'. For high-risk production code, always route findings with a high severity to a human reviewer for confirmation before applying automated refactors.

This prompt is designed to be integrated into a software development lifecycle where test code is treated as a first-class asset. The most common integration points are a CI/CD pipeline step that runs on test file changes, a pre-commit hook for local development, or a manual review tool used by QA engineers during test suite audits. The prompt expects a complete test file or a focused diff of test changes as its primary input. Before calling the model, the harness should extract the test code, determine the testing framework in use (e.g., Jest, pytest, JUnit) from file metadata or package configuration, and inject that framework name into the [FRAMEWORK] placeholder. If the test file is large, consider splitting it into logical blocks (individual describe/test blocks) and running the prompt on each block to stay within context limits and produce more focused findings.

The implementation harness must validate the model's output against the expected JSON schema before surfacing findings. Define a strict schema that requires a findings array, where each finding has a severity (enum: CRITICAL, WARNING, SUGGESTION), an anti_pattern field matching one of the known categories (e.g., assertion_roulette, eager_test, mystery_guest, test_dependency, conditional_test_logic, slow_test), a location object with line_start and line_end integers, a description string, and a refactoring_suggestion string. If the model output fails schema validation, implement a single retry by appending the validation error to the original prompt as a correction instruction. If the retry also fails, log the raw output and flag the file for manual review rather than silently dropping findings. For high-risk test suites (e.g., financial calculations, healthcare data processing), always route CRITICAL findings to a human reviewer queue before auto-applying any refactoring suggestions.

Logging and observability are critical for tuning this prompt over time. Log the model used, prompt version, input file hash, token counts, latency, and a summary of finding counts by severity on every invocation. Track the false positive rate by having reviewers mark findings as accepted, rejected_false_positive, or rejected_not_actionable. Use this feedback to build a golden dataset of known anti-patterns and false positives for your specific codebase, which can then be used to calibrate the prompt's [EXAMPLES] section or to fine-tune a smaller model for cost efficiency. Avoid wiring this prompt directly into a merge-blocking gate until you have measured its precision on at least 50 reviewed test files and confirmed that the false positive rate is below your team's acceptable threshold. Start with a non-blocking advisory check that comments on the PR, and escalate to a blocking gate only after calibration.