Flaky Test Automated Fix Suggestion Prompt

This prompt is for developers and QA engineers who have already captured a concrete flaky test failure signature—a stack trace, an assertion error, or a timeout message—and need to move from diagnosis to remediation. It is a diagnostic and prescriptive tool, not a root cause analysis engine. The job-to-be-done is generating a ranked list of concrete, code-level fix candidates, each mapped to a specific flakiness category (timing, assertion, cleanup, isolation, or concurrency) with a clear rationale. The ideal user has the failing test's source code and a representative failure log in hand and wants to short-circuit hours of manual debugging by getting structured, actionable suggestions.

Use this prompt when you have a known flaky test pattern and a captured failure signature. It is most effective for common categories of flakiness: race conditions in asynchronous waits, non-deterministic assertions on unordered collections or timestamps, test pollution from shared mutable state, and missing cleanup between cases. The prompt requires two mandatory inputs: the full test source code and the failure log. Without both, the suggestions will be speculative and low-quality. Do not use this prompt for initial flakiness discovery, for tests that have never failed, or for root cause analysis when the failure mechanism is completely unknown. It is also not a replacement for a full test suite audit; for that, use the Test Isolation Boundary Audit or Flaky Test Classification and Severity Triage prompts.

The output is a ranked list of fix suggestions, each containing a specific code change, the flakiness category it addresses, and a rationale. This output requires mandatory human review before any code is committed. The suggestions are fix candidates, not verified patches. A developer must evaluate each suggestion for correctness, potential side effects, and fit with the codebase's testing patterns. After review, apply the selected fix, run the test in a loop with order randomization and environment variance, and verify the fix using the Flaky Test Fix Verification Prompt Template. If the prompt produces low-confidence or irrelevant suggestions, the failure likely requires deeper root cause analysis using the Flaky Test Failure Log Analysis or Race Condition Hypothesis Generation prompts before returning here for fix generation.

This prompt template is designed to be dropped into your AI harness—whether that's a CLI tool, a CI/CD pipeline step, or an internal developer bot. It expects two primary inputs: the flaky test's source code and a representative failure log or signature. The model's job is not to guess, but to match the failure against known flaky test patterns (e.g., race conditions, fixed waits, shared mutable state, non-deterministic assertions) and propose a concrete, code-level fix. The output is a structured suggestion that a developer can review, test, and apply. Do not use this prompt for tests where the failure cause is unknown or the log is empty; it requires a failure signature to ground its analysis.

textCopy
You are a senior test reliability engineer. Your task is to analyze a flaky test and propose a concrete, code-level fix.

## INPUT
- Test Code:

[TEST_CODE]

codeCopy
- Failure Log / Signature:

[FAILURE_LOG]

codeCopy
- Additional Context (e.g., framework, known constraints):
[CONTEXT]

## INSTRUCTIONS
1. Identify the most likely flaky test pattern from the failure log and test code. Use these categories:
   - Race Condition / Timing
   - Fixed Wait / Sleep
   - Non-Deterministic Assertion (e.g., unordered collection, timestamp, UUID, floating point)
   - Shared Mutable State / Test Pollution
   - External Dependency / Environment
   - Missing Cleanup / Data Leak
2. Explain *why* this pattern causes the observed failure, citing specific lines from the test code and log.
3. Propose a specific, actionable fix. Provide the modified code snippet. Prefer standard library or framework-native solutions (e.g., `await` conditions, `assertEventually`, ordered collections, test isolation fixtures).
4. If the fix requires a change outside the test file (e.g., a fixture, helper, or configuration), describe it explicitly.
5. Rate your confidence in the fix as HIGH, MEDIUM, or LOW.
6. If confidence is MEDIUM or LOW, list the additional information or experiments needed to increase confidence.

## OUTPUT SCHEMA
Return a single JSON object with this exact structure:
{
  "pattern_identified": "string",
  "explanation": "string",
  "suggested_fix": {
    "description": "string",
    "code_diff": "string",
    "external_changes": ["string"]
  },
  "confidence": "HIGH" | "MEDIUM" | "LOW",
  "next_steps_if_uncertain": ["string"]
}

## CONSTRAINTS
- Do not invent failure causes not present in the log.
- Do not suggest deleting the test unless it is truly redundant and that redundancy is provable from the input.
- Prefer minimal, targeted fixes over broad refactors.
- If the test uses a specific framework (e.g., Jest, pytest, JUnit), your fix must use that framework's idioms.

After pasting this template, replace [TEST_CODE] with the full source of the flaky test, [FAILURE_LOG] with the stack trace or error output from a failed CI run, and [CONTEXT] with any relevant details like the test framework, known environmental quirks, or links to related bugs. The output schema is strict JSON, making it straightforward to parse in an automated pipeline. The next step after receiving the suggestion is always human review: a developer should evaluate the proposed diff, run the test repeatedly to verify the fix, and ensure the change doesn't mask a deeper design problem. For high-risk code paths, consider requiring a second reviewer or running the fix through your full test suite with a flaky-test detector enabled.

This prompt is designed to be called programmatically from a CI failure handler or a developer CLI tool, not as a one-off chat interaction. The harness should extract the failing test code and the relevant failure log snippet automatically from the test runner output (JUnit XML, JSON reports, or raw console logs) and inject them into the [TEST_CODE] and [FAILURE_LOG] placeholders. The [LANGUAGE] and [TEST_FRAMEWORK] fields should be derived from repository metadata or build configuration files to avoid manual entry errors. The prompt returns a structured JSON array of fix candidates, each with a fix_description, code_diff, confidence_score, and rationale field, which the harness must parse and validate before presenting to the developer.

Validation and Post-Processing: After receiving the model response, validate the JSON structure against a strict schema. Reject any fix candidate where code_diff does not apply cleanly to the original test code (use a dry-run patch application in a temporary checkout). Discard suggestions with confidence_score below a configurable threshold (default 0.7) or where the rationale does not reference specific lines or patterns from the failure log. For high-risk test suites (e.g., payment, auth, data integrity), route all suggestions with confidence below 0.9 to a manual review queue. Log every suggestion, the applied threshold, and the human decision (accepted, modified, rejected) to build a feedback dataset for future prompt tuning.

Model Choice and Retries: Use a model with strong code generation capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). Set temperature to 0.1 for deterministic suggestions. If the model returns malformed JSON or fewer than the requested number of candidates, retry once with an explicit error message appended to the prompt: The previous response was invalid. Ensure the output is a valid JSON array with at least [MIN_CANDIDATES] objects matching the schema. Do not retry more than twice; escalate to a human with the raw failure context if the model cannot produce a valid response. Never automatically apply a suggested fix to the main branch. The harness must create a branch, apply the selected diff, and trigger the test suite to rerun N times (where N is configurable, typically 10-50) to verify the flakiness is resolved before a human merges the PR.