Flaky Test Concurrency Root Cause Prompt

This playbook is for QA engineers and platform teams who have a flaky test that fails intermittently and suspect a concurrency root cause. The prompt is designed to analyze structured evidence—test code, execution logs from at least one failure, and timing data or CI environment details—to produce ranked root cause hypotheses. Each hypothesis includes a confidence score, supporting evidence extracted from the logs, and concrete reproduction guidance. The goal is to replace ad-hoc log scrolling with a systematic, evidence-backed differential diagnosis that points directly to the most likely race condition, ordering dependency, or environmental timing factor.

To use this prompt effectively, you must provide the full test source code, the complete console output or log stream from a failed run, and any available metadata such as CPU timing, thread/goroutine scheduling traces, or CI runner specifications. The prompt is not a magic wand for all test failures. Do not use it for deterministic failures that reproduce reliably in a single-threaded context, pure logic errors with no concurrency involvement, or tests that fail solely due to external service unavailability, network timeouts, or third-party API errors without any local concurrency component. The prompt assumes the failure is non-deterministic and that concurrency is a plausible contributing factor.

Before running this prompt, confirm that you have ruled out simple environmental issues like missing environment variables, stale build artifacts, or disk space exhaustion. The prompt performs best when the failure log contains timestamps, thread or goroutine IDs, and stack traces that allow the model to reconstruct a partial ordering of events. If your logs lack this information, consider enabling verbose logging, race detectors (e.g., Go's -race flag, ThreadSanitizer), or adding structured tracing before submitting the evidence. The output is a set of hypotheses ranked by likelihood, not a single definitive answer. Always validate the top hypothesis by attempting the suggested reproduction steps in a controlled environment before applying a fix.

This prompt template is designed to be pasted directly into your AI harness. It accepts structured inputs about a flaky test failure—including the test code, execution logs, and timing data—and instructs the model to act as a senior concurrency engineer. The output is a ranked list of root cause hypotheses, each with a confidence score, supporting evidence, and a concrete reproduction strategy. All dynamic inputs are represented as square-bracket placeholders, which your application must replace before sending the request.

textCopy
You are a senior concurrency engineer specializing in diagnosing flaky tests in distributed and multi-threaded systems. Your task is to analyze a flaky test failure and produce a ranked list of root cause hypotheses, focusing on race conditions, ordering dependencies, and environmental timing factors.

## INPUT DATA

### Test Code
```[LANGUAGE]
[TEST_CODE]

Execution Logs (from a failing run)

textCopy
[FAILURE_LOGS]

Timing Data (if available)

jsonCopy
[TIMING_DATA]

CI Environment Details

• OS: [OS]
• CPU Architecture: [CPU_ARCH]
• Number of Cores: [CPU_CORES]
• Memory: [MEMORY]
• Test Runner: [TEST_RUNNER]
• Retry Policy: [RETRY_POLICY]

Known Non-Deterministic Factors

[KNOWN_FACTORS]

OUTPUT SCHEMA

Produce a JSON object with the following structure: { "analysis_summary": "A concise summary of the failure pattern and the most likely category of root cause.", "hypotheses": [ { "rank": 1, "category": "race_condition | ordering_dependency | deadlock | resource_exhaustion | environmental_timing | assertion_flakiness | other", "description": "A clear, specific description of the hypothesized root cause.", "confidence": 0.0-1.0, "evidence": ["List of specific log lines, code snippets, or timing anomalies that support this hypothesis."], "reproduction_guidance": "Step-by-step instructions to increase the probability of reproducing this failure, including specific tools (e.g., stress, tc, thread scheduling manipulation) and parameters.", "suggested_fix_approach": "A high-level description of the fix strategy without writing the full patch.", "risk_if_ignored": "What could happen in production if this root cause is not addressed." } ], "excluded_causes": ["List of potential causes that were considered but ruled out, with a brief reason for each."] }

CONSTRAINTS

• Do not invent log entries or timing data. If evidence is insufficient, lower the confidence score and state what additional data is needed.
• Prioritize hypotheses that explain the intermittent nature of the failure.
• If the test code contains shared mutable state, explicitly identify it.
• If the failure is likely not concurrency-related, state this clearly and rank non-concurrency causes.
• For Go code, pay special attention to goroutine leaks, channel blocking, and missing context.Context propagation.
• For async Python, check for missing await keywords and event loop blocking.
• For Java, inspect synchronized block scope, volatile usage, and thread pool configurations.

To adapt this template, replace each [PLACEHOLDER] with data from your CI pipeline. The [TEST_CODE] block should contain the full test source, not just the assertion that failed. The [FAILURE_LOGS] should include timestamps and thread or goroutine IDs if available. The [TIMING_DATA] field is optional but highly valuable—include JSON profiling output or test duration percentiles if you have them. After receiving the model's response, validate the JSON structure before surfacing it to a developer. If the confidence for the top hypothesis is below 0.5, flag the analysis for human review and consider re-running with additional instrumentation enabled.

This prompt is designed to be called programmatically after a test failure is flagged as potentially flaky by your CI system. The implementation harness should gather the required inputs—test code, failure logs, and timing data—and assemble them into the [TEST_CODE], [FAILURE_LOGS], and [TIMING_DATA] placeholders. Because the prompt analyzes concurrency, the quality of the timing data is critical; a single timestamped log line is insufficient. You must capture interleaved event logs from all concurrent actors (threads, goroutines, async tasks) with high-resolution timestamps, or the model will not have enough signal to generate useful root cause hypotheses.

Wire this prompt into a post-failure analysis step in your CI pipeline. On a test failure, a script should extract the test source file, the last N lines of the failure output, and any structured timing logs your test harness produces. If your tests don't produce structured concurrency logs, add a thin instrumentation layer that logs entry/exit of shared resource access (locks, channels, database transactions) with thread IDs and nanosecond timestamps. The harness should then call the LLM with the assembled prompt and parse the JSON output against a strict schema. Validate that the hypotheses array is present, each hypothesis has a non-empty description and a likelihood between 0 and 1, and reproduction_guidance is a non-empty string. If validation fails, retry once with a repair prompt that includes the schema and the malformed output. Log both the raw model response and the parsed result for traceability.

For high-confidence hypotheses (likelihood > 0.7), the harness can automatically create a ticket in your issue tracker with the root cause description and reproduction steps pre-filled. For lower-confidence results, route the analysis to a QA engineer for manual review. Never automatically apply a suggested code fix for a concurrency bug—the risk of introducing a new, subtler race condition is too high. The model's output is a diagnostic aid, not a verified patch. Always require human approval before any code change that alters synchronization logic, lock ordering, or transaction boundaries. Finally, store the prompt version, model version, and input hashes alongside the analysis result so you can audit which analyses were generated under which conditions.