Self-Correction Attempt Trace Prompt

This prompt is designed for AI engineers and SREs who need to perform a root-cause diagnosis on a single production trace containing a self-correction event. A self-correction event is a high-signal moment where the system detected a problem—such as a tool-call error, a validator rejection, or an internal model-generated critique—and then attempted a repair. These moments are critical to investigate because while they can recover a failing interaction, they also introduce compounding risks: the correction itself can be wrong, can increase latency unacceptably, or can degrade the user experience by producing a final output that is worse than the initial failure. Use this prompt when you have isolated a raw trace and need to reconstruct the sequence from initial failure to final outcome, not when you are analyzing aggregate self-correction rates across thousands of sessions.

The ideal input is a structured trace object containing at least one explicit retry, a tool-call error response, a validation failure message, or a turn where the model critiqued its own previous output and generated a replacement. The prompt expects you to provide the raw trace data in the [TRACE_DATA] placeholder, along with the expected output schema for the final response in [OUTPUT_SCHEMA] and any known tool definitions in [TOOL_DEFINITIONS]. Do not use this prompt for traces that completed successfully on the first attempt without any corrective action, as the analysis framework will produce a false-positive search for a non-existent correction. Similarly, do not use it for aggregate trend analysis across multiple sessions; it is calibrated for deep, single-trace diagnosis and will produce verbose, overly granular output if fed batch summaries.

Before running this prompt, ensure you have extracted the full trace from your observability platform, including all spans for retrieval, model inference, tool calls, and any validator or guardrail outputs. The prompt's harness includes validation steps that check for span completeness and will flag missing segments, so incomplete traces will result in an explicit gap report rather than a partial or misleading diagnosis. After receiving the analysis, you should use the included evaluation criteria to verify that the identified correction trigger and root cause align with your own understanding of the system's expected behavior. If the trace involves a regulated or high-risk domain, the output must be reviewed by a human operator before any remediation is applied to the production system.

This prompt template is designed to be wired into your trace analysis pipeline. It takes a structured trace log where a model attempted to self-correct—meaning it generated an initial output, received a signal (internal critique, validator error, tool failure), and produced a revised output. The prompt reconstructs this sequence, isolates the trigger, and evaluates whether the correction improved the final response. Replace each square-bracket placeholder with data extracted from your observability platform before sending the request to the model.

textCopy
You are an AI trace analyst reviewing a production session where the model attempted to self-correct. Your task is to reconstruct the correction attempt, evaluate its effectiveness, and produce a structured diagnostic report.

## TRACE DATA
[TRACE_JSON]

## ANALYSIS INSTRUCTIONS
1. Identify the initial failure or suboptimal output that triggered the correction attempt.
2. Identify the correction trigger: was it an internal self-critique, a validator error, a tool-call failure, or an explicit user correction?
3. Reconstruct the repair attempt: what changed between the initial output and the corrected output?
4. Evaluate the outcome: did the correction resolve the original issue, introduce new errors, or leave the problem unchanged?
5. Classify the correction as: successful, partial_fix, no_change, degraded, or overcorrection.

## OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "trace_id": "string",
  "correction_attempt": {
    "initial_output_summary": "string (what the model first produced)",
    "failure_signal": "string (what indicated the output was wrong)",
    "trigger_type": "self_critique | validator_error | tool_failure | user_correction | other",
    "repair_action": "string (what the model changed in its second attempt)",
    "corrected_output_summary": "string (what the model produced after correction)"
  },
  "evaluation": {
    "original_issue_resolved": true | false,
    "new_errors_introduced": true | false,
    "correction_classification": "successful | partial_fix | no_change | degraded | overcorrection",
    "confidence": 0.0_to_1.0,
    "evidence": ["string (specific trace spans or fields supporting the evaluation)"]
  },
  "recommendation": "string (whether to keep, adjust, or disable the self-correction mechanism)"
}

## CONSTRAINTS
- Base all findings on trace evidence only. Do not speculate beyond the provided spans.
- If the trace is missing spans needed for analysis, set confidence low and note what is missing.
- If no correction attempt is detected, return a null correction_attempt and explain why.
- Flag any cases where the correction loop repeated more than [MAX_RETRY_THRESHOLD] times.

To adapt this template, replace [TRACE_JSON] with the full trace object from your observability store—including user input, system prompt, tool calls, model outputs, and any validator or critique signals. Set [MAX_RETRY_THRESHOLD] to your team's retry limit (commonly 3). Before deploying, validate that the model reliably produces the output schema by running this prompt against a golden set of 20-30 labeled traces where you already know the correction outcome. If the model misclassifies correction types or misses failure signals, add few-shot examples to the prompt or tighten the OUTPUT_SCHEMA field descriptions. For high-stakes production systems where a misdiagnosed self-correction could mask a systemic defect, route low-confidence evaluations to a human reviewer before acting on the recommendation.

The Self-Correction Attempt Trace Prompt is designed to be integrated into an AI observability platform or a custom trace analysis service, not run as a one-off chat. The primary integration point is a post-processing step that triggers whenever a production trace contains a self-correction marker—typically a sequence where a model output is followed by a critique, a new output, and a comparison. Your application should detect these patterns by scanning for consecutive assistant messages or tool calls that exhibit a 'generate-evaluate-regenerate' cycle. Once detected, the raw trace spans for that session are extracted and formatted into the [TRACE_JSON] placeholder, which should include the full sequence of user input, model outputs, tool calls, and any internal monologue or critique tokens if your platform captures them.

For a robust implementation, wrap the prompt call in a harness that validates the output against the expected [OUTPUT_SCHEMA]. The schema should define a structured JSON object with fields like correction_trigger, repair_strategy, outcome, and success_rating. After the LLM returns its analysis, run a programmatic validator to ensure all required fields are present, enums match allowed values (e.g., success_rating is one of improved, degraded, no_change), and the trace_id in the response matches the input. If validation fails, retry once with the validation error appended to the prompt as additional context. Log every analysis result—including the raw prompt, the model's response, and the validation status—to your observability store, keyed by trace_id, to build a queryable history of self-correction events. For high-stakes domains like healthcare or finance, route any trace where the model claims the correction degraded the response to a human review queue before the analysis is considered final.

Model choice matters here: use a model with strong reasoning capabilities and a large context window, as production traces can be verbose. The prompt is designed to work with frontier models like GPT-4o or Claude 3.5 Sonnet. Avoid using smaller, faster models for this task unless you've calibrated their performance on a golden dataset of labeled self-correction traces. To measure the harness's effectiveness over time, track the correction_success_rate metric by aggregating the success_rating field across all analyzed traces. Set an alert if the rate of degraded outcomes spikes above a baseline, as this often signals a prompt regression or a model behavior change that is causing the self-correction mechanism to backfire. The next step after implementing this harness is to build a dashboard that visualizes self-correction frequency, trigger types, and success rates, giving your team a real-time view of this critical runtime behavior.