Human-in-the-Loop Correction Prompt Template

This prompt is a data-capture contract for ML engineers, AI platform teams, and operations leads who need every human correction to an agent's output to be stored as a structured, machine-readable record. The core job-to-be-done is not just fixing a mistake, but creating a training artifact that can be used for downstream fine-tuning, evaluation dataset construction, or agent performance monitoring. The ideal user is someone building a correction interface where the human reviewer's actions must be traceable, auditable, and programmatically consumable, not just a free-text comment in a chat window.

Use this prompt when the correction workflow requires strict structure: the original agent output, the corrected version, and explicit metadata explaining what changed and why. This structure is essential for closing the loop in AI systems that learn from their mistakes. For example, an AI coding agent that generates a buggy function needs a human to correct it; this prompt ensures the correction is captured with fields like correction_type (e.g., 'logic_error', 'type_mismatch') and reasoning, which can then be fed into a fine-tuning pipeline to prevent similar errors. The prompt forces the model to act as an intermediary that structures human intent into a predefined schema, making it a critical component in a production AI harness.

Do not use this prompt when the correction does not need to be stored for model improvement, or when the human reviewer is expected to perform the correction directly in a UI without an intermediate structuring step. It is also unsuitable for casual feedback or free-text comments where the goal is communication, not data capture. If the reviewer needs to simply edit a text field and save it, this prompt adds unnecessary friction. The next step after understanding this use case is to review the prompt template itself, paying close attention to the required input variables and the output schema, which form the backbone of the data contract.

This template structures the correction interface between a human reviewer and an AI agent. When a reviewer identifies an error in an agent's output, this prompt captures the original output, the correction, the reason for the change, and metadata that downstream systems can use for fine-tuning, evaluation dataset construction, or agent behavior analysis. The template is designed to be embedded in a review UI or correction workflow where the human provides the correction and the system records it in a machine-readable format.

textCopy
SYSTEM: You are a correction-capture assistant. Your only job is to format a human correction into a structured record. Do not modify, reinterpret, or second-guess the correction provided. Preserve the human's exact correction text and stated reason.

USER:
Capture the following correction as a structured record.

ORIGINAL AGENT OUTPUT:
[ORIGINAL_OUTPUT]

HUMAN CORRECTION:
[CORRECTION_TEXT]

REASON FOR CORRECTION:
[CORRECTION_REASON]

CORRECTION CATEGORY:
[CATEGORY]

TASK CONTEXT:
[TASK_CONTEXT]

AGENT ID:
[AGENT_ID]

SESSION ID:
[SESSION_ID]

REVIEWER ID:
[REVIEWER_ID]

OUTPUT SCHEMA:
{
  "correction_record": {
    "original_output": "string (exact agent output before correction)",
    "corrected_output": "string (human-provided correction)",
    "correction_reason": "string (human-stated reason)",
    "correction_category": "factual_error | missing_information | hallucination | formatting | tone | policy_violation | other",
    "task_context": "string (what the agent was asked to do)",
    "agent_id": "string",
    "session_id": "string",
    "reviewer_id": "string",
    "timestamp": "ISO 8601 UTC",
    "diff_summary": "string (brief description of what changed)"
  }
}

CONSTRAINTS:
- Use the human's exact correction text. Do not paraphrase or improve it.
- If the correction category is not provided, infer it from the reason and mark confidence as "inferred".
- If any field is missing, set its value to null and include a "missing_fields" array in the output.
- Generate the timestamp as the current UTC time.
- The diff_summary should be a one-sentence description of what changed between original and corrected output.

Adapt this template by adjusting the CORRECTION CATEGORY enum to match your domain's error taxonomy. For high-risk workflows such as clinical documentation or legal review, add a [RISK_LEVEL] placeholder and require a second reviewer sign-off before the correction is accepted into the training dataset. The output schema should be validated against your downstream fine-tuning or evaluation pipeline's expected format before storage. Always log the raw correction payload alongside the structured record for auditability.

The correction prompt template is not a standalone interface; it is a structured data-capture step inside a larger human-in-the-loop workflow. The application must present the original agent output, collect the human correction, and then call the LLM with this prompt to normalize the correction into a consistent schema. The harness is responsible for pre-filling the [ORIGINAL_OUTPUT], [CORRECTION_NOTES], and [METADATA] fields from the upstream agent log and the reviewer's UI input before the prompt ever reaches the model. Do not ask the human reviewer to fill in JSON or follow prompt formatting—the harness does that translation.

After the model returns the structured correction payload, the harness must validate it against the expected [OUTPUT_SCHEMA] before accepting it. At minimum, check that correction_id is present and unique, original_output matches the input exactly, corrected_output is non-empty and differs from the original, correction_type is one of the allowed enum values, and reviewer_id and timestamp are populated. If validation fails, retry once with the same prompt and an added [VALIDATION_ERRORS] context block. If the second attempt also fails, log the raw response and flag the correction for manual review rather than silently ingesting malformed data. For high-stakes domains such as healthcare or compliance, route all corrections to a second human reviewer or a senior approver before the corrected output is committed to the system of record.

The corrected output and its metadata must be written to an append-only corrections log or dataset, not used to overwrite the original agent output in place. This preserves the full audit trail: original output, correction, reviewer identity, timestamp, and correction type. Downstream, this dataset feeds fine-tuning preparation, evaluation benchmark construction, and agent performance analysis. Avoid the common failure mode of treating corrections as one-off fixes; the harness should treat every correction as a labeled training example. Wire the correction log into your observability stack so that correction rate, correction type distribution, and reviewer-specific patterns are visible in dashboards. If the correction rate for a particular agent or task type exceeds a defined threshold, trigger an automatic review of that agent's prompts or routing logic rather than silently accumulating corrections.