Silent Failure Detection Prompt for Autonomous Agents

This prompt is designed for agent safety engineers and reliability teams who need to catch a specific class of failure that traditional error handling misses: actions that complete without technical errors but produce semantically wrong outcomes. The job-to-be-done is embedding a post-action verification step into an agent's execution loop that compares the intended outcome against the actual result, flags discrepancies, and packages the failure for human review. The ideal user is an engineer building autonomous agent infrastructure who already has access to execution traces, tool outputs, and the agent's stated intent before taking action.

Use this prompt when your agent operates in environments where a 200 OK or a successful function return does not guarantee correctness—for example, an agent that updates a database record but writes to the wrong field, or one that sends a notification to the incorrect recipient. The prompt requires three concrete inputs to function: the agent's stated goal before the action, the action taken and its parameters, and the observed outcome or tool response. Without all three, the comparison cannot be performed reliably. Do not use this prompt for real-time safety-critical systems where latency constraints preclude human review, or for deterministic, idempotent operations where semantic correctness can be verified programmatically with a unit test.

The primary constraint is that this prompt detects failures but does not fix them. Its output is a structured escalation payload, not a correction. You must wire the output into a human review queue or a supervised rollback workflow. Before deploying, test against a golden dataset of known silent failures—such as wrong-field writes, near-miss entity matches, and off-by-one pagination errors—to calibrate the prompt's sensitivity. If your agent operates in a regulated domain, ensure the escalation includes an audit trail linking the intent, action, and detected discrepancy. The next step after reading this section is to copy the prompt template, substitute your agent's specific action schema, and run it against your last 100 production traces to measure baseline detection coverage.

This prompt template is designed to be injected into an agent's post-action evaluation step. It forces the model to compare the intended outcome of an action against the actual observed result, looking for semantic mismatches that would not trigger a standard error code. The template uses square-bracket placeholders so you can wire in your specific agent's action schema, execution trace, and risk tolerance before deploying it in a production harness.

textCopy
You are an agent safety auditor. Your job is to detect silent failures: actions that reported technical success but produced a semantically wrong or unexpected outcome.

Review the following agent execution record:

[AGENT_ACTION_LOG]

For each action taken, compare the [EXPECTED_OUTCOME] against the [OBSERVED_RESULT]. A silent failure exists if:
- The action returned a success status code, but the observed state does not match the expected state.
- The action completed without error, but a critical side effect is missing or altered.
- The action produced a valid output that is logically inconsistent with the input parameters.

For each detected silent failure, generate a structured escalation object:
{
  "action_id": "string",
  "failure_type": "state_mismatch | missing_side_effect | logical_inconsistency",
  "expected_summary": "Brief description of what should have happened",
  "observed_summary": "Brief description of what actually happened",
  "severity": "low | medium | high | critical",
  "diagnostic_hypothesis": "Most likely reason for the mismatch",
  "recommended_remediation": "Immediate next step for a human operator"
}

If no silent failures are detected, return an empty list.

[CONSTRAINTS]
- Do not flag actions that explicitly returned an error code; those are overt failures.
- Only flag actions where the success signal is misleading.
- If the [OBSERVED_RESULT] is incomplete or missing, treat that as a silent failure with severity 'high'.
- Use the [RISK_TOLERANCE] level to calibrate severity: at 'low' tolerance, default to 'high' severity for any ambiguity.

To adapt this template, replace the placeholders with your agent's specific data structures. [AGENT_ACTION_LOG] should be a structured trace containing action IDs, input parameters, success status codes, and observed state snapshots. [EXPECTED_OUTCOME] and [OBSERVED_RESULT] can be inline fields within that log or separate context blocks. [RISK_TOLERANCE] should be set to low, medium, or high based on the operational domain. Before shipping, validate that the output JSON schema matches your escalation queue's expected payload format and run the prompt against a golden set of known silent failures and known clean runs to measure detection recall and false positive rate.

This prompt is designed to sit after an agent action completes and returns a success status, but before the agent proceeds to the next step. It acts as a semantic circuit breaker. The harness should invoke this prompt with the agent's original intent, the action taken, the tool output, and the expected outcome. The model's response determines whether the agent continues autonomously, retries with a corrected plan, or escalates to a human operator with a structured diagnostic payload.

Integration point: Insert this check into the agent's execution loop immediately after any tool call that modifies state (write operations, API calls, database transactions) or produces a result that downstream steps depend on. For read-only operations, you may skip this check or run it asynchronously for monitoring. Input assembly: The prompt expects four variables: [AGENT_INTENT] (the goal the agent was trying to achieve), [ACTION_TAKEN] (the specific tool call and arguments), [TOOL_OUTPUT] (the raw response from the tool), and [EXPECTED_OUTCOME] (what the agent predicted would happen). Package these into a single structured context block before calling the model. Model choice: Use a capable reasoning model (GPT-4, Claude 3.5 Sonnet, or equivalent) for this detection task. Smaller or faster models often miss subtle semantic mismatches. For latency-sensitive loops, consider running this check on a separate, asynchronous thread that can interrupt the agent if a silent failure is detected within a time budget.

Validation and retry logic: Parse the model's JSON output and check the silent_failure_detected boolean field. If false, allow the agent to proceed. If true, read the severity field. For severity: low, you may log the discrepancy and continue with a warning flag. For severity: medium, inject the correction_suggestion back into the agent's planning context and retry the action once. For severity: high or critical, immediately halt the agent, serialize the full escalation_payload (including expected_vs_actual comparison, failure_mode classification, and diagnostic_context), and route to a human review queue. Logging: Log every invocation of this prompt, including cases where no failure was detected. Store the full input context, model response, and agent state snapshot. This audit trail is essential for debugging false negatives and improving the prompt over time. Human review integration: When escalating, present the escalation_payload in a review interface that highlights the semantic gap between expected and actual outcomes. Include a one-click option for the reviewer to approve the correction, revert the action, or mark the detection as a false positive. Feed false positive labels back into your eval dataset.

What to avoid: Do not use this prompt as a replacement for schema validation or HTTP status code checks. Those are prerequisite gates that should pass before this semantic check runs. Do not run this prompt on every agent thought or planning step—reserve it for post-action verification to avoid excessive latency and cost. Do not ignore severity: medium detections; they are often early signals of prompt drift or tool behavior changes that will become critical failures later. Finally, never allow the agent to override a critical severity escalation without human review, even if the agent believes it can self-correct.