Function Call Error Code Classification Prompt

This prompt is designed for operations teams and platform engineers who need to transform raw, unstructured error payloads from production tool calls into a standardized, actionable taxonomy. The core job-to-be-done is aggregating reliability signals across thousands of traces. Instead of manually reading error messages, you use this prompt to classify each failure into a predefined category—such as timeout, rate_limit, auth, validation, server_error, or unknown. The structured output is then ingested into dashboards, used to trigger incident response playbooks, or fed into reliability tracking systems to identify which tools or services are degrading.

The ideal input for this prompt is a structured error payload that includes, at minimum, an HTTP status code, an error message string, and a tool or function identifier. The prompt's classification logic relies on these signals to disambiguate errors. For example, a 429 status code with a Retry-After header is a strong signal for rate_limit, while a 500 status code with an upstream service error message points to server_error. This prompt is not a general-purpose error explainer; it is a classification engine. It should be used in a batch processing pipeline where you are classifying errors from logs, not in a real-time user-facing chat where a user is asking 'why did this fail?'.

Do not use this prompt for classifying user-facing errors, model refusal events, or errors that lack a tool-call context. If the error originates from a model's refusal to answer, a content policy violation, or a malformed output that never reached a tool, this is the wrong playbook. For those cases, refer to the Safety, Refusal, and Policy Compliance Trace Prompts or the Output Repair and Validation Prompts. To start, gather a batch of raw error objects from your trace store, format them with their tool_id, status_code, and error_message, and run them through this prompt to build your first error taxonomy report.

This prompt template is designed to be dropped directly into an observability pipeline or a manual trace review workflow. It takes a list of raw production errors and returns a structured JSON array that can be ingested by monitoring dashboards, alerting systems, or incident reports. The template enforces a strict taxonomy—timeout, rate_limit, auth, validation, server_error, and unknown—to prevent category sprawl and ensure consistent reporting across teams. Before using this prompt, ensure your error list contains at least a trace ID, tool name, and the original error code or message; missing fields will degrade classification confidence.

textCopy
Classify the following tool-call errors from production traces into standard categories. Use only the categories defined below. Return a JSON array of classification objects, one per input error. If an error does not clearly fit a category, classify it as 'unknown' and include a brief reason.

Error Categories:
- timeout: The tool call exceeded its time limit.
- rate_limit: The tool's API rate limit was exceeded.
- auth: Authentication or authorization failed.
- validation: The request arguments were invalid or malformed.
- server_error: The tool's server returned a 5xx error or equivalent internal failure.
- unknown: The error does not match any defined category.

Input Errors:
[ERROR_LIST]

Output Format:
[
  {
    "trace_id": "string",
    "tool_name": "string",
    "original_error_code": "string or number",
    "classification": "timeout|rate_limit|auth|validation|server_error|unknown",
    "confidence": "high|medium|low",
    "reasoning": "brief explanation of the classification decision"
  }
]

Constraints:
- Do not invent categories.
- If the error message is empty or missing, classify as 'unknown' with low confidence.
- Preserve the original error code in the output.

To adapt this template, replace the [ERROR_LIST] placeholder with a JSON array of error objects from your logging system. Each object should include at minimum a trace_id, tool_name, and error_message or error_code. If your internal error schema uses different field names, add a mapping instruction to the Constraints section—for example, 'Map the err_code field to original_error_code.' For high-volume pipelines, consider batching errors into groups of 50–100 to stay within context window limits while maintaining classification quality. Always validate the output JSON against the expected schema before forwarding it to your aggregation layer; a malformed classification record can corrupt dashboard metrics and delay incident response.

This prompt is designed to be called programmatically within a batch processing pipeline, not as a one-off chat interaction. The pipeline should be triggered on a schedule (e.g., every 15 minutes) or by a log aggregation event when a new batch of function-call errors is ready for analysis. The core integration involves serializing the list of error objects into the [ERROR_LIST] placeholder, calling the LLM, and then strictly validating the structured JSON response before it enters your observability dashboard. Treat the model's output as untrusted input to your system until it passes schema validation.

To integrate, construct the full prompt by injecting the serialized error list into the template. Call the LLM with a low temperature setting (e.g., 0.1) to maximize deterministic, schema-compliant output. Parse the response and validate every classification object against the expected schema: error_id must be a string, category must be one of the predefined enums (timeout, rate_limit, auth, validation, server_error, unknown), and confidence must be a float between 0.0 and 1.0. Implement a retry layer with exponential backoff (1s, 2s, 4s) for malformed JSON or schema validation failures, up to a maximum of 3 attempts. Any classification object that still fails validation after retries must be logged to a dead-letter queue for human review, not silently dropped.

For high-throughput production systems, batch errors into groups of 50–100 per LLM call to stay within token limits and manage per-call latency. Store the raw prompt, the raw model response, and the final validated output in an append-only log for full auditability. This allows you to replay classifications, debug model drift, and provide evidence for compliance reviews. Before deploying, run a suite of eval cases with known error types to ensure the prompt's classification accuracy meets your operational threshold. The next step is to define the alerting rules in your dashboard that will act on the structured taxonomy this pipeline produces.