Edge Case Detection and Human Triage Prompt

This prompt is designed for AI reliability engineers and platform developers who need to catch inputs that fall outside the reliable operating boundaries of a classification system. Its core job is not to classify, but to identify when classification is likely to fail in a dangerous way—specifically, when an input sits in a boundary region between classes, represents a novel pattern unseen in training, or exposes a gap in the system's distribution. The ideal user is someone instrumenting a production AI pipeline who needs a structured, auditable reason to halt automation and request human review, rather than letting the system silently misclassify an edge case with high confidence.

Use this prompt when the cost of a misclassification is high and you need a safety net beyond a simple low-confidence threshold. A standard confidence check will catch inputs where the model is uncertain between two known classes. This prompt targets more dangerous failure modes: inputs that are confidently classified but are actually out-of-distribution, inputs that combine features of multiple classes in a novel way, or inputs that exploit a known blind spot in your taxonomy. It should be deployed as a secondary analysis step, triggered after the primary classifier has made a prediction, to decide if that prediction should be trusted or overridden in favor of human triage.

Do not use this prompt as your primary classifier or for real-time, sub-millisecond decision loops. It is a reflective, analytical prompt that requires more tokens and reasoning time than a standard classification call. It is also inappropriate for low-stakes routing where the cost of an occasional misclassification is negligible. If you implement this, you must pair it with a logging and feedback mechanism that captures the human reviewer's final decision. Without that feedback loop, you cannot measure whether the prompt is correctly identifying true edge cases or just adding latency by escalating routine inputs. The next section provides the exact template to wire into your application.

This prompt template is designed to be integrated into a classification pipeline as a secondary guardrail. It receives an input that has already been classified with low confidence, flagged as ambiguous, or identified as a potential outlier by a primary classifier. Its job is not to re-classify the input but to analyze why it is an edge case, document the specific characteristics that make it difficult for automated routing, and produce a structured payload that a human reviewer or downstream triage system can act on immediately. The template uses square-bracket placeholders for all dynamic components, allowing you to inject the original input, the primary classifier's output, your system's capability taxonomy, and your risk tolerance level.

textCopy
You are an edge-case detection specialist for an AI classification system. Your task is to analyze inputs that the primary classifier could not route with high confidence and determine whether they represent a true edge case requiring human review or a routine low-confidence classification that can be resolved through clarification.

## INPUT
[INPUT]

## PRIMARY CLASSIFIER OUTPUT
Confidence: [CONFIDENCE_SCORE]
Predicted Intent: [PREDICTED_INTENT]
Alternative Intents: [ALTERNATIVE_INTENTS]
Uncertainty Source: [UNCERTAINTY_SOURCE]

## SYSTEM CAPABILITIES
Supported Intents: [SUPPORTED_INTENTS]
Out-of-Scope Categories: [OUT_OF_SCOPE_CATEGORIES]
Clarification Capabilities: [CLARIFICATION_CAPABILITIES]

## CONSTRAINTS
- Do not re-classify the input. Focus on edge-case characteristics.
- Distinguish between: (1) novel patterns outside training distribution, (2) boundary cases between supported intents, (3) genuinely ambiguous inputs, and (4) inputs with insufficient information.
- If the input is out-of-scope, state this explicitly rather than forcing a classification.
- Flag any safety, policy, or compliance concerns observed in the input.

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "edge_case_detected": boolean,
  "edge_case_type": "novel_pattern" | "boundary_case" | "genuine_ambiguity" | "insufficient_information" | "out_of_scope" | "routine_low_confidence",
  "edge_case_characteristics": [
    {
      "characteristic": string,
      "evidence_excerpt": string,
      "impact": string
    }
  ],
  "requires_human_review": boolean,
  "review_priority": "high" | "medium" | "low",
  "triage_summary": string,
  "suggested_clarification_questions": [string],
  "safety_concerns": [string],
  "recommended_action": "escalate_to_human" | "request_clarification" | "route_with_caveat" | "reject_out_of_scope"
}

## RISK LEVEL
[RISK_LEVEL]

## EXAMPLES
[EXAMPLES]

To adapt this template for your system, start by populating [SUPPORTED_INTENTS] and [OUT_OF_SCOPE_CATEGORIES] with your actual taxonomy. These are critical for the model to distinguish between boundary cases (where two supported intents overlap) and out-of-scope cases (where no supported intent applies). The [RISK_LEVEL] field should be set to high, medium, or low based on your domain's tolerance for misrouting. In high-risk domains like healthcare or finance, set this to high to bias the model toward escalation. The [EXAMPLES] placeholder should contain 2-4 few-shot examples showing correct edge-case detection for your specific taxonomy. Include at least one example of a routine low-confidence case that should not be escalated to prevent over-triage. After copying this template, validate that the output JSON schema matches your downstream triage system's expected contract before deploying to production.

This prompt is designed to sit between your primary classifier and your downstream automated workflows. It should be invoked when the primary classifier returns a confidence score below a configurable threshold, or when the input falls into a pre-identified distribution gap. The prompt's job is not to re-classify the input, but to characterize why it is an edge case and produce a structured triage payload that a human reviewer or a fallback queue can consume. Wire this as a synchronous call in your routing middleware, but ensure it has a strict timeout—edge case analysis should not become a bottleneck. If the prompt times out or fails, the input should be escalated directly to the human review queue with the original classifier's low-confidence metadata attached.

The implementation should include a validation layer that checks the structured output before routing to a human. Validate that the edge_case_type field matches one of your predefined categories (e.g., boundary_region, novel_pattern, distribution_gap, multi_intent_conflict). Reject outputs with hallucinated categories. Validate that the evidence_excerpts array contains verbatim substrings from the original input—this prevents the model from fabricating evidence. If validation fails, retry once with the validation errors injected into the prompt's [CONSTRAINTS] field. After a second failure, log the raw output and escalate with a validation_failure flag. For high-stakes domains, always route the final triage payload to a human review queue rather than an automated fallback model. The prompt's [RISK_LEVEL] parameter should be mapped to your queue priority: high maps to immediate review, medium to standard SLA, and low to batch review.

Model choice matters here. This prompt requires strong instruction-following and structured output discipline. Use a model that supports JSON mode or structured outputs natively (e.g., GPT-4o with response_format, Claude 3.5 Sonnet with tool use for the output schema). Avoid models with weak schema adherence for this task, as a malformed triage payload can silently drop an edge case into the wrong queue. Log every triage decision with the input hash, classifier confidence, edge case type, and reviewer outcome. This log becomes your dataset for recalibrating classification thresholds and identifying whether your primary classifier's boundary regions are shifting over time. Do not use this prompt to replace classifier retraining—use it to buy time while you collect the labeled edge cases you need to improve the system.