Fallback and Graceful Degradation System Prompt

This prompt is for reliability engineers and AI product developers who need to define a system-level behavioral contract for failure. Its job is not to answer a user's question, but to instruct the model on how to behave when it cannot answer, when a required tool is unavailable, when retrieved context is empty or contradictory, or when its internal confidence falls below a defined threshold. The ideal user is someone wiring an AI system into a production product where silent failures, hallucinated answers, or raw stack traces exposed to end users are unacceptable. You need this prompt when your application depends on external APIs, databases, or retrieval systems that can fail, time out, or return degraded results, and you want the model to handle those states with a predictable, safe, and user-appropriate response instead of guessing.

You should not use this prompt if your AI workflow is purely conversational and stateless, with no tool integrations or external dependencies that can fail. It is also the wrong approach if you are trying to fix a model that hallucinates on its own knowledge boundaries—that requires a refusal boundary prompt, not a degradation system. This prompt assumes you have already instrumented your application to detect tool failures, empty retrieval sets, or low-confidence scores and can inject those signals as structured context (e.g., [TOOL_STATUS: failed] or [CONFIDENCE_SCORE: 0.3]). The prompt does not perform the detection itself; it defines the model's response policy once the application signals a degraded state. For high-risk domains like healthcare or finance, this prompt must be paired with a human-in-the-loop escalation path—graceful degradation should include an offer to escalate, not just a polite refusal.

Before implementing this prompt, map out every failure mode your system can encounter: tool timeouts, malformed tool outputs, empty search results, contradictory evidence, missing required parameters, rate limits, and authentication failures. Each failure mode needs a corresponding degradation behavior defined in the prompt. The prompt template below uses placeholders like [FAILURE_SCENARIOS] and [DEGRADATION_ACTIONS] so you can inject your specific failure-to-behavior mappings. After deploying, test each path with failure injection—deliberately break tools, return empty context, and feed low-confidence signals to verify the model follows the degradation policy rather than improvising.

This prompt template establishes a behavioral contract for graceful degradation in AI products. It defines what the model should do when primary capabilities are unavailable—tools return errors, retrieved context is empty, confidence falls below threshold, or required inputs are missing. The template uses square-bracket placeholders so you can inject your specific fallback policies, tool names, confidence thresholds, and escalation paths without rewriting the core degradation logic.

textCopy
You are an AI assistant operating in a production system where tool failures, missing context, and low-confidence states are expected and must be handled gracefully.

## DEGRADATION LEVELS
When responding, assess your operational state against these levels and apply the corresponding behavior:

### LEVEL 1: FULL CAPABILITY
All required tools and context are available. Confidence is above [CONFIDENCE_THRESHOLD].
- Respond normally with complete answers.
- Include citations and source references where applicable.

### LEVEL 2: PARTIAL DEGRADATION
One or more non-critical tools failed, or some context is missing, but core capability remains.
- Provide the best answer you can with available information.
- Explicitly state what is missing: "I was unable to retrieve [MISSING_CAPABILITY], so this answer may be incomplete."
- If the missing information is critical to correctness, escalate to Level 3.

### LEVEL 3: SIGNIFICANT DEGRADATION
Critical tools failed, primary context is unavailable, or confidence dropped below [CONFIDENCE_THRESHOLD].
- Do not fabricate answers or guess.
- State clearly: "I cannot complete this request because [SPECIFIC_REASON]."
- Offer specific alternatives: "You could try [ALTERNATIVE_ACTION] or I can [FALLBACK_ACTION]."
- If [ESCALATION_ENABLED] is true, add: "I recommend escalating this to [ESCALATION_TARGET]."

### LEVEL 4: FULL OUTAGE
No tools are available, no context can be retrieved, or the system is in a degraded state that prevents meaningful responses.
- Respond only with: "I'm currently unable to assist with this request due to a system issue. Please try again later or contact [SUPPORT_CHANNEL]."
- Do not attempt partial answers or speculation.

## FAILURE DETECTION RULES
Apply these rules to determine your degradation level before responding:

1. **Tool Failure Detection**: If any tool call returns an error, timeout, or empty result, treat that tool as unavailable for this turn. Do not retry the same tool more than [MAX_RETRIES] times.

2. **Context Adequacy Check**: If retrieved context is empty, irrelevant to the query, or shorter than [MIN_CONTEXT_LENGTH], treat context as insufficient.

3. **Confidence Assessment**: Before responding, internally assess your confidence. If confidence is below [CONFIDENCE_THRESHOLD] for any factual claim, either qualify that claim explicitly or escalate to the appropriate degradation level.

4. **Input Validation**: If the user's request is missing required information and you cannot reasonably infer it, ask for clarification rather than proceeding with assumptions. This is not a degradation—it's normal operation.

## TOOL-SPECIFIC FALLBACKS
Apply these fallback behaviors for specific tool failures:

- **[TOOL_NAME_1]**: If unavailable, [FALLBACK_BEHAVIOR_1].
- **[TOOL_NAME_2]**: If unavailable, [FALLBACK_BEHAVIOR_2].
- **[TOOL_NAME_3]**: If unavailable, [FALLBACK_BEHAVIOR_3].

## RESPONSE FORMAT DURING DEGRADATION
When operating at Level 2 or 3, prefix your response with a degradation notice:

[DEGRADATION_NOTICE_FORMAT]

Example: "⚠️ Note: [SPECIFIC_ISSUE]. I'll provide what I can with available information."

## CONSTRAINTS
- Never hide degradation from the user. Transparency is required.
- Never fabricate tool results or context to appear fully capable.
- Do not apologize excessively. State the issue and move to alternatives.
- Log degradation events internally for observability, but do not expose internal system details to users.
- If [HUMAN_APPROVAL_REQUIRED] is true for an action, do not proceed even if tools are available—wait for approval.

To adapt this template, replace each square-bracket placeholder with your specific values. Start by defining your confidence threshold (typically 0.7-0.9 for high-stakes applications). Map your actual tool names and their fallback behaviors—for example, if a vector search tool fails, you might fall back to keyword search or declare the retrieval unavailable. Set MAX_RETRIES low (1-2) to avoid latency spirals. Define your degradation notice format to match your product's UX patterns. If you don't need tool-specific fallbacks, remove that section. Test the prompt by injecting failures at each degradation level and verifying the model follows the correct behavioral path without fabricating capabilities or hiding degradation from users.

This prompt is not a standalone artifact; it is a behavioral policy layer that must be integrated into your model request assembly pipeline. The system prompt should be injected as the first message in the messages array, before any user, tool, or conversation context. In a typical implementation, you will maintain this prompt as a versioned template in your prompt registry, substituting the [FAILURE_SCENARIOS], [FALLBACK_ACTIONS], and [ESCALATION_POLICY] placeholders at request time based on the specific tools and workflows active for that session. The prompt is designed to be model-agnostic, but you should test it against each model in your routing matrix, as different models interpret refusal and fallback instructions with varying levels of strictness.

At the application layer, you must implement a tool-call result interceptor that inspects the output of every tool invocation before it is returned to the model. If a tool returns an error code, a null result, a timeout, or a response that fails schema validation, the interceptor should inject a structured failure message into the conversation context rather than passing the raw error to the model. This structured message should include the tool name, the failure type, and any partial results. The system prompt then instructs the model how to interpret these structured failure messages and which fallback action to take. For high-risk workflows, add a confidence gate: if the model's response includes a low-confidence marker or an explicit abstention, route the output to a human review queue before it reaches the end user. Log every fallback trigger with the session ID, tool name, failure type, and chosen degradation path for later analysis.

Retry logic should live in the application harness, not in the prompt. If a tool call fails with a transient error such as a network timeout or rate limit, retry up to three times with exponential backoff before surfacing the failure to the model. For deterministic failures such as schema mismatches or authorization errors, do not retry; surface the failure immediately. Model choice matters: smaller or faster models may ignore nuanced fallback instructions under pressure, so test your degradation paths with the exact model and quantization level you intend to deploy. If you are using a model router, ensure that the fallback prompt is included in the system message regardless of which model is selected. Finally, monitor fallback frequency as a key reliability metric. A sudden spike in fallback triggers often indicates an upstream tool degradation, a schema change, or a context-window pressure issue that requires attention before it becomes a user-facing outage.