Fallback Model Routing Instruction Prompt

This prompt is a machine-to-machine instruction designed for a routing proxy or orchestration layer, not for direct human interaction. Its primary job is to activate a fallback model when a primary model request fails due to specific, detectable conditions: capacity exhaustion (e.g., rate limiting, 503 errors), unacceptable latency, a hard refusal from a safety policy, or an output that fails a defined quality threshold. The prompt's purpose is to assemble a new, self-contained request for a secondary model, carrying forward the essential context and explicitly constraining the fallback model's behavior to prevent it from attempting actions it isn't authorized or equipped to perform.

You should integrate this prompt into your application's failure-handling middleware. When your primary model call throws a qualifying exception, your code must capture the original [INPUT], the [CONVERSATION_HISTORY], and the specific [FAILURE_REASON]. These are injected into the prompt's placeholders before it is sent to the designated fallback model. The prompt instructs the fallback model to acknowledge its secondary role, operate within a strict [CAPABILITY_CONSTRAINT], and produce an output that conforms to the original [OUTPUT_SCHEMA] where possible. A critical implementation detail is to log the routing_criteria and fallback_model_id alongside the response for observability, allowing you to monitor cascade frequency and fallback model performance.

Do not use this prompt for simple retry logic against the same model; that wastes tokens and latency on a model that has already failed. It is also not a substitute for application-level routing decisions based on cost or intent classification—those decisions should be made before a request fails. This prompt is exclusively for the reactive 'break-glass' scenario where a primary model is unavailable or non-performant, and you need a structured, constrained handoff to a backup system to maintain service availability.

This prompt template is designed to be injected as the system instruction for a secondary, fallback model in a model-cascading architecture. It is activated when the primary model's request fails due to a timeout, an error response, or a content filter refusal. The prompt's core job is to prevent a dead-end user experience by acknowledging the handoff, summarizing what it knows, and transparently declaring its reduced capability set. It is not a general-purpose assistant prompt; it is a reliability contract.

codeCopy
<SYSTEM>
You are the fallback assistant. The primary AI model that was handling this user's request is currently unavailable. Your job is to provide a graceful, honest, and helpful degraded experience. You must follow these rules in strict priority order:

1.  **Acknowledge the Handoff:** Begin your first response by briefly and calmly informing the user that the primary assistant is temporarily unavailable and that you are a backup system with limited capabilities.
2.  **Summarize the Context:** Review the provided conversation history in [CONVERSATION_HISTORY]. Identify the user's last clear intent and any unresolved questions. State what you understand the user was trying to do.
3.  **Declare Your Capabilities:** You are a fallback model. Your capabilities are strictly limited to the following:
    - Answering general knowledge questions based on your training data.
    - Summarizing the conversation so far.
    - Helping the user draft a message to save for later.
    - Guiding the user to alternative support channels listed in [ESCALATION_PATHS].
    You are FORBIDDEN from performing any action not on this list, including calling tools, accessing external data, or making changes to user accounts.
4.  **Offer a Path Forward:** Based on the user's intent, suggest one of the following:
    - Retrying the request later when the primary system may be available.
    - Saving the current state or draft for later use.
    - Escalating to a human support channel as defined in [ESCALATION_PATHS].
5.  **Maintain Tone:** Your tone must be helpful, apologetic for the inconvenience, and transparent about your limitations. Do not pretend to be the primary assistant. Do not fabricate capabilities.
6.  **Safety First:** The safety policies in [SAFETY_POLICY] are your highest priority and override all other instructions. If a request violates these policies, you must refuse using the standard refusal language.

[CONVERSATION_HISTORY]
[ESCALATION_PATHS]
[SAFETY_POLICY]
</SYSTEM>

To adapt this template, your orchestration layer must replace the bracketed placeholders before the request reaches the fallback model. [CONVERSATION_HISTORY] should be a compressed, structured summary of the last N turns, not the raw full history, to manage context window limits. [ESCALATION_PATHS] must be a concrete list of available human queues, help center URLs, or support ticket forms. [SAFETY_POLICY] should be a copy of the same top-level safety and refusal instructions used by the primary model to ensure consistent policy enforcement. The prompt's strict capability declaration is a defense-in-depth measure; it prevents the fallback model from hallucinating tool calls or actions it cannot execute, which is a common failure mode in degraded states.

This prompt is designed to sit inside a model routing proxy or orchestration layer, not as a standalone chat instruction. When a primary model returns a failure signal—such as a refusal, a timeout, a malformed output that fails validation, or an explicit FALLBACK_REQUIRED flag—the routing layer should inject this prompt as the system instruction for the fallback model call. The prompt expects the routing layer to provide [ORIGINAL_USER_REQUEST], [PRIMARY_MODEL_FAILURE_REASON], [PRIMARY_MODEL_PARTIAL_OUTPUT] (if any), and [FALLBACK_MODEL_CAPABILITIES] as runtime variables. Do not hardcode these values into the prompt template; they must be populated dynamically by the harness at invocation time.

Wiring pattern: In a typical proxy implementation, you'll maintain a model registry with capability tags (e.g., supports_function_calling, max_context_tokens, supports_vision). When the primary model fails, the router queries this registry to select an appropriate fallback and populates [FALLBACK_MODEL_CAPABILITIES] with a concise, machine-readable list of what the fallback can and cannot do. The harness should also attach the original conversation context, but truncate it to fit the fallback model's context window before injection. A pre-call validator should confirm that all required placeholders are non-empty; if [PRIMARY_MODEL_PARTIAL_OUTPUT] is null, the harness should substitute an explicit [NO_PARTIAL_OUTPUT_AVAILABLE] token to prevent the fallback model from hallucinating prior work.

Validation and retries: After the fallback model responds, run the output through the same schema validator used for the primary model. If the fallback also fails validation, do not retry the fallback model more than once with the same prompt—instead, escalate to a human review queue with the full failure chain logged. For high-risk domains, implement a mandatory human-approval gate before any fallback-generated output reaches the end user. Log every routing decision with: primary model ID, failure reason, fallback model ID, latency for both calls, placeholder population audit (were all variables filled?), and whether the fallback output passed validation. This trace data is essential for tuning routing thresholds and detecting when fallback models are being overused due to primary model degradation.

What to avoid: Do not use this prompt as a catch-all error handler that swallows failures silently. If the fallback model also fails, surface a clear degradation message to the user rather than looping through additional models indefinitely. Never pass raw stack traces or internal error codes into [PRIMARY_MODEL_FAILURE_REASON]—sanitize them into user-safe descriptions. Finally, test your harness with simulated primary failures (timeouts, refusals, malformed JSON, empty responses) before production deployment to ensure the routing logic correctly populates all placeholders and that the fallback model's output contract matches what downstream consumers expect.