Circuit-Breaker-Aware Router Prompt Template

This prompt is for site reliability engineers and platform engineers building AI dispatch systems that must remain resilient when downstream services fail. Use it when your router needs to make a real-time decision about where to send a request based on the current health of your infrastructure. The prompt ingests circuit breaker states, health check results, and error budget status to produce a dispatch decision that avoids degraded or failed services. It is designed for systems where a wrong routing decision causes cascading failures, not just a single bad response.

Do not use this prompt for simple static routing tables or when all downstream services are guaranteed to be healthy. This prompt assumes you have a live feed of service health data and that the model's decision will be executed by an orchestration layer that enforces the circuit breaker logic in code. The model's role is to interpret the health signals and produce a structured routing decision, but the actual circuit breaking, retry enforcement, and fail-open safety mechanisms must be implemented in the application layer. The prompt is not a replacement for infrastructure-level circuit breakers; it is a decision layer that sits on top of them.

Before deploying this prompt, verify that your health data feed is reliable and low-latency. Stale health data will cause the model to route to degraded services. Implement a hard timeout on the model's decision and a fail-open default route that preserves system availability even if the model call fails. Test the prompt against scenarios where multiple services degrade simultaneously, where error budgets are exhausted, and where health checks return ambiguous results. The eval suite should include checks for cascading failure prevention, fail-open safety, and correct prioritization of healthy services.

This prompt template is designed to be the core instruction set for an AI routing agent. It forces the model to act as a resilient dispatcher, making decisions based on the provided circuit breaker states, health check results, and defined degradation rules. You must replace every square-bracket placeholder with the concrete topology, policies, and constraints of your production environment before use. The prompt is structured to produce a deterministic, machine-readable JSON output that your application harness can parse and execute.

markdownCopy
# SYSTEM DIRECTIVE
You are a resilient service dispatcher. Your only job is to route a request to the most appropriate downstream service based on the provided topology, current circuit breaker states, health check results, and defined degradation rules. You do not process the request itself.

# ROUTING TOPOLOGY
[TOPOLOGY_DEFINITION]
# Example:
# - Service A: primary, handles "payment" intents.
# - Service B: secondary, handles "payment" intents, higher latency.
# - Service C: handles "account" intents.

# CURRENT SYSTEM STATE
- Circuit Breaker States: [CIRCUIT_BREAKER_STATES]
  # Example: {"Service A": "OPEN", "Service B": "HALF_OPEN", "Service C": "CLOSED"}
- Health Check Results: [HEALTH_CHECK_RESULTS]
  # Example: {"Service A": {"status": "FAILING", "latency_p99_ms": 5000}, "Service B": {"status": "DEGRADED", "latency_p99_ms": 800}}
- Error Budget Remaining: [ERROR_BUDGETS]
  # Example: {"Service A": 0, "Service B": 45}

# DEGRADATION RULES
[DEGRADATION_RULES]
# Example:
# 1. Never route to a service with an OPEN circuit breaker.
# 2. If primary service is OPEN, route to secondary if its circuit is CLOSED or HALF_OPEN and error budget > 0.
# 3. If no service is available for an intent, route to the "fallback" queue.
# 4. A service with a FAILING health check should be treated as OPEN.

# REQUEST TO ROUTE
- Request Intent: [REQUEST_INTENT]
- Request Payload ID: [REQUEST_ID]
- Priority: [REQUEST_PRIORITY]

# OUTPUT INSTRUCTIONS
Produce a single JSON object with the following schema. Do not include any other text.
{
  "decision": {
    "target_service": "<service_name_or_fallback>",
    "action": "ROUTE" | "QUEUE" | "REJECT",
    "reason": "<concise explanation referencing the specific rule and state that led to this decision>"
  },
  "degradation_metadata": {
    "is_degraded": true | false,
    "failed_primary": "<service_name_or_null>",
    "triggered_rule": "<the specific rule from DEGRADATION_RULES that was applied>"
  }
}

To adapt this, start by encoding your service topology in a structured format like JSON inside the [TOPOLOGY_DEFINITION] placeholder. The [DEGRADATION_RULES] are the most critical part; they must be written as explicit, deterministic logic statements that a model can follow without ambiguity. Avoid vague rules like 'prefer faster services.' Instead, use concrete conditions: 'Route to Service B only if Service A is OPEN and Service B's error budget is greater than 10.' After pasting this into your harness, run it against a test suite of known system states to ensure the model's output matches your expected routing logic before deploying to production.

This prompt is not a standalone classifier; it is a decision node inside a resilient dispatch loop. The application layer must supply the prompt with live circuit breaker state, downstream health metrics, and error budget status before invocation. The model's output is a structured routing decision that the harness validates, logs, and executes. Because the prompt can recommend fail_open (route to a healthy fallback) or fail_closed (reject the request), the harness must enforce safety constraints: a fail_open decision must never route to a known-dead endpoint, and a fail_closed decision must produce a user-visible degradation notice, not a silent drop.

Wire the prompt into a dispatch function that runs after intent classification but before request execution. The function should: (1) fetch current circuit breaker states and health check results from your service mesh or monitoring store; (2) assemble the prompt with [CIRCUIT_STATES], [HEALTH_METRICS], [ERROR_BUDGETS], [REQUEST_CONTEXT], and [ROUTING_TABLE] populated from live data; (3) call the model with temperature=0 and a strict JSON output schema; (4) validate the response against the expected schema, rejecting any decision that references an unknown queue or a circuit that is forced-open by policy; (5) log the decision, circuit states, and model reasoning to your observability platform for post-incident analysis. If validation fails, retry once with the validation error injected into the prompt as additional [CONSTRAINTS]. If the retry also fails, escalate to the on-call rotation and route the request to a static fallback queue defined in infrastructure config, not model output.

Model choice matters. Use a fast, instruction-following model (GPT-4o-mini, Claude Haiku, or equivalent) because this prompt runs on the critical path of every request. Latency budget should be under 500ms for the model call. Cache circuit breaker states locally with a short TTL (5-15 seconds) rather than querying your service mesh on every invocation. For high-throughput systems, consider batching circuit state updates and running the prompt only when circuit states change or error budgets cross thresholds, using a cached routing decision otherwise. Always instrument the dispatch function with metrics: router_decision_total by outcome, router_fail_open_total, router_validation_failure_total, and router_latency_seconds. These metrics feed back into your error budgets and capacity planning.

The most dangerous failure mode is a stale circuit state. If your health check data is 30 seconds old and a downstream service has just failed, the prompt will route traffic to a dead endpoint. Mitigate this with client-side retries, request hedging, and a short deadline on downstream calls. The second most dangerous failure mode is the model hallucinating a queue name that passes schema validation but doesn't exist in your infrastructure. Prevent this by validating all queue identifiers against a static allowlist in the harness, not in the prompt. The third failure mode is cascading: if the fallback queue becomes overloaded because all traffic is failing open to it, you've traded one outage for another. Monitor fallback queue depth and set alerts before it saturates. For high-risk deployments, add a human approval step when the prompt recommends routing more than a configured percentage of traffic to fallback queues within a time window.