gRPC Deadline Exceeded and Unavailable Retry Prompt Template

This prompt is designed for service engineers and SRE teams who need to automate recovery decisions when a gRPC call fails with DEADLINE_EXCEEDED or UNAVAILABLE status codes. The job-to-be-done is interpreting a structured gRPC error context, calculating a safe new deadline, and producing a machine-readable instruction that a retry harness can execute directly. The ideal user is an engineer integrating this prompt into an incident response automation or a custom gRPC interceptor that catches failures and calls an LLM before blindly retrying or failing over.

Use this prompt when you have access to the original gRPC status code, the current retry count, the original deadline, and the target service's SLO. It is particularly effective for unary RPCs where idempotency is guaranteed. Do not use this prompt for streaming RPCs without additional context about the last received message offset, as the retry semantics differ significantly. Avoid using it when the error is a deterministic application-layer failure (e.g., INVALID_ARGUMENT or FAILED_PRECONDITION), as retrying will not resolve the root cause. The prompt assumes the caller has already verified that the error is transient and worth retrying.

Before wiring this into a production harness, ensure you have a clear retry budget defined. The prompt will respect a maximum retry count, but it is your responsibility to enforce circuit-breaker state and total latency budgets outside the model call. If the prompt recommends a fallback to a REST transport, you must have that endpoint pre-configured and tested. The model will not invent a fallback URL; it will only signal that a fallback is the correct next step. For high-risk financial or healthcare transactions, always route the final retry decision through a human approval queue, even if the model's instruction is machine-readable.

This template is designed to be injected into a retry harness. It instructs the model to act as a gRPC reliability expert, interpreting the specific error code, current retry count, and method type to produce a machine-readable decision. The prompt forces a clear distinction between transient failures that benefit from a deadline recalculation and persistent failures that require a transport fallback, such as switching from gRPC to REST.

markdownCopy
You are a gRPC reliability expert. Analyze the failed gRPC call details and produce a structured retry decision.

**Input Details:**
- Service Method: [SERVICE_METHOD]
- Call Type: [UNARY_OR_STREAMING]
- gRPC Status Code: [STATUS_CODE]
- Error Message: [ERROR_MESSAGE]
- Current Retry Count: [RETRY_COUNT]
- Maximum Retries Allowed: [MAX_RETRIES]
- Original Deadline (ms): [ORIGINAL_DEADLINE_MS]
- Available Fallback Transports: [AVAILABLE_TRANSPORTS]

**Decision Constraints:**
- For DEADLINE_EXCEEDED, recalculate a new deadline using exponential backoff: new_deadline = original_deadline * (2 ^ retry_count).
- For UNAVAILABLE, check if the current retry count is less than the maximum allowed. If so, retry with the same deadline.
- If retries are exhausted for the primary transport, select the first available fallback from [AVAILABLE_TRANSPORTS] (e.g., REST).
- For streaming calls, a retry is only valid if the stream can be resumed from the last received sequence ID. If no sequence ID is available, the decision must be NO_RETRY.

**Output Schema:**
Return a valid JSON object with the following structure:
{
  "decision": "RETRY" | "FALLBACK" | "NO_RETRY",
  "reasoning": "A concise explanation of the decision.",
  "retry_parameters": {
    "new_deadline_ms": <integer>,
    "backoff_delay_ms": <integer>,
    "resume_sequence_id": <string or null>
  },
  "fallback_transport": "<string or null>"
}

To adapt this template, replace the square-bracket placeholders with values from your gRPC interceptor or service mesh telemetry. The [AVAILABLE_TRANSPORTS] field should be a list derived from your service's routing configuration. The critical adaptation for production is ensuring the resume_sequence_id logic is wired to your streaming client's internal state; a null value for a streaming call must always result in a NO_RETRY decision to prevent data loss. Before deployment, validate the output JSON against your application's schema to catch any malformed responses from the model.

The gRPC Deadline Exceeded and Unavailable retry prompt is not a standalone decision engine—it is a structured reasoning step inside a broader retry harness. The harness is responsible for extracting the gRPC status code, current deadline, retry count, and request metadata from the failed call, injecting them into the prompt's placeholders, and then executing the model's output as a machine-readable retry instruction. The prompt must be treated as a structured decision generator, not a free-text advisor. Its output should be parsed into a typed retry action: RETRY_WITH_DEADLINE, FALLBACK_TO_REST, FAIL_FAST, or ESCALATE. The harness validates that the action is one of these known values before acting on it.

Streaming vs. unary differentiation is the most critical harness check. For unary RPCs, a full retry is safe when idempotent. For server-side streaming calls, the harness must extract the last received sequence number or resume token from the error context and inject it into the prompt's [STREAM_STATE] placeholder. The prompt should then produce a RESUME_STREAM action with a resume offset rather than a naive RETRY. For client-side streaming, the harness must decide whether to replay the entire stream or request a checkpoint from the server before resuming. The harness should reject any retry action that does not match the RPC type—e.g., a RESUME_STREAM action for a unary call is a validation failure and should trigger an ESCALATE fallback.

Deadline recalculation must happen outside the model. The prompt receives the original deadline and elapsed time via [DEADLINE_CONTEXT] and [ELAPSED_MS], and it may suggest a new deadline in its output. The harness must enforce a maximum cumulative deadline derived from the upstream SLO. If the model's suggested deadline exceeds this bound, the harness clamps it and logs the override. Additionally, the harness must track retry budget exhaustion: if the retry count exceeds a configured maximum or the cumulative latency approaches the upstream caller's deadline, the harness short-circuits the prompt entirely and emits a FAIL_FAST or ESCALATE action without invoking the model.

Fallback to REST transport requires pre-configured endpoint mappings. The prompt's FALLBACK_TO_REST action must include a suggested REST endpoint path, but the harness validates that path against a known allowlist of REST fallback endpoints. If the model hallucinates an unmapped endpoint, the harness rejects the fallback and escalates. The harness must also inject the original gRPC request payload into the REST call with proper content-type headers and authentication token forwarding. All fallback calls must carry an X-Grpc-Fallback-Reason header for observability. Log every retry decision—model-invoked or harness-short-circuited—with the gRPC status code, retry count, chosen action, and deadline consumed.

Model choice and latency budget: this prompt is latency-sensitive because it sits on the critical path of request recovery. Use a fast, small model (e.g., a distilled or quantized variant) with a strict inference timeout—typically 200–500 ms. If the model does not respond within that window, the harness must fall back to a static retry policy (exponential backoff with jitter, capped retries) and log the model timeout. Never let the retry decision prompt become a bottleneck that worsens tail latency. Pre-warm model endpoints and consider caching identical error signatures for short TTLs to skip model invocation entirely for repeated, identical failures.