Fallback Chain Sequence Audit Prompt

This prompt is designed for agent developers and AI platform engineers who need to audit multi-model fallback chains from production traces. The core job-to-be-done is reconstructing the sequence of model attempts that occurred when a primary model failed and the system tried several alternatives sequentially. You should use this prompt when you have raw trace data showing multiple model invocations for a single logical request and you need to understand whether the fallback chain is efficient or wasteful. The ideal user has access to production observability data, understands the configured fallback policies, and needs to produce optimization recommendations backed by trace evidence rather than intuition.

The prompt requires structured trace context as input, including timestamps, model identifiers, request parameters, response metadata, error codes, and latency measurements for each attempt in the chain. Without this granular trace data, the prompt cannot accurately reconstruct attempt order or measure cumulative latency. You should also provide the configured fallback policy rules so the prompt can compare intended behavior against observed behavior. The output includes a sequence diagram representation, identification of redundant or unnecessary fallback steps, cumulative latency measurements, and specific optimization recommendations. This is not a real-time routing prompt—it is a post-hoc audit tool for offline trace analysis.

Do not use this prompt when you lack complete trace data for the full fallback chain, when you need real-time routing decisions rather than retrospective analysis, or when the fallback chain involves fewer than two model attempts. This prompt is also inappropriate for diagnosing why the primary model failed—use the Fallback Activation Root-Cause Prompt for that investigation. For teams running continuous monitoring, the output of this prompt can feed into the Fallback Frequency Trend Analysis Prompt and the Fallback Latency Impact Analysis Prompt to build a complete picture of fallback health. Before running this prompt at scale, validate its output against a small set of manually reviewed traces to ensure the sequence reconstruction logic matches your trace schema.

Use this prompt to analyze a production trace where multiple models were attempted sequentially before a final response was delivered. The prompt is designed to reconstruct the exact attempt order, measure the latency contribution of each step, and identify redundant or unnecessary fallback activations. This is essential for AI platform engineers and SREs who need to optimize routing configurations and reduce cumulative request latency. Before executing, ensure you have extracted the relevant trace segment containing all model invocation events, their timestamps, error codes, and the final response metadata.

markdownCopy
You are an AI platform reliability engineer auditing a multi-model fallback chain from a production trace. Your task is to reconstruct the sequence of model attempts, diagnose inefficiencies, and produce a structured optimization report.

## INPUT DATA

[TRACE_JSON]

## INSTRUCTIONS

1. **Parse the trace**: Extract every model invocation event from the provided trace data. Identify the model name, start timestamp, end timestamp, and outcome (success, error code, timeout) for each attempt.
2. **Reconstruct the sequence**: Order the attempts chronologically. Label the first attempt as "Primary" and subsequent attempts as "Fallback 1", "Fallback 2", etc.
3. **Calculate latency**: For each attempt, compute the duration in milliseconds. Calculate the cumulative latency from the start of the first attempt to the end of the final successful attempt.
4. **Identify the trigger**: For each fallback activation, determine the trigger condition (e.g., timeout, 4xx error, 5xx error, quality threshold breach, capability mismatch).
5. **Flag redundancies**: Identify any fallback attempts that were unnecessary. A fallback is redundant if:
   - It was attempted after a success response was already received.
   - It used a model with identical or inferior capabilities to a previously failed model without a differentiating factor (e.g., lower cost, different region).
   - It was triggered by a transient error that a retry on the same model would have resolved faster.
6. **Generate a sequence diagram**: Produce a Mermaid.js sequence diagram showing the request flow through each model, including error responses and the final success.

## OUTPUT FORMAT

Respond with a JSON object conforming to this schema:

```json
{
  "trace_id": "string",
  "total_attempts": "integer",
  "cumulative_latency_ms": "integer",
  "attempts": [
    {
      "sequence": "integer",
      "role": "Primary | Fallback 1 | Fallback 2",
      "model": "string",
      "start_timestamp": "ISO 8601",
      "end_timestamp": "ISO 8601",
      "latency_ms": "integer",
      "outcome": "success | error",
      "error_code": "string | null",
      "trigger_condition": "string | null",
      "redundant": "boolean",
      "redundancy_reason": "string | null"
    }
  ],
  "redundancy_summary": {
    "total_redundant_attempts": "integer",
    "wasted_latency_ms": "integer",
    "primary_cause": "string"
  },
  "sequence_diagram_mermaid": "string",
  "optimization_recommendations": [
    {
      "finding": "string",
      "severity": "high | medium | low",
      "recommendation": "string",
      "expected_latency_savings_ms": "integer"
    }
  ]
}

CONSTRAINTS

• Do not invent data. If a field cannot be determined from the trace, use null.
• If the trace contains only one model attempt, set total_attempts to 1 and leave redundancy_summary fields as 0 or null.
• The Mermaid diagram must be syntactically valid and renderable in a standard Mermaid viewer.
• Base all optimization recommendations strictly on evidence present in the trace.

To adapt this prompt for your environment, replace the [TRACE_JSON] placeholder with your actual trace data. The trace must include at minimum: a unique trace identifier, an array of model invocation events with timestamps, and outcome metadata. If your observability platform uses a different schema, pre-process the trace into the expected structure before calling the model. For high-stakes production audits, always route the output through a validation step that checks the JSON schema, verifies timestamp ordering, and flags any null fields that should have been populated. Pair this prompt with a human review step before applying any recommended configuration changes to your model router.

This prompt is designed to be invoked programmatically as part of a trace analysis pipeline, not as a one-off chat interaction. The primary integration point is a trace ingestion or replay system that has already captured the full sequence of model calls, timestamps, error codes, and token counts for a single session. Before calling the prompt, assemble a structured [TRACE_CONTEXT] object containing the ordered list of attempts, each with model identifier, start time, end time, latency, status (success, timeout, error), and any error metadata. The prompt expects this context to be complete and chronologically ordered; incomplete traces will produce unreliable audits.

Wire the prompt into a batch or streaming trace processor. For each session trace that contains two or more model attempts, construct the prompt payload and call a capable long-context model (GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro). Validation layer: Before passing the output downstream, validate that the returned JSON contains the required fields: attempt_sequence, redundancy_flags, cumulative_latency_breakdown, and optimization_recommendations. If the model returns malformed JSON, apply a repair step using a structured output repair prompt or retry with stricter schema instructions. Retry logic: Retry once on validation failure with the error message appended to [CONSTRAINTS]. If the second attempt fails, log the trace ID and escalate for human review rather than silently dropping the audit.

Logging and observability: Log every audit invocation with trace ID, model used for audit, token consumption, latency, and validation status. Store the structured audit output alongside the original trace for downstream querying. Human review gate: For traces where the prompt flags redundant attempts or recommends configuration changes, route the audit to a review queue before applying any automated routing rule changes. The prompt identifies optimization opportunities, but modifying fallback chain configuration should require human approval. Model choice: Use a model with strong JSON adherence and long-context handling. Avoid small or local models for this task unless you have validated their ability to maintain schema compliance across complex multi-attempt traces. Cost control: Batch audit runs during low-traffic windows and cache prompt prefixes to reduce per-invocation token costs. If processing high-volume traces, sample sessions with fallback activations rather than auditing every session.