Model Routing SLA Compliance Prompt

This prompt is for AI operations teams and platform engineers who need to verify that their model router is meeting defined Service Level Agreements (SLAs) for latency, availability, and output quality. The job-to-be-done is turning raw production trace segments into a structured compliance report. Instead of manually cross-referencing trace timestamps against a spreadsheet of SLA thresholds, you feed the trace data and the SLA definitions into this prompt to get a computed compliance rate, breach patterns, and a dashboard-ready summary. The ideal user is someone responsible for AI infrastructure reliability who can provide both the trace data and the specific, measurable SLA targets the system is supposed to hit.

Use this prompt when you have a batch of production traces that include per-request metadata such as model selected, end-to-end latency, HTTP status codes, and any quality scores your system already generates. You must also provide a clear SLA specification, for example: 'Model X must respond in under 800ms for 99% of requests' or 'Model Y must have a success rate above 99.5%'. The prompt is designed to compute compliance per model and per time window, so it works best with traces that span a meaningful operational period, like an hour, a day, or a deployment window. It is not a real-time alerting system; it is a batch analysis tool for periodic review, post-deployment validation, or incident postmortem.

Do not use this prompt if you lack concrete SLA thresholds or if your traces do not contain the metrics needed to evaluate them. The prompt cannot invent what 'good' looks like for your system. It also should not be used as the sole arbiter for financial penalties or external contractual reporting without human review. The output is a compliance dashboard feed, meaning it is structured for ingestion into monitoring tools or internal reports, but the calculations and breach classifications must be spot-checked before they become an official record. If your traces contain personally identifiable information (PII) or sensitive payloads, redact them before passing them to this prompt.

This prompt ingests a batch of trace segments from model routing decisions and evaluates each against defined SLA thresholds for latency, availability, and quality. It is designed to be wired into an automated compliance pipeline where trace data is collected from production observability systems and fed to the model for structured analysis. The output is a compliance dashboard feed that surfaces breach patterns, per-model compliance rates, and time-window summaries.

textCopy
You are an SLA compliance auditor for an AI model routing system. Your task is to evaluate a batch of routing trace segments against defined Service Level Agreement (SLA) thresholds and produce a structured compliance report.

## INPUT DATA
[TRACE_SEGMENTS]

## SLA THRESHOLDS
- Latency SLA (p95): [LATENCY_THRESHOLD_MS] ms
- Availability SLA: [AVAILABILITY_THRESHOLD_PCT]% success rate
- Quality SLA: [QUALITY_THRESHOLD] (defined as [QUALITY_METRIC_DEFINITION])

## TIME WINDOW
Evaluation window: [START_TIMESTAMP] to [END_TIMESTAMP]
Aggregation granularity: [AGGREGATION_GRANULARITY] (e.g., hourly, daily, per-model)

## OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "evaluation_window": {
    "start": "ISO timestamp",
    "end": "ISO timestamp",
    "granularity": "string"
  },
  "overall_compliance": {
    "latency_compliance_pct": number,
    "availability_compliance_pct": number,
    "quality_compliance_pct": number,
    "overall_sla_met": boolean
  },
  "per_model_compliance": [
    {
      "model_id": "string",
      "total_requests": number,
      "latency_p95_ms": number,
      "latency_breach_count": number,
      "latency_compliance_pct": number,
      "availability_pct": number,
      "availability_breach_count": number,
      "availability_compliance_pct": number,
      "quality_score": number,
      "quality_breach_count": number,
      "quality_compliance_pct": number,
      "sla_met": boolean
    }
  ],
  "breach_patterns": [
    {
      "pattern_type": "string (e.g., 'latency_spike', 'availability_drop', 'quality_degradation')",
      "affected_models": ["model_id"],
      "time_range": {"start": "ISO", "end": "ISO"},
      "severity": "string (critical|high|medium|low)",
      "description": "string",
      "recommended_action": "string"
    }
  ],
  "anomalies": [
    {
      "model_id": "string",
      "metric": "string",
      "observed_value": number,
      "expected_baseline": number,
      "deviation_pct": number,
      "timestamp": "ISO"
    }
  ]
}

## CONSTRAINTS
- Only evaluate traces within the specified time window.
- If a trace segment is missing latency, availability, or quality data, mark that metric as null for the affected model and exclude it from breach counts.
- Flag any model with fewer than [MIN_SAMPLE_SIZE] requests in the window as "insufficient_data" and set sla_met to null.
- Breach patterns must be supported by at least [BREACH_PATTERN_MIN_OCCURRENCES] occurrences before reporting.
- Anomalies are defined as deviations exceeding [ANOMALY_DEVIATION_THRESHOLD_PCT]% from the baseline.
- If the overall SLA is not met, include a summary of the top three contributing factors.
- Do not fabricate data. If a metric cannot be computed, state why.

## RISK LEVEL
[RISK_LEVEL]

## EXAMPLES
[EXAMPLES]

To adapt this template, replace each square-bracket placeholder with concrete values from your routing infrastructure. The [TRACE_SEGMENTS] placeholder expects structured trace data containing at minimum: trace_id, model_id, timestamp, latency_ms, status_code, and quality_score. If your traces use different field names, add a mapping instruction in the constraints section. The [QUALITY_METRIC_DEFINITION] placeholder should describe how quality is measured in your system (e.g., 'BLEU score above 0.8', 'human eval rating >= 4/5', 'toxicity score below 0.1'). For high-risk production environments where SLA breaches trigger financial penalties or customer-facing degradation, always route the output through a human review step before publishing to dashboards or triggering automated remediation. Validate the JSON output against the schema programmatically before ingestion, and log any parse failures with the raw model response for debugging.

This prompt is designed to run as a batch trace analysis job, not a real-time synchronous call. You should feed it trace segments that have already been enriched with SLA threshold metadata—latency budgets, availability targets, and quality score minimums per model. The prompt expects a structured input payload containing trace IDs, timestamps, model identifiers, observed metrics, and the corresponding SLA thresholds. Running this prompt against raw, unenriched traces will produce unreliable results because the model cannot infer your SLA definitions from bare latency numbers alone.

Input assembly should happen in application code before the prompt is called. For each trace segment, join the observed metrics (p95 latency, HTTP status, quality eval score) with the SLA policy for that model and time window. The [TRACE_SEGMENTS] placeholder expects a JSON array where each object includes trace_id, timestamp, model_id, observed_latency_ms, observed_availability, observed_quality_score, sla_latency_ms, sla_availability, and sla_quality_score. If your observability stack doesn't capture quality scores inline, run a separate eval prompt first and merge the results before calling this SLA compliance prompt. Do not ask the model to compute missing metrics—it will hallucinate plausible values.

Validation and retry logic is critical because the output feeds a dashboard. After the model returns the [OUTPUT_SCHEMA] JSON, validate that every trace_id from the input appears in the output with a compliant boolean, a breach_type array, and a severity enum value. If the output is missing records, contains malformed enums, or has breach_type values that don't match the input SLA dimensions, retry once with a repair prompt that includes the validation errors. If the second attempt still fails, log the failed batch and escalate for human review rather than silently ingesting partial data. The severity field should be constrained to `[

none

warning

critical