Frozen Prompt Monitoring Alert Configuration Prompt

This prompt is designed for observability engineers and SRE teams who need to translate a behavioral baseline into precise, actionable alert configurations. The core job-to-be-done is generating monitoring rules that detect when a frozen system prompt's production behavior deviates from its captured baseline. The ideal user has already completed a freeze event, meaning a Production Behavior Baseline Capture has been executed and a stable snapshot of expected inputs and outputs exists. Without this baseline, the prompt cannot produce meaningful thresholds and will generate generic, high-noise alerts.

Use this prompt when you have a concrete baseline dataset and need to define alerts for specific degradation patterns, such as increased refusal rates, format drift, or sentiment shifts. The prompt requires structured inputs including the baseline data, the target monitoring platform (e.g., Datadog, Grafana), and the acceptable deviation tolerance. It produces a strict output schema containing alert names, severity levels (e.g., P1 for critical drift, P3 for minor anomalies), evaluation windows, and direct links to remediation runbooks. Do not use this prompt if the system prompt is still under active iteration or if you lack a quantitative baseline, as the resulting thresholds will be arbitrary and cause immediate alert fatigue.

Before executing this prompt, ensure you have a validated baseline and clear operational runbooks. The prompt's value is in automating the translation of statistical baselines into monitoring syntax, not in discovering what to monitor. After generating the alert definitions, you must validate them against historical data to check the signal-to-noise ratio and ensure the alert coverage gaps identified in the eval checks are addressed before deployment to production.

This prompt template produces a structured alert configuration for detecting when a frozen system prompt's behavior degrades in production. It requires you to supply the frozen prompt's baseline behavior description, the metrics you are monitoring, the alerting platform constraints, and the acceptable degradation thresholds. The output is a set of alert rule definitions with severity levels, threshold values, evaluation windows, and runbook links—ready for review and deployment into your observability stack.

textCopy
You are an observability engineer configuring production alerts for a frozen system prompt. The system prompt has been locked and must not change behavior. Your task is to produce alert rule definitions that detect behavioral degradation, not just availability failures.

## INPUTS
- Frozen Prompt ID: [PROMPT_ID]
- Frozen Prompt Baseline Behavior Description: [BASELINE_BEHAVIOR]
- Monitored Metrics (with current healthy ranges): [MONITORED_METRICS]
- Alerting Platform Constraints (e.g., tool, query language, notification channels): [ALERTING_PLATFORM]
- Acceptable Degradation Thresholds (per metric): [DEGRADATION_THRESHOLDS]
- Existing Runbook References: [RUNBOOK_LINKS]
- Severity Classification Policy: [SEVERITY_POLICY]

## OUTPUT SCHEMA
Return a JSON array of alert rule objects. Each object must include:
- `alert_name`: string (unique, descriptive)
- `metric`: string (the metric being monitored)
- `threshold`: number or string (the trigger condition)
- `evaluation_window`: string (e.g., "5m", "1h")
- `severity`: "critical" | "warning" | "info"
- `runbook_link`: string (URL or reference)
- `description`: string (what this alert detects and why it matters)
- `false_positive_risk`: "low" | "medium" | "high"
- `recommended_action`: string (first response steps)

## CONSTRAINTS
- Do not generate alerts for metrics not listed in [MONITORED_METRICS].
- Every alert must reference a runbook from [RUNBOOK_LINKS] or note "runbook needed" if none exists.
- Severity must follow [SEVERITY_POLICY]. Do not inflate severity.
- If a metric has no healthy baseline, flag it with `"baseline_unknown": true` and set severity to "info".
- Include at least one alert for signal-to-noise ratio if multiple alerts are generated.
- Flag any metric that is not directly observable with `"observability_gap": true`.

## EXAMPLES
Input: Monitored metric "response_refusal_rate" with healthy range 0-2%, threshold 5%
Output alert: `{"alert_name": "High Refusal Rate on Frozen Prompt", "metric": "response_refusal_rate", "threshold": "> 5%", "evaluation_window": "10m", "severity": "warning", ...}`

Input: Monitored metric "output_schema_compliance" with healthy range 99.5-100%, threshold below 99%
Output alert: `{"alert_name": "Schema Compliance Drop", "metric": "output_schema_compliance", "threshold": "< 99%", "evaluation_window": "15m", "severity": "critical", ...}`

## RISK LEVEL: [RISK_LEVEL]
If RISK_LEVEL is "high", add a `human_review_required` boolean field to each alert and set it to true for all critical and warning alerts.

After pasting this prompt, replace every square-bracket placeholder with your actual values. The [BASELINE_BEHAVIOR] field is the most important—it grounds the alert logic in what the frozen prompt is supposed to do. If you skip it, the model will generate generic alerts that may not catch real degradation. The [MONITORED_METRICS] field should include only metrics you are already collecting or can instrument within your evaluation window. Do not list aspirational metrics; the model will generate alerts for them, and you will have dead alerts that never fire. Before deploying any generated alert, validate that the threshold values match your actual production data distributions, not just the healthy ranges you provided. Run each alert rule in silent mode for at least one full evaluation window before enabling notifications.

This prompt is not a one-off query; it is a configuration generator for your production monitoring stack. The output—alert rule definitions—must be parsed, validated, and loaded into an alerting system such as Prometheus, Datadog, or PagerDuty. The harness must treat the LLM output as untrusted configuration that requires structural validation before it touches any production routing. A common failure mode is generating syntactically valid YAML that references non-existent metrics or runbooks, so the harness must verify every referenced resource exists.

Implement a three-stage pipeline: generate, validate, activate. In the generate stage, call the model with the frozen prompt baseline, behavioral drift metrics, and current alerting context. In the validate stage, parse the output against a strict schema that checks required fields (alert_name, metric_query, threshold, severity, runbook_url), verifies metric names against your observability platform's API, and confirms runbook URLs return 200. Reject and retry if validation fails, feeding the specific error back to the model. In the activate stage, use a dry-run API call to test the alert rule before deployment, and require a human approval step for any severity: critical or page: true rule. Log every generated rule, validation result, and approval decision for audit.

Model choice matters here. Use a model with strong structured output support and low hallucination rates on technical configuration. Set temperature=0 and enforce JSON mode or function-calling output. Implement a retry budget of 3 attempts with exponential backoff. If all retries fail, escalate to an on-call engineer with the failed generations and validation errors attached. Never auto-deploy alert rules that fail validation, and never suppress validation failures silently—an invalid alert rule that creates a false sense of security is worse than no alert at all.