Safety Filter Activation Rate Monitoring Prompt Template

This prompt is designed for trust and safety engineers and AI operations teams who need to systematically evaluate safety filter activation rates from production trace data against defined operational thresholds. The primary job-to-be-done is transforming raw, aggregated trace spans—containing safety filter decisions, policy categories, and request metadata—into a structured analysis of activation trends, false-positive patterns, and policy boundary pressure points. The ideal user has already extracted relevant safety-filter spans from an observability platform like Datadog, Grafana, or a custom trace store, and has defined acceptable threshold ranges for each filter category (e.g., a 2-5% activation rate for a toxicity filter is normal, while a sustained 10% rate requires investigation).

Use this prompt when you need a batch analysis of a historical trace window—typically an hour, a day, or a week of production data—to inform threshold tuning, policy adjustments, or stakeholder reporting. The prompt expects structured input that includes filter activation counts, total request volumes, policy category labels, and false-positive flags if available. It is not designed for real-time per-request safety decisions; do not use it inside a request path where latency matters or where a single activation decision must be made. The analysis assumes your safety filters are already deployed and generating trace spans with consistent attribute schemas. If your trace data is missing required fields like policy_category or false_positive_marker, you must enrich the spans before feeding them to this prompt.

Before running this prompt, confirm that your threshold definitions are explicit and documented—vague thresholds like 'too many activations' will produce vague analysis. The prompt works best when you provide concrete numeric ranges per category, a defined evaluation window, and any known false-positive patterns from prior incident reviews. After receiving the output, validate the trend calculations against your own dashboard metrics before sharing with stakeholders or using the results to adjust filter sensitivity. The next step is typically to route identified pressure points to a policy review workflow or to feed the false-positive analysis into a filter tuning prompt.

The prompt below is designed to be dropped directly into your trace analysis pipeline. It expects production trace data containing safety filter events, your defined threshold configurations, and filter category taxonomies. The template is structured to produce a structured analysis of activation rates, false-positive patterns, and policy boundary pressure signals. Before using it, ensure you have extracted the relevant safety-filter spans from your observability platform and have your operational thresholds defined in a machine-readable format.

textCopy
You are a trust and safety observability analyst reviewing production trace data for safety filter activation patterns. Your task is to evaluate activation rates against defined operational thresholds, identify false-positive clusters, and detect policy boundary pressure.

## INPUT DATA

### Trace Spans with Safety Filter Events
[SAFETY_FILTER_TRACE_SPANS]

### Filter Configuration and Category Taxonomy
[FILTER_CONFIGURATION]

### Operational Threshold Definitions
[THRESHOLD_DEFINITIONS]

### Historical Baseline Activation Rates (Optional)
[HISTORICAL_BASELINE]

## OUTPUT SCHEMA

Return a JSON object with the following structure:

{
  "analysis_timestamp": "ISO 8601 timestamp of analysis",
  "evaluation_window": {
    "start": "ISO 8601 start time",
    "end": "ISO 8601 end time",
    "total_requests_evaluated": 0
  },
  "activation_summary": {
    "overall_activation_rate": 0.0,
    "total_activations": 0,
    "rate_change_vs_baseline_pct": null
  },
  "category_breakdown": [
    {
      "category_name": "string",
      "activation_count": 0,
      "activation_rate": 0.0,
      "threshold_status": "within_threshold | approaching_threshold | exceeded_threshold",
      "threshold_value": 0.0,
      "false_positive_estimated_count": 0,
      "false_positive_indicators": ["string"]
    }
  ],
  "threshold_violations": [
    {
      "category_name": "string",
      "threshold_breached": "string",
      "observed_value": 0.0,
      "threshold_value": 0.0,
      "severity": "warning | critical",
      "recommended_action": "string"
    }
  ],
  "false_positive_analysis": {
    "total_estimated_false_positives": 0,
    "false_positive_rate": 0.0,
    "top_false_positive_categories": ["string"],
    "pattern_observations": ["string"]
  },
  "policy_boundary_pressure": {
    "pressure_detected": true,
    "affected_categories": ["string"],
    "pressure_indicators": ["string"],
    "boundary_adjustment_recommendations": ["string"]
  },
  "alert_recommendations": [
    {
      "alert_type": "string",
      "category": "string",
      "condition": "string",
      "severity": "warning | critical",
      "rationale": "string"
    }
  ]
}

## CONSTRAINTS

- Only analyze spans where the safety filter was evaluated, not every request in the trace.
- Distinguish between true safety activations and likely false positives using context from the user input and model response captured in the trace.
- When a threshold is breached, classify severity as "warning" if the breach is less than 20% above threshold, and "critical" if 20% or more above threshold.
- If historical baseline data is provided, calculate rate changes against that baseline. If not provided, set rate_change_vs_baseline_pct to null.
- For false-positive estimation, look for activations where the user input appears benign in context and the model response was blocked unnecessarily.
- Policy boundary pressure is indicated by activation rates clustering near threshold boundaries or increasing trend velocity.
- Do not fabricate activation events. Only report on data present in the provided trace spans.

## RISK_LEVEL
HIGH — Safety filter analysis directly impacts user safety and content policy enforcement. All threshold violations and false-positive classifications must be reviewed by a human trust and safety analyst before triggering automated policy changes or filter adjustments.

To adapt this template, replace each square-bracket placeholder with your production data. The [SAFETY_FILTER_TRACE_SPANS] should contain the raw span objects from your observability platform, including the user input, model response, filter decision, and filter category. The [FILTER_CONFIGURATION] should include your current filter taxonomy and any category-specific rules. The [THRESHOLD_DEFINITIONS] must specify the acceptable activation rate for each category and the overall system. If you have historical data, include it in [HISTORICAL_BASELINE] to enable trend comparison. After running the prompt, validate the output JSON against the schema before ingesting it into your alerting pipeline. Any threshold violation classified as "critical" should trigger an immediate human review workflow, not an automated response.

Before deploying this prompt into a production monitoring pipeline, test it against a set of known trace spans where you have manually verified the expected activation classifications and threshold statuses. Run at least 20 traces with a mix of true positives, false positives, and no-activation cases through the prompt and compare the output against your manual labels. If the false-positive estimation deviates by more than 15% from your manual review, adjust the prompt's false-positive identification criteria or add few-shot examples of your specific filter categories. After validation, wire the prompt into a scheduled observability job that runs at your monitoring interval, logs the full JSON output for auditability, and routes threshold violations to your incident management system with the severity classification intact.

This prompt is designed to operate as a scheduled analysis step within your observability pipeline, not as a one-off chat interaction. It expects structured trace data from your monitoring backend (e.g., Datadog, Grafana, or a custom trace store) and produces a machine-readable analysis that should be validated before triggering alerts. The primary integration point is a cron-driven function that queries recent safety filter activation events, formats them into the prompt's [TRACE_DATA] input, calls the LLM, and routes the validated output to your alerting platform and dashboard.

Pipeline wiring: 1) A query layer fetches safety filter activation events for the evaluation window from your trace store, including filter_category, timestamp, session_id, and input_snippet fields. 2) The data is serialized into the [TRACE_DATA] JSON array and combined with [THRESHOLD_CONFIG] (defining acceptable activation rates per category) and [EVALUATION_WINDOW] parameters. 3) The assembled prompt is sent to a model with strong structured output capabilities (GPT-4o or Claude 3.5 Sonnet recommended). 4) The response is parsed and validated against the expected output schema before any alert is fired. Validation checks must include: schema conformity (all required fields present, enums match allowed values), mathematical consistency (activation counts sum correctly, rates are within 0-1), and threshold comparison accuracy (severity correctly derived from rate vs. threshold). If validation fails, retry once with the error message injected into the prompt's [CONSTRAINTS] section before escalating to a human reviewer.

Failure modes to instrument: The most common production failures are stale threshold configurations (a category's threshold drifts without updating the prompt input), trace data gaps (the query returns empty or partial data, causing false-negative "all clear" signals), and LLM hallucination of activation counts that don't match the input data. Mitigate these by logging the input trace count alongside the LLM's reported counts and flagging discrepancies. Route confirmed threshold breaches to your incident management system with the full analysis payload, and route false-positive analyses (where the LLM flags a breach but validation shows it's within threshold) to a review queue for prompt tuning. Never allow this prompt's output to directly mute or disable a safety filter without human approval.