Guardrail Violation Trend Analysis Prompt for Trust and Safety

This prompt is designed for safety program managers and trust and safety engineers who need to convert raw guardrail violation logs into actionable trend intelligence. Use it when you have a batch of violation records spanning a defined time window and you need a structured summary that identifies distribution shifts, emerging attack patterns, frequency anomalies, and policy coverage gaps. The primary job-to-be-done is strategic oversight: understanding whether your guardrail surface is improving or degrading over time, and where to allocate engineering or policy resources next.

This prompt assumes violation records are already classified and enriched with severity, policy reference, and timestamp data. It does not perform real-time detection or single-violation classification. For those workflows, use the Guardrail Violation Detection and Classification Prompt Template or the Severity Scoring Rubric Prompt for Escalation Triage instead. The trend analysis prompt works best with batch inputs of 50 or more violation records across a window of at least 7 days; smaller datasets will produce low-confidence trend signals. Before invoking this prompt, ensure your input data includes consistent violation_type, severity, policy_id, and timestamp fields. Missing or inconsistent enrichment will degrade the analysis quality and may produce misleading trend lines.

Do not use this prompt for real-time alerting, individual case adjudication, or as a substitute for a metrics dashboard. Its output is a narrative summary with supporting statistics, not a live monitoring feed. After receiving the trend analysis, validate the statistical claims against your source data, review the policy gap recommendations with legal or compliance stakeholders before acting on them, and log the output as part of your safety program review cadence. If the analysis flags a novel attack pattern, escalate the finding to your red-team or security engineering team for immediate investigation rather than waiting for the next review cycle.

This prompt is designed for safety program managers and trust and safety engineers who need to move from individual violation tickets to aggregate patterns. It expects a structured dataset of violation records as input and produces a trend summary that includes distribution analysis, frequency changes, emerging attack patterns, and policy gap recommendations. The prompt is self-contained: all analysis instructions, the output schema, and the required constraints are embedded so that the model can operate on the provided data without external retrieval.

textCopy
You are a trust and safety trend analyst. Your task is to analyze a batch of guardrail violation records and produce a structured trend report.

[INPUT]
A JSON array of violation records. Each record contains:
- violation_id: string
- timestamp: ISO 8601 datetime
- violation_type: string (enum: [POLICY_VIOLATION_TYPES])
- severity: string (enum: [SEVERITY_LEVELS])
- policy_reference: string
- evidence_excerpt: string (max 500 chars)
- adjudication_outcome: string (enum: [ADJUDICATION_OUTCOMES])
- reviewer_notes: string (optional)

[CONSTRAINTS]
1. Only analyze data present in [INPUT]. Do not invent or assume violations not in the dataset.
2. If the dataset contains fewer than 10 records, note low confidence in trend claims and avoid overgeneralizing.
3. For any emerging attack pattern claim, cite at least two specific violation_ids as evidence.
4. Mark any recommendation that would require policy changes with the tag [POLICY_CHANGE].
5. If violation_type distribution is uniform (no type exceeds 25%), explicitly state that no dominant pattern exists.
6. Do not include personally identifiable information from evidence_excerpt fields in the output.

[OUTPUT_SCHEMA]
Return a single JSON object with this exact structure:
{
  "analysis_period": {
    "start": "ISO 8601",
    "end": "ISO 8601",
    "total_violations": integer,
    "data_completeness": "HIGH | MEDIUM | LOW"
  },
  "violation_type_distribution": [
    {
      "violation_type": "string",
      "count": integer,
      "percentage": float,
      "trend_direction": "INCREASING | DECREASING | STABLE | INSUFFICIENT_DATA"
    }
  ],
  "severity_distribution": [
    {
      "severity": "string",
      "count": integer,
      "percentage": float
    }
  ],
  "frequency_analysis": {
    "weekly_trend": "INCREASING | DECREASING | STABLE | INSUFFICIENT_DATA",
    "notable_spikes": [
      {
        "date_range": "string",
        "violation_type": "string",
        "spike_factor": float,
        "hypothesized_trigger": "string | null"
      }
    ]
  },
  "emerging_attack_patterns": [
    {
      "pattern_name": "string",
      "description": "string",
      "evidence_violation_ids": ["string", "string"],
      "affected_policies": ["string"],
      "confidence": "HIGH | MEDIUM | LOW"
    }
  ],
  "policy_gap_recommendations": [
    {
      "recommendation": "string",
      "rationale": "string",
      "requires_policy_change": boolean,
      "priority": "CRITICAL | HIGH | MEDIUM | LOW"
    }
  ],
  "limitations": ["string"]
}

[EXAMPLES]
Input excerpt:
[
  {"violation_id": "v-001", "timestamp": "2025-01-15T10:00:00Z", "violation_type": "JAILBREAK_ATTEMPT", "severity": "HIGH", "policy_reference": "POL-004", "evidence_excerpt": "User prompt contained DAN-style roleplay override...", "adjudication_outcome": "CONFIRMED_VIOLATION"},
  {"violation_id": "v-002", "timestamp": "2025-01-15T11:00:00Z", "violation_type": "JAILBREAK_ATTEMPT", "severity": "CRITICAL", "policy_reference": "POL-004", "evidence_excerpt": "Multi-turn injection with encoding bypass...", "adjudication_outcome": "CONFIRMED_VIOLATION"}
]

Expected pattern detection: The analyst should identify a JAILBREAK_ATTEMPT cluster, note the encoding bypass technique as an emerging pattern, and cite both v-001 and v-002 as evidence.

[RISK_LEVEL]
HIGH. This analysis informs safety policy decisions and may trigger enforcement actions. Outputs must be reviewed by a human safety program manager before policy changes are enacted. Inaccurate trend claims could lead to misallocated safety resources or missed attack patterns.

To adapt this prompt, replace the placeholder enums with your organization's actual taxonomies. [POLICY_VIOLATION_TYPES] should be your defined violation categories, [SEVERITY_LEVELS] your severity scale, and [ADJUDICATION_OUTCOMES] your review decision states. If your violation records include additional fields such as model version, user segment, or geographic region, add them to the input schema and consider whether the output schema needs corresponding breakdown dimensions. Before running this prompt in production, validate that the model reliably produces the exact JSON structure by running it against a golden dataset of 20-50 labeled violation records where you already know the expected trend conclusions. Any output that fails JSON schema validation should trigger a retry with the same prompt; if retries exceed three attempts, escalate the raw violation data to a human analyst with a note that automated trend analysis failed.

This prompt is not a dashboard replacement; it is a structured analytical step inside a periodic safety review pipeline. Wire it as a scheduled job (daily or weekly) that consumes a batch of adjudicated violation records from your guardrail detection system, not raw model outputs. The prompt expects a clean, structured [VIOLATION_LOG] input—typically a JSON array of closed cases with fields like violation_id, policy_id, severity, detection_timestamp, adjudication_outcome, and violation_category. Feed it only cases that have completed human review to avoid analyzing noisy, unconfirmed detections. The output [TREND_REPORT] should conform to a strict JSON schema so downstream systems can ingest it without parsing free text.

Before calling the LLM, validate the input batch: check that the date range is bounded, that required fields are present and non-null, and that the batch size stays within the model's effective context window (typically 200–500 records for a detailed trend analysis). Implement a retry wrapper around the model call that catches malformed JSON, missing required output fields, or schema validation failures. On failure, re-prompt once with the validation error message appended to [CONSTRAINTS]. If the second attempt also fails, log the raw output and escalate to a human analyst with a trend_analysis_failure alert. Do not silently serve a partial or hallucinated trend report to downstream consumers.

The analysis output must pass an eval gate before publication. Run a set of deterministic checks against the [TREND_REPORT]: verify that the reported total violation count matches the input batch size, that category percentages sum to approximately 100%, and that any claimed 'emerging attack pattern' references at least two concrete violation IDs from the input. For higher assurance, maintain a small golden dataset of labeled violation batches with known trends and run the prompt against it weekly as a regression test. If the model misses a known spike or fabricates a non-existent pattern, investigate prompt drift or model version changes before the next production run.

Human review is mandatory for the final artifact. The generated trend report should land in a review queue, not auto-publish to stakeholders. A trust and safety program manager must confirm that the identified patterns, severity distributions, and policy gap recommendations align with their operational knowledge before the report is shared with engineering or compliance teams. The review UI should display the input batch summary, the generated report, and the eval gate results side by side. Only after explicit approval should the report be persisted to the audit trail and distributed.

Choose a model with strong structured output and reasoning capabilities. GPT-4o or Claude 3.5 Sonnet work well for this task. Avoid smaller, faster models that may struggle with multi-dimensional trend synthesis across dozens of categories. If cost is a concern, experiment with routing simpler summary statistics to a lightweight model and reserving the full trend analysis prompt for the primary model, but always validate that the lightweight path does not silently drop anomaly detection. Log every run's input hash, output, eval results, reviewer identity, and approval timestamp for audit readiness.