Safety Filter False Positive Trace Diagnosis Prompt

This prompt is for trust and safety engineers investigating a user-reported false positive refusal. The model refused a request that the user believes was legitimate. Your job is to compare the user's stated intent against the safety filter's activation signal captured in the production trace to determine whether the refusal was unwarranted. Use this prompt when you have access to the full trace data including the user input, the safety filter's activation reason, the refusal message, and any preceding conversation context. This prompt produces a structured misclassification verdict, the evidence supporting that verdict, and a recommended remediation action.

Do not use this prompt for real-time blocking decisions or for investigating false negatives where the model failed to refuse harmful content. It is designed for post-hoc, human-in-the-loop diagnosis of a single trace. The prompt requires a complete trace payload: the raw user input, the exact refusal message returned, the safety filter's activation reason and confidence score, and the conversation history leading up to the refusal. Without this full context, the model cannot reliably distinguish between a legitimate safety activation and an over-refusal triggered by a keyword collision or ambiguous phrasing. The output is a structured JSON verdict that includes a misclassification boolean, a confidence score, an evidence array citing specific trace fields, and a remediation recommendation such as adjusting a keyword list, refining a system prompt boundary, or adding a clarifying user prompt.

Before running this prompt, confirm that the trace is complete and that the user has provided a clear statement of their original intent. If the trace is missing the safety filter's activation reason or the conversation history is truncated, the diagnosis will be unreliable. After receiving the output, always apply human review to the verdict before taking action—especially if the recommended remediation involves modifying a safety policy or system prompt. Use the structured output to log the decision in your incident tracking system and to build a dataset of false positive patterns that can inform future filter tuning.

This prompt is designed to be pasted directly into your investigation workflow. It instructs the model to act as a senior trust and safety engineer, comparing the user's original intent against the safety filter's activation signal captured in a production trace. The goal is to produce a structured misclassification verdict, the evidence supporting that verdict, and a recommended remediation action. Before using this prompt, ensure you have extracted the relevant trace data, including the user's full input, the system prompt or safety instructions active at the time, and the exact refusal message or filter tag that was triggered.

textCopy
You are a senior trust and safety engineer reviewing a production trace for a false positive safety refusal. Your task is to determine if the model's refusal was unwarranted given the user's intent.

## INPUT DATA

### User Request (Full Text)
[USER_REQUEST]

### Active Safety Instructions / System Prompt
[SYSTEM_PROMPT]

### Refusal Output or Filter Tag
[REFUSAL_OUTPUT]

### Additional Trace Context (Tool Calls, Prior Turns)
[TRACE_CONTEXT]

## INSTRUCTIONS

1.  **Intent Analysis:** Summarize the user's most likely intent in one sentence. Identify any ambiguous terms that could be misinterpreted.
2.  **Policy Mapping:** Identify the specific clause or instruction within the [SYSTEM_PROMPT] that most likely triggered the refusal.
3.  **Misclassification Verdict:** Classify the event as `TRUE_POSITIVE`, `FALSE_POSITIVE`, or `UNCLEAR`. A false positive means the user's intent was benign and the safety filter over-reacted to a keyword or out-of-context phrase.
4.  **Evidence:** Quote the exact text from the [USER_REQUEST] and [SYSTEM_PROMPT] that supports your verdict.
5.  **Remediation:** If the verdict is `FALSE_POSITIVE`, recommend a specific, actionable change to the [SYSTEM_PROMPT] or a keyword filter rule to prevent this class of error. If `TRUE_POSITIVE`, explain why the refusal was correct. If `UNCLEAR`, state what additional context is needed.

## CONSTRAINTS
- Do not fabricate user intent. Base your analysis strictly on the provided trace.
- If the trace data is insufficient for a high-confidence verdict, you MUST classify the event as `UNCLEAR`.
- Your remediation suggestion must be a concrete text edit or rule change, not a general principle.

## OUTPUT SCHEMA
{
  "user_intent_summary": "string",
  "ambiguous_terms": ["string"],
  "triggering_policy_clause": "string",
  "verdict": "TRUE_POSITIVE" | "FALSE_POSITIVE" | "UNCLEAR",
  "evidence": {
    "user_request_quote": "string",
    "policy_quote": "string"
  },
  "remediation_action": "string",
  "confidence_score": 0.0-1.0
}

To adapt this template, replace the square-bracket placeholders with data from your observability platform. The [TRACE_CONTEXT] field is critical for multi-turn conversations where the refusal may be triggered by an earlier exchange. If your safety system uses a numeric filter tag instead of a text refusal, place that identifier in [REFUSAL_OUTPUT]. For high-risk domains, always route UNCLEAR verdicts and any FALSE_POSITIVE with a confidence score below 0.9 to a human review queue before implementing the suggested remediation. The structured JSON output is designed to be logged directly into your incident tracking system for trend analysis.

This prompt is designed to be the analytical core of a false positive review pipeline, not a standalone chat interaction. The input trace must be pre-assembled from your observability platform (e.g., LangSmith, Arize, Datadog LLM, or custom logging) before the prompt is called. The prompt expects a structured [TRACE_JSON] input containing the user's original request, the safety filter's activation signal, the refusal message, and any relevant system prompt or policy metadata. Do not pass raw, unparsed log lines; the model needs the trace fields explicitly labeled to produce a reliable verdict.

Wire the prompt into an automated workflow with a pre-processing step that fetches the full trace by trace_id and a post-processing step that validates the JSON output against a strict schema. The expected output includes a verdict enum (FALSE_POSITIVE, LEGITIMATE_REFUSAL, UNCLEAR), an evidence array, and a remediation string. Implement a JSON schema validator in your application layer that rejects malformed responses and triggers a single retry with the validation error message appended to the prompt. If the retry also fails or the verdict is UNCLEAR, escalate the trace to a human review queue with the raw trace and the model's partial analysis attached. Log every verdict, the model version used, and the reviewer's final decision to build a ground-truth dataset for future eval.

For model choice, use a capable reasoning model (e.g., GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro) with a low temperature setting (0.0–0.2) to maximize consistency in verdict classification. Avoid using smaller or faster models for this task, as the comparative reasoning required to distinguish user intent from filter activation signals demands strong instruction-following and nuance. If you are processing a high volume of user-reported false positives, batch the traces and run the prompt asynchronously, but always preserve the ability to trace each verdict back to the exact prompt version and model snapshot. Never auto-apply the model's recommended remediation (e.g., updating a safety keyword list) without human approval; the prompt's remediation field is a suggestion, not an automated action.