API Response Safety Gate Prompt

This prompt is a purpose-built safety net for API platform teams who need a reliable, automated gate to inspect every AI-generated response before it reaches a client. The job-to-be-done is a single-pass, binary safe/block decision that enforces content safety, PII redaction, and structural validity against a configurable policy. The ideal user is an engineering lead or backend developer integrating this gate into an existing API response path where latency budgets are tight and a false positive means a broken user experience, not just a flagged message.

Use this prompt when you have a generated response payload and need to enforce multiple policies in one call: checking for disallowed content categories, detecting and redacting personally identifiable information, and validating that the output conforms to the expected schema. It is designed for the final pre-delivery step in a stateless API pipeline. Do not use this prompt as a general-purpose moderation tool for user-generated content, a real-time chat filter where conversational context is required, or a replacement for a dedicated PII redaction service that must handle unstructured free-text with high recall. It is not a legal compliance review and does not replace human judgment for high-severity policy decisions.

Before wiring this prompt into your application, define your policy configuration explicitly. You must provide a clear list of blocked content categories, the expected output schema for validation, and the PII entity types to detect. The prompt works best when the input is a single, complete AI-generated response. If your system streams responses, buffer the full output before calling the gate. The next step is to copy the prompt template, adapt the placeholders to your policy, and integrate it with a validation harness that logs every decision, measures latency, and provides a fallback response when the gate blocks delivery.

This template implements a multi-policy safety gate that inspects an AI-generated API response and returns a binary safe/block decision with detailed violation evidence. Before using it, you must configure the specific policies, output schema, and latency budget for your API surface. The prompt is designed to be called as a synchronous pre-delivery check, where a block decision triggers a fallback response instead of the original output.

textCopy
You are an API Response Safety Gate. Your job is to inspect an AI-generated API response and determine whether it is safe to deliver to the client. You must check the response against all configured policies and return a structured verdict.

## POLICIES TO ENFORCE
[POLICIES]

## RESPONSE TO INSPECT
[RESPONSE]

## REQUEST CONTEXT (if available)
[REQUEST_CONTEXT]

## OUTPUT SCHEMA
Return a JSON object with exactly these fields:
{
  "verdict": "safe" | "block",
  "violations": [
    {
      "policy_id": "string",
      "policy_name": "string",
      "severity": "critical" | "high" | "medium" | "low",
      "location": "string (path or excerpt where violation found)",
      "evidence": "string (the violating content)",
      "reason": "string (why this violates the policy)"
    }
  ],
  "pii_detected": [
    {
      "category": "email" | "phone" | "ssn" | "credit_card" | "address" | "name" | "other",
      "value_preview": "string (masked, show first 2 chars only)",
      "location": "string"
    }
  ],
  "format_valid": true | false,
  "format_errors": ["string (description of each format violation)"],
  "block_reason_summary": "string (one-sentence summary if blocked, null if safe)",
  "confidence": 0.0-1.0
}

## CONSTRAINTS
[CONSTRAINTS]

## INSTRUCTIONS
1. Check the response against every policy in [POLICIES] in order of priority.
2. If any critical or high-severity violation is found, verdict must be "block".
3. Scan for PII using the categories listed. Mask previews to first 2 characters only.
4. Validate format compliance against [OUTPUT_SCHEMA] if provided.
5. If verdict is "block", populate block_reason_summary with a clear, actionable explanation.
6. Set confidence based on how certain you are about each violation. Lower confidence when policies are ambiguous.
7. Do not modify the response. Only inspect and report.
8. If no violations are found, violations and pii_detected must be empty arrays, and verdict must be "safe".

Adapt this template by replacing [POLICIES] with your specific content safety rules, data handling requirements, and business logic constraints. Each policy should include an id, name, description, severity, and detection criteria. Replace [CONSTRAINTS] with hard limits such as maximum response length, required fields, or disallowed content categories. The [REQUEST_CONTEXT] placeholder should receive the original user request that generated the response, which helps detect prompt leakage and context-inappropriate outputs. Wire the output into your API gateway so that a "block" verdict returns a pre-approved fallback response and logs the full violation details for review.

The API Response Safety Gate prompt is not a standalone safety check—it is one component of a multi-policy evaluation harness that sits between your model's raw output and the client response. In production, you will typically deploy this prompt inside an API proxy or middleware layer that intercepts every AI-generated response, runs it through a sequence of safety gates, and either forwards the response, blocks it, or replaces it with a fallback. The prompt itself produces a structured binary verdict with policy violation details, but the harness is responsible for ordering the checks, managing latency, handling partial failures, and deciding what the client actually receives.

Harness architecture. Wire the prompt into a response-processing pipeline with these stages: (1) Response capture—intercept the model output before it reaches the client. (2) Policy evaluation ordering—run cheaper, faster checks first (format validity, schema compliance) before more expensive semantic checks (PII detection, content policy violations). If a response fails an early check, skip downstream checks to conserve latency budget. (3) Prompt invocation—call the safety gate prompt with the raw response and your policy definitions. Use response_format or structured output mode to enforce the binary verdict schema. (4) Validation layer—validate that the gate's output contains the required fields (safe, violations, blocked_reason), that safe is a boolean, and that violations is an array. If the gate itself produces malformed output, treat it as a fail-closed event: block the response and log the gate failure. (5) Decision enforcement—if safe is true, forward the original response. If safe is false, execute the fallback path.

Latency budget integration. Safety gates add latency to every response. Set a hard timeout for the entire gate evaluation pipeline (recommended starting point: 500ms for synchronous checks, 2s if including LLM-based semantic checks). If the gate times out, fail closed. For high-throughput APIs, consider running format checks synchronously and semantic policy checks asynchronously with a short grace window—if the async check returns a block verdict after the response was already sent, flag the interaction for review and trigger any required notifications. Never skip the gate to meet latency targets; instead, optimize by caching policy definitions in the system prompt, using smaller/faster models for the gate itself, or pre-computing PII scans with regex before invoking the LLM gate.

Fallback response generation. When the gate blocks a response, the harness must return something to the client. Prepare a fallback response template that is policy-compliant by construction: 'I'm unable to provide that response. [REASON_CODE].' Map each blocked_reason from your policy definitions to a user-facing reason code. For PII blocks, never echo the detected PII back to the client. For content policy blocks, avoid revealing the specific policy threshold that was triggered if that information could be exploited. Log the full gate output internally for audit and improvement, but keep the client-facing fallback generic and safe.

Retry and self-correction considerations. Do not naively retry the safety gate on failure—a gate that produces malformed output or times out may indicate a systemic issue (model degradation, prompt drift, context overflow). Implement circuit breaking: if the gate failure rate exceeds a threshold (e.g., 5% over a rolling 5-minute window), stop forwarding responses and alert the on-call engineer. For responses that fail with format or schema violations, you may attempt a single repair pass using a separate output-repair prompt before re-running the gate, but cap the total repair attempts at one to avoid infinite loops. Human review integration is required for high-risk domains: if the gate blocks a response in a regulated context (healthcare, finance, legal), queue the interaction for human review rather than silently dropping it. The harness should emit structured logs with the original response, gate verdict, timestamp, and session ID for audit trails.