Jailbreak Detection

Identifies prompts where the semantic intent diverges from the surface-level syntax. Jailbreak prompts often use encoded language, hypothetical scenarios, or role-playing frameworks to mask malicious intent.

• Compares prompt embeddings against known safe distributions
• Flags out-of-distribution linguistic patterns
• Detects obfuscated requests hidden within benign-looking text

Example: A prompt that asks the model to 'write a fictional story' but embeds instructions for generating malware within the narrative structure.

Measures the statistical likelihood of a prompt under a reference language model. Adversarial prompts often exhibit abnormally high or low perplexity scores due to token manipulation or nonsensical concatenation.

• High perplexity: gibberish strings designed to confuse the model
• Low perplexity: overly generic templates used in automated attacks
• Real-time scoring against a calibrated threshold

This technique catches token smuggling attacks where malicious payloads are split across seemingly random character sequences.

Implements a strict privilege model where system-level instructions cannot be overridden by user-level inputs. The model is trained to recognize and reject prompts that attempt to re-prioritize or contradict its core directives.

• System prompt is placed at the highest privilege level
• User inputs are parsed for override attempts
• Conflicts trigger immediate refusal responses

Defends against 'ignore all previous instructions' and 'you are now DAN' style attacks that attempt to demote the system prompt's authority.

Scans non-text inputs for embedded adversarial instructions. Attackers increasingly use images, audio files, or formatted documents to smuggle jailbreak prompts past text-only filters.

• OCR analysis of uploaded images for hidden text
• Metadata extraction from file uploads
• Steganography detection for concealed payloads

Example: An image containing white text on a white background that instructs the model to bypass safety protocols when processed by a vision-language model.

Leverages the model's own reasoning capabilities to evaluate user prompts for deceptive intent before processing. A lightweight classifier or the model itself performs a preliminary safety assessment.

• Prompt is analyzed for coercion patterns
• Model generates an internal safety score
• Ambiguous prompts trigger clarification requests

This approach catches sophisticated social engineering attacks that exploit the model's helpfulness to gradually erode safety boundaries through multi-turn conversations.

Embeds unique, decoy strings within the system prompt that act as digital tripwires. If these canary tokens appear in model outputs, it signals a successful prompt extraction or system prompt leak.

• Unique UUIDs embedded in hidden system instructions
• Output monitoring for canary token appearance
• Triggers immediate session termination and alerting

Provides forensic evidence of prompt injection success and enables real-time response to active jailbreak attempts before further damage occurs.

A dedicated model or scoring layer that evaluates inputs and outputs against a toxicity and policy threshold. These classifiers assign a continuous risk score across multiple harm categories:

• Violence and hate speech detection
• Sexual content filtering
• Self-harm ideation flagging
• Illegal activity identification

When a score exceeds the configured threshold, the system triggers a refusal response or routes the interaction for human review. Often implemented as a fine-tuned DeBERTa-v3 or similar compact model for low-latency inference.

A safety technique that directly manipulates the internal activations of a neural network to control high-level behaviors without relying on prompt instructions. Rather than filtering inputs or outputs, this method identifies and modifies safety vectors in the model's latent space.

• Extracts activation patterns associated with harmful outputs
• Computes a 'refusal direction' via linear probing
• Adds the safety vector during the forward pass
• Induces reliable refusal without degrading general capability

Pioneered by research showing that concepts like honesty and harmlessness have linearly separable representations in transformer architectures.

An automated operational safeguard that immediately halts inference or revokes API access when a critical threshold of policy violations is detected within a defined time window. Functions as a rate-limiting safety mechanism:

• Monitors violation density per user session
• Triggers progressive enforcement (warn → throttle → block)
• Prevents automated adversarial probing at scale
• Logs all interventions for forensic audit

Essential for production deployments where attackers may attempt high-volume jailbreak sweeps to find bypass techniques.

A defense-in-depth architecture that combines multiple heterogeneous safety classifiers via voting or cascading logic to minimize both false negatives and false positives. No single classifier catches every attack vector:

• Regex filters catch known patterns and keyword lists
• Semantic embedding models detect conceptual similarity to harmful content
• Neural classifiers identify subtle adversarial constructions
• Heuristic rules flag unusual prompt structures

Outputs are aggregated through weighted voting or a cascading pipeline where each layer escalates uncertain cases to more sophisticated analysis.