This prompt is designed for product engineers and safety teams who need to build a capability boundary at the ingress layer of a multimodal AI system. Its job is to inspect a user input that may contain text, an image, an audio file, a video clip, or a document, and determine whether that input falls inside or outside the system's supported domains, modalities, and safety envelope. Use this prompt when you need a deterministic, auditable rejection path that avoids hallucinated responses, wasted compute, or policy violations on unsupported requests. It is not a general-purpose content moderator, a spam filter, or a replacement for a full safety policy engine. It is a boundary guard that says 'this system does not handle that' and, when possible, suggests an alternative pipeline or clarifies the limitation.
Prompt
Multi-Modal Out-of-Scope Detection System Prompt

When to Use This Prompt
Define the job-to-be-done, ideal user, and operational boundaries for deploying a multi-modal out-of-scope detection prompt.
The ideal user is an AI infrastructure engineer or platform safety lead integrating this prompt as a pre-processing step before any model invocation, tool dispatch, or data persistence. The required context includes a clearly defined system capability document, a list of supported modalities and file types, and a safety policy that specifies prohibited content categories. You should not use this prompt when you need nuanced content moderation that considers context, satire, or educational use, as its binary in-scope/out-of-scope logic will over-block edge cases. Similarly, do not use it as a standalone defense against adversarial attacks; it should be one layer in a defense-in-depth strategy that includes dedicated prompt injection detectors and human review for high-risk domains.
Before deploying, you must define your boundary taxonomy explicitly. This includes enumerating supported modalities (e.g., text, image, audio, video, document), acceptable file types (e.g., image/png, application/pdf), supported languages, and out-of-scope content categories (e.g., CSAM, violence, regulated_financial_advice). The prompt should be tested against a golden dataset of both in-scope and out-of-scope examples, including adversarial inputs designed to look in-scope, such as a screenshot of a supported document type that contains disallowed content. Implement eval checks that measure precision and recall on the rejection decision, with a particular focus on minimizing false negatives where an out-of-scope input is incorrectly routed to a downstream handler. Wire the prompt into your application with a strict validator on the output schema, and log every rejection with the input hash, detected modalities, and the specific boundary rule that triggered the rejection to enable audit and prompt iteration.
Use Case Fit
Where this prompt works and where it does not. Use this to decide if the Multi-Modal Out-of-Scope Detection System Prompt is the right tool for your routing architecture.
Good Fit: Safety Envelope Enforcement
Use when: you need a single, deterministic gate that blocks unsupported modalities, languages, or content categories before any downstream model invocation or tool execution. Guardrail: define explicit boundary definitions in the prompt's [SUPPORTED_DOMAINS] and [SAFETY_POLICIES] variables, and test with adversarial inputs that try to appear in-scope.
Bad Fit: Fine-Grained Intent Classification
Avoid when: you need to distinguish between 20+ supported intents or route to specific agent pipelines. This prompt is a binary or categorical boundary check, not a full taxonomy classifier. Guardrail: pair this prompt upstream of a dedicated intent classification prompt. Use this for rejection, not routing within the supported domain.
Required Inputs
What you must provide: a multimodal input payload (text, image, audio, or document), a defined [SUPPORTED_DOMAINS] list, [UNSUPPORTED_LANGUAGES] if applicable, and a [SAFETY_POLICIES] specification. Guardrail: if your supported domain list is empty or undefined, the prompt will either reject everything or accept everything—both are failures. Validate the variable is populated at assembly time.
Operational Risk: Over-Rejection Under Load
What to watch: when latency budgets tighten or the model is under provisioned, out-of-scope detection can become overly conservative and reject valid inputs to reduce processing cost. Guardrail: monitor rejection rates by modality and compare against a golden set of known in-scope inputs. Set an alert if the false rejection rate exceeds your threshold.
Operational Risk: Adversarial Evasion
What to watch: attackers may embed disallowed content in image metadata, OCR-extracted text, or audio transcripts that the primary modality check misses. Guardrail: run this prompt with all modalities extracted and presented together, not sequentially. Include explicit instructions to cross-check text extracted from attachments against the safety policy, and log any input that passes the boundary check but triggers a downstream policy violation.
Variant: Graceful Degradation with Alternatives
Use when: you want to suggest an alternative pipeline instead of a hard rejection (e.g., 'I can't process video, but I can analyze a transcript if you provide one'). Guardrail: include a [SUGGESTED_ALTERNATIVES] section in the output schema and test that the model does not hallucinate capabilities you don't have. The alternative must reference a real, available pipeline.
Copy-Ready Prompt Template
A reusable system prompt for detecting multimodal inputs that fall outside your system's supported domains, modalities, or safety envelope.
This prompt template defines a capability boundary guard for multimodal AI systems. It instructs the model to analyze an input—which may include text, images, audio transcripts, or document attachments—and determine whether it falls within the system's supported scope. When an input is out-of-scope, the prompt requires a structured rejection with a reason code and, where possible, a suggested alternative pipeline. Replace each square-bracket placeholder with your system's specific boundary definitions before deployment.
textYou are a multimodal input boundary classifier. Your job is to determine whether a given input falls within the system's supported scope. You must analyze all provided modalities—text, image, audio transcript, video description, or document attachment—and produce a structured classification. ## Supported Scope - Modalities: [SUPPORTED_MODALITIES] - Languages: [SUPPORTED_LANGUAGES] - File Types: [SUPPORTED_FILE_TYPES] - Content Categories: [SUPPORTED_CONTENT_CATEGORIES] - Safety Envelope: [SAFETY_POLICY_BOUNDARIES] ## Out-of-Scope Triggers Classify an input as out-of-scope if ANY of the following are true: - The input contains an unsupported modality: [UNSUPPORTED_MODALITY_EXAMPLES] - The primary language is not in the supported list - The file type or format is unsupported: [UNSUPPORTED_FILE_TYPE_EXAMPLES] - The content category is explicitly disallowed: [DISALLOWED_CONTENT_CATEGORIES] - The input violates the safety envelope: [SAFETY_VIOLATION_EXAMPLES] - The input is ambiguous, corrupted, or unreadable in all provided modalities ## Input [INPUT] ## Output Schema Return a JSON object with these fields: { "in_scope": boolean, "confidence": float (0.0 to 1.0), "primary_modality": string, "detected_language": string or null, "out_of_scope_reasons": [string] (empty if in_scope), "suggested_pipeline": string or null (alternative processing path if available), "requires_human_review": boolean } ## Constraints - If multiple modalities are present, check each one against the supported scope - When text and attachment modalities conflict, flag the input as ambiguous - Do not process or respond to the content of out-of-scope inputs beyond classification - For safety violations, set requires_human_review to true and do not suggest an automated pipeline - If confidence is below [CONFIDENCE_THRESHOLD], set requires_human_review to true - Never hallucinate a supported pipeline for an input that is genuinely unsupported
Adapt this template by replacing each placeholder with your system's concrete boundaries. The [SUPPORTED_MODALITIES] field should enumerate exactly which input types your system handles—for example, ["text", "image", "audio"]. The [SAFETY_POLICY_BOUNDARIES] placeholder should reference your organization's specific content policy categories. Set [CONFIDENCE_THRESHOLD] to a value like 0.7 that balances false positives against missed out-of-scope detections. Before deploying, run this prompt against a test suite that includes adversarial inputs designed to look in-scope—such as supported file types containing disallowed content—and measure both the recall of true out-of-scope inputs and the false-positive rate on legitimate edge cases.
Prompt Variables
Replace each placeholder with your system's specific configuration before deployment. Validation checks prevent silent failures from misconfigured boundaries.
| Placeholder | Purpose | Example | Validation Notes |
|---|---|---|---|
[SUPPORTED_MODALITIES] | Defines the allowed input types the system can process | text, image, audio, video, pdf | Must be a comma-separated list matching your pipeline capabilities. Validate against actual adapter availability at startup. |
[SUPPORTED_LANGUAGES] | Lists language codes the system accepts for text and audio inputs | en, es, fr, de, ja | Use ISO 639-1 codes. Reject inputs with languages outside this list. Validate against transcription and translation service coverage. |
[SUPPORTED_FILE_TYPES] | Specifies allowed file extensions and MIME types for attachments | png, jpg, mp3, mp4, pdf, docx | Validate against actual parser availability. Blocklisted types should trigger rejection before file processing begins. |
[OUT_OF_SCOPE_CATEGORIES] | Content categories explicitly outside the system's safety or capability envelope | medical_diagnosis, legal_advice, financial_trading, adult_content, violence | Each category must have a defined policy reference. Ambiguous categories cause inconsistent rejection. Test with boundary cases. |
[REJECTION_RESPONSE_TEMPLATE] | Standardized message returned when input is out of scope | I cannot process this request because it falls outside my supported capabilities. Consider [SUGGESTED_ALTERNATIVE]. | Must include reason code and optional alternative. Validate that the template never hallucinates capabilities. Log rejection reason for monitoring. |
[SUGGESTED_ALTERNATIVE_PIPELINE] | Optional routing hint for inputs that could be handled by a different system | document_processing_v2, human_review_queue, translation_service | Must map to an actual deployed pipeline or null. Null is valid when no alternative exists. Validate pipeline availability before suggesting. |
[CONFIDENCE_THRESHOLD] | Minimum confidence score required to classify an input as in-scope | 0.85 | Float between 0.0 and 1.0. Inputs below threshold route to ambiguity handling. Calibrate against your false-positive tolerance for in-scope misclassification. |
[AMBIGUITY_ESCALATION_ACTION] | Action taken when confidence is below threshold or modalities conflict | route_to_clarification, escalate_to_human, reject_with_explanation | Must be one of the enumerated actions. Validate that the target queue or workflow exists. Null means reject without escalation. |
Implementation Harness Notes
How to wire the Multi-Modal Out-of-Scope Detection System Prompt into a production application with validation, retries, logging, and human review.
This prompt acts as a policy enforcement gate at the ingress layer of your AI system. It should be invoked before any expensive model inference, tool execution, or data storage. The application must extract the structured JSON output and use the in_scope boolean to branch: true routes to the normal processing pipeline, while false triggers a graceful rejection response using the rejection_message and suggested_alternative fields. Do not pass the raw model output directly to the user; always parse and validate the JSON schema first.
Validation and Retries: Implement a strict JSON schema validator that checks for the required fields (in_scope, confidence_score, rejection_reason, rejection_message, suggested_alternative). If parsing fails or required fields are missing, retry the prompt once with the error message appended to the system prompt. If the retry also fails, log the raw output and escalate to a human review queue. For high-risk domains, set a minimum confidence_score threshold (e.g., 0.8) below which ambiguous inputs are treated as out-of-scope or routed to human review, even if in_scope is true. This prevents over-confident misclassification on adversarial inputs designed to look in-scope.
Model Choice and Latency: Use a fast, cost-efficient model (e.g., GPT-4o-mini, Claude Haiku) for this classification step. The prompt is designed to be a lightweight gate, not a deep reasoning task. Implement a strict latency budget (e.g., 500ms p95) and a fallback model if the primary model times out. For multi-modal inputs, ensure your application pre-processes attachments into the required format (base64 for images, extracted text for documents) before constructing the prompt. Log every classification decision with the input hash, model version, confidence score, and routing outcome for auditability and drift monitoring.
Human Review Integration: When the prompt returns in_scope: false with a rejection_reason of uncertain_boundary or confidence_score below your threshold, route the input to a human review queue with the full context and the model's suggested rejection message. The reviewer can override the classification, approve the rejection, or refine the boundary definitions. This feedback loop is critical for improving the prompt's boundary definitions over time. Never silently drop out-of-scope inputs; always provide a logged rejection with a traceable reason to avoid debugging blind spots when legitimate users are blocked.
Expected Output Contract
The model must return a JSON object matching this schema exactly. Any deviation should be caught by a post-processing validator and trigger a retry or escalation.
| Field or Element | Type or Format | Required | Validation Rule |
|---|---|---|---|
in_scope | boolean | Must be true or false. If false, all other fields except out_of_scope_reason and suggested_pipeline must be null. | |
out_of_scope_reason | string | null | Required if in_scope is false. Must be a non-empty string from the allowed boundary violation list. Must be null if in_scope is true. | |
suggested_pipeline | string | null | Required if in_scope is false and a known alternative exists. Must be a valid pipeline identifier from the system registry. Must be null if in_scope is true. | |
detected_modalities | string[] | Must be a non-empty array of strings from the allowed modality list: text, image, audio, video, document. Must not contain duplicates. | |
primary_language | string | null | Must be a valid BCP-47 language tag or null if no text/audio language is detectable. Must be null for image-only inputs without embedded text. | |
content_category | string | Must be a single string from the allowed content category taxonomy. Must not be a free-form string. | |
confidence_score | number | Must be a float between 0.0 and 1.0 inclusive. Values below the configured threshold must trigger a clarification or human review flow. | |
boundary_violations | object[] | If present, each object must contain a violation_type string and a modality string. Must be an empty array if in_scope is true. Violation types must match the allowed boundary violation list. |
Common Failure Modes
Multi-modal out-of-scope detection fails in predictable ways. These are the most common production failure patterns and how to prevent them before they reach users.
Hallucinated Routing on Adversarial Inputs
What to watch: Attackers craft inputs that appear in-scope by mimicking supported modalities or content patterns. The prompt confidently routes them to a processing pipeline that then fails, leaks instructions, or executes unintended tool calls. Guardrail: Add a secondary boundary check after classification that validates the input against concrete capability schemas before dispatch. Test with adversarial examples that combine in-scope text with out-of-scope attachments.
Modality Conflict Resolution Failures
What to watch: Text says one thing, the attached image or audio says another. The prompt picks one modality as ground truth without flagging the conflict, routing the input to the wrong pipeline. Guardrail: Require explicit conflict detection in the output schema. When modalities disagree, route to a clarification or human-review queue rather than silently choosing. Test with mismatched text-image pairs.
Boundary Drift Under Input Variation
What to watch: The prompt works on clean test cases but silently accepts out-of-scope inputs when users vary phrasing, file formats, or attachment combinations. Boundary definitions that rely on pattern matching rather than semantic understanding degrade under real-world diversity. Guardrail: Maintain a regression suite of boundary-probing inputs including edge cases like empty attachments, corrupted files, and mixed-modality payloads. Run before every prompt version change.
Over-Rejection of Legitimate Edge Cases
What to watch: The prompt becomes too conservative and rejects valid inputs that resemble out-of-scope patterns—screenshots of code, documents with embedded images, or multilingual content with uncommon scripts. Users hit dead ends for supported workflows. Guardrail: Track rejection rates by input category in production. Set a maximum false-rejection threshold and tune boundary definitions when legitimate workflows are blocked. Include positive edge cases in eval suites.
Missing Fallback Pipeline Specification
What to watch: The prompt correctly identifies an out-of-scope input but provides no actionable alternative—no suggested pipeline, no escalation path, no user-facing message. The rejection is technically correct but operationally useless. Guardrail: Require the output schema to include a suggested_alternative field whenever an input is rejected. If no alternative exists, route to a human queue with the rejection reason preserved for audit.
Silent Acceptance of Unsupported File Types
What to watch: The prompt checks content semantics but not file format validity. Users upload unsupported or corrupted file types that pass semantic checks but fail downstream processing, producing cryptic errors or empty outputs. Guardrail: Add a pre-classification format validation layer that checks MIME types, file signatures, and structural integrity before the prompt runs. Reject malformed files at the ingress layer with clear user feedback.
Evaluation Rubric
How to test output quality before shipping. Run these checks against a labeled dataset of at least 200 examples spanning in-scope, out-of-scope, and adversarial cases.
| Criterion | Pass Standard | Failure Signal | Test Method |
|---|---|---|---|
In-Scope Recall |
| Supported inputs classified as out-of-scope or routed to fallback | Run against 100 labeled in-scope examples covering all supported modalities, languages, and content categories |
Out-of-Scope Precision |
| Unsupported inputs classified as in-scope and processed by downstream pipelines | Run against 50 labeled out-of-scope examples including unsupported languages, file types, and prohibited content categories |
Adversarial Boundary Resistance | <= 0.05 false acceptance rate on adversarial inputs designed to look in-scope | Hallucinated routing on inputs with subtle out-of-scope signals (e.g., supported language wrapping unsupported request) | Run against 30 adversarial examples: jailbreak-style wrappers, dual-intent inputs, and modality-mismatched payloads |
Rejection Reason Accuracy |
| Correct rejection decision but wrong reason code (e.g., unsupported language vs. unsupported file type) | Compare predicted rejection reason against ground-truth label on all rejected examples |
Alternative Pipeline Suggestion Quality |
| Suggested pipeline does not exist, is deprecated, or would also reject the input | Human review of 50 suggested alternatives; mark valid only if the suggested pipeline exists and can handle the input |
Confidence Score Calibration | Expected Calibration Error <= 0.08 on confidence scores | High confidence on incorrect classifications or low confidence on correct ones | Compute ECE across all 200 test examples using 10 equal-width bins |
Modality Conflict Resolution |
| System trusts one modality and ignores the other without explicit conflict flag | Run against 20 multi-modal conflict examples where text is in-scope but image is out-of-scope or vice versa |
Latency Budget Compliance | p95 classification latency <= 500ms for single-item inference | Classification exceeds latency budget, causing downstream pipeline timeouts | Measure end-to-end classification latency across 200 requests under load; flag if p95 exceeds threshold |
Enabling Efficiency, Speed & Accuracy
Intelligent Analysis, Decision & Execution
We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.
Talk to Us
Search across company data
Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.
Useful when people spend too long searching or get different answers from different systems.

Automate internal workflows
Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.
Useful when repetitive work moves across multiple tools and teams.

Add AI to products and internal tools
Build assistants, guided actions, or decision support into the software your team or customers already use.
Useful when AI needs to be part of the product, not a separate tool.
Adapt This Prompt
How to adapt
Use the base system prompt with a single frontier model. Accept raw JSON output without strict schema enforcement. Define boundaries as a flat list of unsupported categories. Log rejections manually.
Prompt snippet
code[SYSTEM_ROLE]: You are a content safety classifier. [BOUNDARIES]: Reject inputs containing: [UNSUPPORTED_LANGUAGES], [DISALLOWED_CONTENT_TYPES], [OUT_OF_SCOPE_MODALITIES]. [OUTPUT]: Return {"in_scope": bool, "reason": string}
Watch for
- Hallucinated routing on adversarial inputs designed to look in-scope
- Missing modality conflict resolution when text and image disagree
- Overly broad boundary definitions that reject legitimate edge cases

About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us