Capability Hallucination Prevention System Prompt

This prompt is a system-level instruction block for agent framework developers who need to prevent agents from inventing capabilities they do not possess. It is designed to be placed at the top of the system prompt for any agent that has access to a dynamic tool registry. The prompt constrains the agent to only claim, describe, or offer to use tools that are explicitly present in its provided [TOOL_REGISTRY]. It also provides explicit refusal language for when a user asks for an unregistered capability. Use this prompt when you are building an agent that discovers tools at runtime from MCP servers, internal APIs, or plugin ecosystems, and you need a hard boundary against capability hallucination.

The ideal user is an AI platform engineer or agent framework developer who controls the system prompt assembly pipeline. The required context is a populated [TOOL_REGISTRY]—a structured list of tool names, descriptions, and parameter schemas that the agent can reference. Without this registry injected into the prompt context, the instruction block has nothing to enforce against and will produce false negatives (refusing to use tools that actually exist). The prompt assumes the registry is authoritative and up-to-date; stale registries will cause the agent to claim capabilities that have been removed or to refuse valid tools that were recently added. Pair this prompt with a Dynamic Tool Registry Refresh Prompt to keep the injected registry current.

Do not use this prompt as a standalone safety layer for agents that perform high-risk write operations, financial transactions, or clinical actions without additional human-in-the-loop approval prompts. This prompt prevents capability hallucination—it does not prevent an agent from misusing a tool it legitimately has access to. For agents with destructive or regulated tool access, combine this prompt with an Action Permission, Scope, and Safety Boundaries prompt and a Human-in-the-Loop Approval and Escalation for Tools prompt. Also avoid using this prompt when the tool registry is empty or undefined; the refusal language will cause the agent to reject every user request, creating a broken experience. If you need the agent to gracefully handle an empty registry, modify the refusal template to offer alternative help rather than flat denial.

This system prompt is designed to be placed at the top of your agent's system instructions. It establishes a strict boundary between what the agent can actually do—based on its registered tool set—and what it might otherwise hallucinate. The prompt forces the agent to introspect its tool registry before making any capability claim and provides explicit refusal templates when a user requests something outside its available tools. This is critical for production agents where capability hallucination erodes user trust and causes silent failures.

textCopy
You are an agent with access to a fixed set of tools registered in [TOOL_REGISTRY]. Your capabilities are strictly limited to what these tools can perform. You do not have any capabilities beyond those explicitly provided by the registered tools.

Before responding to any user request that implies a capability, you MUST:
1. Check whether the requested capability is supported by at least one tool in [TOOL_REGISTRY].
2. If supported, proceed with the appropriate tool call or explanation.
3. If NOT supported, respond using one of the following refusal templates:

REFUSAL TEMPLATES:
- Direct refusal: "I don't have access to a tool that can [REQUESTED_CAPABILITY]. My available capabilities are limited to [CAPABILITY_SUMMARY]."
- Partial match: "I can [RELATED_CAPABILITY] using [TOOL_NAME], but I cannot [REQUESTED_CAPABILITY]. Would you like me to proceed with what I can do?"
- No related capability: "I cannot [REQUESTED_CAPABILITY]. I have no tools registered for this type of task. Is there something else I can help with within my available capabilities?"

CAPABILITY CLAIM RULES:
- Never claim you can perform an action unless a specific tool in [TOOL_REGISTRY] enables it.
- Never imply future capabilities or say you "might be able to" without tool evidence.
- When listing your capabilities, only enumerate what is directly supported by registered tools.
- If [TOOL_REGISTRY] is empty, state: "I currently have no tools registered and cannot perform any external actions."

TOOL REGISTRY REFERENCE:
[TOOL_REGISTRY]

CAPABILITY SUMMARY (auto-generated from registry):
[CAPABILITY_SUMMARY]

CONSTRAINTS:
[CONSTRAINTS]

To adapt this template for your deployment, replace [TOOL_REGISTRY] with a structured representation of your agent's available tools—ideally a JSON array of tool names, descriptions, and parameter schemas. The [CAPABILITY_SUMMARY] placeholder should contain a concise, auto-generated summary of what the registry enables, refreshed whenever the registry changes. The [CONSTRAINTS] block is where you add domain-specific restrictions, such as read-only boundaries, rate limits, or compliance rules. Test this prompt by feeding it requests for capabilities you know are absent and verifying the refusal templates fire correctly. If your agent operates in a regulated domain, add a human escalation path in the refusal templates and log every capability mismatch for audit review.

This system prompt is not a standalone chat instruction; it is a runtime guardrail that must be injected into the agent's context immediately after tool discovery completes and before any user task is processed. The prompt's job is to freeze the agent's perceived capability boundary to exactly what the tool registry contains at that moment. In a typical implementation, you will maintain a ToolRegistry object (populated by MCP server introspection, OpenAPI spec parsing, or a static manifest) and serialize its contents into the [TOOL_MANIFEST] placeholder. The [TOOL_MANIFEST] should be a structured JSON array of tool definitions, each containing at minimum name, description, parameters (JSON Schema), and constraints (e.g., read-only, requires_approval). Do not pass raw server responses or unvalidated user input into this slot; the manifest must be the single source of truth for capability claims.

Integration flow: (1) On agent session start or registry refresh, serialize the current ToolRegistry to a compact JSON string. (2) Inject the system prompt template with [TOOL_MANIFEST] replaced by that serialized registry. (3) Append this as a high-priority system message above any developer or user instructions, ensuring it cannot be overridden by lower-priority prompts. (4) Before every tool call, validate that the requested tool name and arguments exist in the registry snapshot; if not, abort the call and return the refusal template from the prompt. Model choice: This prompt works best with models that have strong instruction-following and low hallucination rates on structured constraints (e.g., Claude 3.5 Sonnet, GPT-4o). Smaller or older models may still hallucinate capabilities despite the guardrail; for those, add a deterministic pre-flight check in your harness that rejects any tool call not present in the registry before the model's output reaches the execution layer.

Validation and evals: Build an eval suite that tests the agent with requests for unregistered capabilities (e.g., 'Can you delete the production database?' when no delete tool exists). The expected output must match the refusal template exactly or contain the REFUSAL marker. Track two metrics: capability hallucination rate (how often the agent claims a capability not in the manifest) and refusal accuracy (how often it correctly refuses unknown capabilities without refusing valid ones). For high-risk deployments, route all tool calls through a human approval queue if the tool's requires_approval constraint is set, and log every capability claim and refusal to an audit trail with the registry snapshot hash for traceability. Avoid wiring this prompt into a loop that refreshes the registry mid-task without re-injecting the updated manifest; stale manifests are a primary failure mode.