Tool Argument Hallucination Prevention Few-Shot Prompt

This playbook is for AI engineers and agent developers who are moving beyond single-tool demos and into production multi-tool systems. The core job-to-be-done is eliminating fabricated parameter values—invented IDs, dates that don't exist in the context, or entity names the model guessed instead of extracted—when the model constructs a tool call. You need this prompt when your agent has access to a defined set of tools with strict schemas, and you are observing hallucinations that a schema description alone cannot prevent. The ideal user is someone who has already written a system prompt and tool definitions but is seeing argument drift in production logs.

This approach is most effective when you have a catalog of real failure patterns. You should use it when you can collect 3–6 concrete examples of incorrect tool calls from your logs, pair each with a corrected version, and present them as negative/positive demonstration pairs. The prompt is designed for workflows where the available context (e.g., a user message, a retrieved document, or a database record) contains the ground truth for argument values, and the model's job is strict extraction and mapping, not creative generation. It works well with structured output formats like JSON function calls or XML tool blocks.

Do not use this prompt as a substitute for proper tool schema definitions or for validation in the application layer. If your tool API can return a clear error for invalid IDs, you should still implement that check. This prompt reduces hallucination probability but does not guarantee correctness. Avoid it when your tool arguments require reasoning over implicit knowledge the model must infer, rather than extracting explicit values from the provided context. It is also a poor fit for open-ended agentic loops where the model needs to explore and discover valid arguments through trial and error; in those cases, pair it with a validation and retry harness instead.

Before implementing, gather your negative examples from production traces. Look for patterns: hallucinated UUIDs, dates outside the context's range, entity names that appear in the model's training data but not in the user's input. Each negative example should show the incorrect tool call and the context that was available. The positive example should show the correct extraction or, where no valid argument exists, a clarification request or abstention. After deploying, monitor argument validity rates and be prepared to add new examples as you discover novel failure modes.

The following prompt template uses negative and positive example pairs to teach the model the critical difference between inventing plausible-sounding parameter values and extracting only the values explicitly present in the provided context. This pattern is essential for production agents where hallucinated IDs, dates, or entity names cause downstream API errors, data corruption, or incorrect transactions. The template is designed to be copied directly and adapted by replacing the square-bracket placeholders with your specific tool definitions, context sources, and known hallucination patterns.

textCopy
You are an agent that calls tools using only information present in the provided context. Never invent or guess parameter values. If a required parameter value is not present in the context, do not call the tool. Instead, respond with a clarification request.

Available tools:
[TOOLS]

Context:
[CONTEXT]

User request:
[USER_REQUEST]

Here are examples of incorrect and correct tool calls:

---
Example 1 (INCORRECT - Hallucinated ID):
Context: "Customer Acme Corp recently signed up. Their account is active."
User request: "Cancel the Acme Corp account."
INCORRECT tool call: cancel_account(account_id="ACC-12345")
Why wrong: The account ID "ACC-12345" was not in the context. The model invented it.

CORRECT response: "I need the account ID for Acme Corp before I can cancel it. Could you provide that?"

---
Example 2 (INCORRECT - Hallucinated Date):
Context: "The Q3 report was published last week."
User request: "Send me the Q3 report from October 1st."
INCORRECT tool call: fetch_report(report_name="Q3 Report", date="2024-10-01")
Why wrong: The date "2024-10-01" was not in the context. The model assumed it from the user's mention, but the user's date may be incorrect or refer to a different report.

CORRECT tool call: fetch_report(report_name="Q3 Report")

---
Example 3 (INCORRECT - Hallucinated Entity):
Context: "Meeting with the design team is scheduled."
User request: "Invite Sarah to the design meeting."
INCORRECT tool call: add_meeting_participant(meeting_id="meet_design_01", participant_email="sarah@company.com")
Why wrong: Neither the meeting ID nor Sarah's email were in the context.

CORRECT response: "I don't have the meeting ID or Sarah's email in the current context. Can you provide those details?"

---
Now, process the following user request using only the context provided above. If any required parameter is missing, request clarification instead of calling the tool.

To adapt this template, replace [TOOLS] with your actual function definitions in JSON Schema or a concise description format. Replace [CONTEXT] with the source material the model should ground its arguments in—this could be retrieved documents, a user profile, conversation history, or database records. Replace [USER_REQUEST] with the incoming user message. The three example pairs should be customized to reflect the most common hallucination patterns observed in your specific tool set: invented IDs, assumed dates, fabricated entity names, or made-up enum values. For high-risk domains such as finance or healthcare, add a fourth example showing a dangerous hallucination that could cause harm, and include a system-level instruction to escalate to a human reviewer when critical parameters are missing.

After copying this template, test it against a golden set of inputs where you know exactly which parameters are and are not present in the context. Measure two failure modes: false positives where the model calls a tool with a hallucinated argument, and false negatives where the model requests clarification for a parameter that is actually present. The negative examples in the prompt should be updated whenever you discover a new hallucination pattern in production traces. Do not rely on this prompt alone for safety—always implement a post-generation validation layer that checks tool call arguments against the source context before execution.

This prompt is designed to sit directly before the model's tool-calling step in your agent architecture. It is not a standalone chat prompt; it must be injected as a system or developer message immediately preceding the user turn that requires a tool call. The prompt template expects a pre-assembled context containing the user's request, the available tool schemas, and the few-shot examples. The model's output should be a single tool call object, not a conversational reply. In a typical implementation, you will maintain a small, curated example store (3–5 negative examples paired with their corrected positive counterparts) that is dynamically selected based on the tool being invoked or the argument type at risk (e.g., dates, IDs, enumerated values).

After the model returns a tool call, you must run a validation harness before executing the function. This harness should check: (1) that all argument values for fields like id, date, customer_reference, or project_code are exact string matches present in the provided context or tool schema enums—not fabricated values; (2) that required fields are not null or empty; and (3) that the tool name exactly matches one of the available tools. If validation fails, do not execute the tool. Instead, construct a retry turn that includes the original context, the model's failed tool call, and a specific error message like Validation failed: 'customer_id' value 'CUST-9999' not found in provided context. Re-read the context and correct the argument. This error message becomes the user turn for the next invocation of the same prompt, creating a tight self-correction loop. Log every validation failure with the full prompt context and model output for later example curation.

For model choice, use a model with strong instruction-following and native tool-calling support (e.g., GPT-4o, Claude 3.5 Sonnet). Avoid weaker models that struggle to maintain argument discipline even with few-shot examples. Set temperature to 0 or very low (0.1) to reduce variance in argument selection. If your agent uses many tools, consider maintaining separate example sets per tool or per argument risk category, and select the most relevant 2–3 negative/positive pairs to keep the prompt within token budgets. Do not overload the prompt with examples for tools not relevant to the current turn. Finally, implement a hard limit on retries (2–3 attempts) before escalating to a human or falling back to a clarification request to the user. This prevents infinite retry loops when the context genuinely lacks the required information.