Tool Selection Policy to Argument Construction Example Prompt

This prompt is for agent developers and AI engineers who need to convert a written tool selection policy—a set of rules dictating which tool to call and how to populate its arguments—into a compact set of few-shot examples. The job-to-be-done is behavioral teaching: you want the model to internalize a decision boundary through demonstrations rather than through a long, procedural system prompt. The ideal user has a documented policy, a defined set of function schemas, and a need to reduce token costs or improve tool-selection accuracy in production. Required context includes the raw policy text, the JSON schema for each available tool, and a set of representative user requests that span the policy's decision surface.

You should use this prompt when your current instruction-based approach is brittle, verbose, or produces inconsistent tool choices. It is particularly effective when the policy involves nuanced trade-offs—for example, choosing between a fast cached lookup and a live API call based on data freshness requirements. The prompt works by having the model analyze the policy, identify distinct decision scenarios, and generate user_request -> tool_call pairs that demonstrate correct selection and argument construction. The output is not just a list of examples; it should include a coverage map showing which policy clauses each example addresses, allowing you to audit for gaps before deployment.

Do not use this prompt when the tool selection policy is trivial (a single tool or a simple keyword match), when the policy changes faster than you can regenerate and revalidate examples, or when the available tools have overlapping functionality that requires real-time context the model cannot access during example generation. This prompt is also inappropriate for high-risk agentic workflows where tool calls trigger irreversible actions (e.g., financial transactions, patient record modifications) without a human-in-the-loop review step. In those cases, the generated examples should feed into a separate approval harness, not directly into an autonomous agent. After using this prompt, your next step is to run the generated examples through an eval harness that measures tool selection accuracy and argument hallucination rates against a golden test set derived from your policy.

This prompt template transforms a written tool selection policy into a set of few-shot examples that teach a model which tool to pick and how to construct its arguments. The template is designed for agent developers who need to move from abstract rules—such as 'use the search tool for factual queries and the calculator for arithmetic'—to concrete user-request-to-tool-call pairs that can be inserted directly into a system prompt or fine-tuning dataset. Each generated example includes the user's natural-language request, the correct tool name, and a fully populated arguments object. The prompt also produces negative examples showing incorrect tool selections and common argument errors, which are critical for preventing hallucinated parameters and tool misuse in production.

textCopy
You are converting a tool selection policy into few-shot function-calling demonstrations.

## TOOLS AVAILABLE
[TOOLS]

## TOOL SELECTION POLICY
[POLICY]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "positive_examples": [
    {
      "user_request": "string (the natural-language user input)",
      "selected_tool": "string (exact tool name from TOOLS)",
      "arguments": { },
      "rationale": "string (one-sentence explanation of why this tool was selected)"
    }
  ],
  "negative_examples": [
    {
      "user_request": "string",
      "incorrect_tool": "string (a plausible but wrong tool choice)",
      "incorrect_arguments": { },
      "error_type": "string (wrong_tool | hallucinated_parameter | missing_required | type_mismatch)",
      "correct_tool": "string",
      "correct_arguments": { },
      "rationale": "string (why the incorrect choice fails and what the correct choice is)"
    }
  ],
  "boundary_examples": [
    {
      "user_request": "string (ambiguous, multi-intent, or out-of-scope request)",
      "expected_behavior": "string (ask_clarification | refuse | escalate | fallback_tool)",
      "rationale": "string"
    }
  ]
}

## CONSTRAINTS
- Generate exactly [NUM_POSITIVE] positive examples and [NUM_NEGATIVE] negative examples.
- Cover every tool listed in TOOLS at least once across the positive examples.
- Negative examples must include at least one instance of each error_type.
- Arguments must use the exact parameter names and types defined in TOOLS.
- Never invent parameters that are not in the tool definitions.
- For boundary examples, include requests that fall between tool capabilities, have conflicting intents, or are outside the policy scope.
- If [RISK_LEVEL] is "high", add an "approval_required" boolean field to each example and set it to true for any tool call that modifies data, sends messages, or accesses sensitive resources.

## EXAMPLES OF DESIRED OUTPUT
[EXAMPLES]

## INPUT
Generate the demonstration set for the policy and tools above.

To adapt this template, replace the square-bracket placeholders with your actual values. [TOOLS] should contain the full function definitions in your model's native tool format—include parameter names, types, descriptions, and required fields exactly as the model will see them at inference time. [POLICY] is the natural-language policy document you are converting; paste it in full, including any priority rules, conflict resolution instructions, and domain-specific constraints. [NUM_POSITIVE] and [NUM_NEGATIVE] control the size of the generated example set; start with 5–8 positive and 3–5 negative examples for most policies, then increase if coverage gaps appear during evaluation. [EXAMPLES] is optional but strongly recommended—provide 1–2 completed demonstrations in the exact output schema to anchor the model's format and quality expectations. Set [RISK_LEVEL] to "high" if any tool in your set can modify data, send external communications, or access production systems; this activates the approval flag and ensures your downstream harness can gate those calls behind human review. After generating the examples, run them through your tool selection eval suite to measure selection accuracy and argument hallucination rates before inserting them into your agent's system prompt.

This prompt is designed to sit inside an agent's tool-calling loop, converting a user request and a list of available tool schemas into a specific function call. The implementation harness must treat the model's output as a structured function invocation, not free text. The application layer should parse the model's response into a tool name and a dictionary of arguments, then validate both against the actual tool registry before execution. This prompt is not a standalone chatbot; it is a component that feeds into a tool executor, and the harness is responsible for enforcing the contract between the model's suggestion and the system's capabilities.

To wire this into an application, start by constructing the prompt payload with three dynamic inputs: the user's natural-language request, a JSON array of available tool definitions (each with a name, description, and a JSON Schema for parameters), and a set of few-shot examples that demonstrate correct tool selection and argument construction. The model should be called with tool_choice set to "required" or the equivalent for your provider, forcing a structured tool-call response rather than a text completion. After receiving the response, validate the tool name against the registry; if the model hallucinates a tool that doesn't exist, log the mismatch and either retry with a stronger constraint or fall back to a clarification request. For the arguments, validate them against the tool's parameter schema using a JSON Schema validator. Common failure modes include the model inventing parameter names, omitting required fields, or passing values of the wrong type. If validation fails, construct a retry prompt that includes the original request, the invalid tool call, and the specific validation errors, asking the model to correct only the argument payload. Limit retries to two attempts before escalating to a human or a fallback model.

For production observability, log every tool selection event with the user request, the model's chosen tool, the raw arguments, the validation result, and whether the tool execution succeeded. This trace data is essential for measuring tool selection accuracy and argument hallucination rates over time. Use these logs to build an eval set: sample 100–200 real user requests, run them through the prompt, and check whether the selected tool and arguments match expected behavior. Pay special attention to near-miss failures where the model picks a plausible but incorrect tool—these are the hardest to detect without structured evaluation. If your agent operates in a high-risk domain where incorrect tool execution could cause harm, insert a human approval step before any tool call that modifies external state, creates financial transactions, or accesses sensitive data. The prompt itself should not be the only safety boundary; the harness must enforce execution gates.