Parallel Tool Call Planning Few-Shot Prompt

This prompt is designed for agent developers and AI engineers who need to reduce end-to-end latency in multi-tool workflows. The core job-to-be-done is teaching a model to analyze a set of available tools and a user's complex intent, then output a plan that groups independent tool calls for parallel execution. The ideal user is someone building a production agent where sequential tool calling creates unacceptable wait times for users, such as in customer-facing copilots or real-time data enrichment pipelines. Required context includes a complete list of available tool schemas, the user's objective, and any known dependencies between tools.

Use this prompt when you have a mature agent with 3+ tools and you've already solved basic tool selection and argument construction. It is most effective when latency profiling shows that sequential execution is the primary bottleneck, not model inference time. Do not use this prompt if your tool set changes dynamically per session without clear dependency documentation, as the few-shot examples will teach stale dependency assumptions. It is also inappropriate for workflows where all tool calls are inherently sequential—for example, a call that requires the output of a previous call as an input—as parallel planning adds unnecessary complexity and risks race conditions. The prompt assumes the model can reason about data dependencies, not just semantic similarity, so it requires a capable frontier model.

Before implementing this prompt, ensure you have a validation harness that can detect false dependencies (marking calls as sequential when they are independent) and missed parallelization opportunities (executing independent calls one by one). The next step is to pair this planning prompt with an execution runtime that respects the parallel groups and handles partial failures without deadlocking. Avoid using this prompt as a standalone solution; it must be integrated into an agent loop where the plan is parsed, validated, and executed with observability into which calls actually ran in parallel.

This prompt template teaches the model to analyze a set of available tools and a user's request, then output a dependency graph and an execution plan that maximizes parallel execution. It uses few-shot examples to demonstrate the difference between true data dependencies (where one call's output is required as another's input) and false dependencies (where calls appear related but can execute independently). The template is designed for agent systems where reducing end-to-end latency is critical, such as multi-source research agents, data aggregation pipelines, and complex workflow orchestrators.

textCopy
You are a tool-call planning assistant. Your job is to analyze a user request and a set of available tools, then produce a parallel execution plan that minimizes total latency while respecting true data dependencies.

## AVAILABLE TOOLS
[TOOLS]

## USER REQUEST
[INPUT]

## INSTRUCTIONS
1. Identify every tool call required to fulfill the user request.
2. For each tool call, determine its required inputs and whether those inputs depend on the output of any other tool call.
3. Classify dependencies as:
   - **HARD**: The output of call A is a required input for call B. B must wait for A.
   - **NONE**: The calls share no data dependency and can execute in parallel.
4. Group tool calls into execution waves, where all calls within a wave can run simultaneously.
5. For each tool call, specify the exact arguments using only information available at that wave (from the user request or from completed prior waves).
6. Flag any potential race conditions where parallel calls might conflict (e.g., writes to the same resource).

## OUTPUT FORMAT
Return a JSON object with this exact structure:
[OUTPUT_SCHEMA]

## CONSTRAINTS
[CONSTRAINTS]

## EXAMPLES

### Example 1: Independent Calls
**User Request:** "Get the weather for Tokyo and the latest news about AI."
**Available Tools:** get_weather(city: string), search_news(query: string)

**Analysis:**
- Call A: get_weather(city="Tokyo") — no dependencies
- Call B: search_news(query="AI") — no dependencies
- Dependency: NONE between A and B

**Plan:**
```json
{
  "waves": [
    {
      "wave_id": 1,
      "parallel_calls": [
        {"tool": "get_weather", "arguments": {"city": "Tokyo"}, "call_id": "A"},
        {"tool": "search_news", "arguments": {"query": "AI"}, "call_id": "B"}
      ]
    }
  ],
  "dependency_graph": {"A": [], "B": []},
  "race_condition_risks": []
}

Example 2: Sequential Dependency

User Request: "Find the capital of France, then get the weather there." Available Tools: get_capital(country: string), get_weather(city: string)

Analysis:

• Call A: get_capital(country="France") — no dependencies
• Call B: get_weather(city=<output of A>) — HARD dependency on A
• Dependency: B depends on A

Plan:

jsonCopy
{
  "waves": [
    {
      "wave_id": 1,
      "parallel_calls": [
        {"tool": "get_capital", "arguments": {"country": "France"}, "call_id": "A"}
      ]
    },
    {
      "wave_id": 2,
      "parallel_calls": [
        {"tool": "get_weather", "arguments": {"city": "$A.capital"}, "call_id": "B"}
      ]
    }
  ],
  "dependency_graph": {"A": [], "B": ["A"]},
  "race_condition_risks": []
}

Example 3: False Dependency Detection

User Request: "Get stock prices for AAPL and MSFT, and also get the latest news about AAPL." Available Tools: get_stock_price(symbol: string), search_news(query: string)

Analysis:

• Call A: get_stock_price(symbol="AAPL") — no dependencies
• Call B: get_stock_price(symbol="MSFT") — no dependencies
• Call C: search_news(query="AAPL") — no dependencies (does NOT require call A's output)
• Dependency: NONE among A, B, C — all three can run in parallel despite sharing the ticker "AAPL"

Plan:

jsonCopy
{
  "waves": [
    {
      "wave_id": 1,
      "parallel_calls": [
        {"tool": "get_stock_price", "arguments": {"symbol": "AAPL"}, "call_id": "A"},
        {"tool": "get_stock_price", "arguments": {"symbol": "MSFT"}, "call_id": "B"},
        {"tool": "search_news", "arguments": {"query": "AAPL"}, "call_id": "C"}
      ]
    }
  ],
  "dependency_graph": {"A": [], "B": [], "C": []},
  "race_condition_risks": []
}

[EXAMPLES]

RISK LEVEL

[RISK_LEVEL]

Now analyze the user request and produce the parallel execution plan.

To adapt this template, replace the square-bracket placeholders with your specific context. The [TOOLS] placeholder should contain your tool definitions in a structured format (OpenAPI, JSON Schema, or a simple name-and-parameters list). The [OUTPUT_SCHEMA] should define the exact JSON structure your execution harness expects, including fields for waves, dependency_graph, and race_condition_risks. The [EXAMPLES] placeholder is where you add domain-specific scenarios that reflect your actual tool set and common user requests—these are the most critical element because they teach the model to distinguish true data dependencies from coincidental parameter overlap. The [CONSTRAINTS] field should specify limits like maximum parallel calls per wave, timeout assumptions, or idempotency requirements for write operations. For high-risk workflows where parallel execution could cause data corruption (e.g., concurrent database writes), set [RISK_LEVEL] to "high" and add explicit instructions requiring human review of the plan before execution.

Before deploying this prompt, validate it against a test suite that includes: requests where all calls are independent (should produce a single wave), requests with a clear linear dependency chain (should produce sequential waves), and requests with false dependencies (should correctly identify parallelism opportunities). Common failure modes include the model serializing calls that share a topic but not data, failing to propagate intermediate results between waves, and missing race conditions when parallel calls target the same mutable resource. Run these test cases through your eval harness and measure both plan correctness and the achieved parallelism ratio (independent calls executed in parallel divided by total independent calls). If the model consistently misses parallelization opportunities, add more false-dependency examples to the [EXAMPLES] section. If it creates unsafe parallel plans, strengthen the [CONSTRAINTS] and add negative examples showing rejected plans with race conditions.

This prompt is designed to sit between the agent's planning step and its tool execution layer. It receives a task description and a list of available tool signatures, then outputs a dependency graph and parallel execution groups. The harness must parse this structured output and feed it into an executor that respects the declared dependencies. Do not treat this prompt as a black box that directly invokes tools—its job is to produce a plan, and the harness is responsible for enforcing that plan's constraints at runtime.

The integration flow should follow a strict sequence: (1) assemble the prompt with the current task, available tool list, and few-shot examples; (2) call the model with a low temperature (0.0–0.2) and a structured output mode targeting your dependency graph schema; (3) validate the output against a schema that requires each tool call to declare its dependencies as an array of tool call IDs, and reject any plan where a dependency references a non-existent ID or creates a cycle; (4) execute the plan using a directed acyclic graph (DAG) executor that runs independent groups in parallel and serializes dependent calls; (5) if any tool call fails, re-invoke the planning prompt with the failure context and remaining task to generate a revised plan. Log every plan, its validation result, and any runtime dependency violations for debugging false dependencies or missed parallelization opportunities.

For production deployments, add a parallelism cap in the executor (e.g., max 5 concurrent calls) to avoid overwhelming downstream APIs, even if the plan suggests more. Implement a race condition detector that flags when two supposedly independent calls write to the same resource or mutate shared state—this is a common failure mode where the prompt's dependency analysis is correct for data flow but wrong for side effects. Include an eval harness that compares the generated plan against a golden set of known parallelizable and dependent tool call scenarios, measuring precision and recall on dependency edge detection. When the prompt misses a parallelization opportunity, log the case for few-shot example updates. When it creates a false dependency, trace whether the model confused data ordering with logical dependency. Finally, gate any plan that proposes more than [MAX_TOOL_CALLS] total invocations and route it to a human reviewer or a decomposition prompt to split the task further.