Task Granularity Optimization Prompt Template

This prompt is for agent framework developers and orchestration engineers who need to decide whether a complex task should be split into smaller sub-tasks or executed as a single unit. The core job-to-be-done is balancing the overhead of decomposition and agent handoffs against the benefits of parallel execution and specialized agent assignment. Use this prompt when you have a task that could plausibly be broken down, and you need a structured recommendation that accounts for context budget, latency, compute cost, and the risk of over-decomposition. The ideal user has a working understanding of their agent pool's capabilities, the approximate token costs of their models, and the latency tolerance of their application.

The prompt requires several concrete inputs to produce a useful recommendation. You must provide the full task description, the available agent pool with their capabilities and costs, the total context window budget, and the target latency or cost constraints. Without these, the model will produce generic advice that cannot be actioned in a production system. The prompt is designed to output a structured granularity plan with a split-or-keep decision, a breakdown of sub-tasks if splitting is recommended, cost and latency estimates for each option, and explicit flags for over-decomposition and under-decomposition risks. This output should be validated against actual execution metrics over time to calibrate the model's estimates.

Do not use this prompt for trivial tasks where decomposition overhead clearly outweighs any benefit, or for tasks that are already atomic and cannot be meaningfully split. Avoid it when the agent pool is unknown or when cost and latency constraints are undefined, because the recommendation will lack the grounding needed for a production decision. This prompt is also not a substitute for a full orchestration engine; it provides a recommendation, not an execution plan. After receiving the output, you should wire the recommendation into your task dispatch logic, log the decision alongside actual execution outcomes, and periodically review whether the granularity decisions are improving throughput or adding unnecessary handoff complexity.

The following prompt template is designed to be embedded directly into an agent orchestrator or planning node. It forces the model to reason about the trade-off between splitting a task further (parallelism, specialization) and keeping it whole (lower overhead, fewer handoffs). The template uses square-bracket placeholders for all dynamic inputs, making it safe to inject into code without accidental template resolution. Before using this prompt, ensure you have defined the task description, available agent capabilities, and the current context budget. The prompt is structured to produce a machine-readable JSON output that can be validated by your orchestration engine before any sub-tasks are dispatched.

textCopy
You are a task granularity optimizer for a multi-agent system. Your job is to analyze a candidate task and recommend whether it should be executed as a single unit or decomposed into smaller sub-tasks. You must balance parallelism benefits against coordination overhead, context transfer costs, and agent capability constraints.

## INPUT

**Task Description:**
[TASK_DESCRIPTION]

**Available Agents and Capabilities:**
[AGENT_CAPABILITIES]

**Current Context Budget Remaining (tokens):**
[CONTEXT_BUDGET]

**Maximum Parallel Sub-Tasks Allowed:**
[MAX_PARALLEL_TASKS]

**Latency Budget (milliseconds):**
[LATENCY_BUDGET]

**Risk Level for Incorrect Output:**
[RISK_LEVEL]

## CONSTRAINTS

[CONSTRAINTS]

## OUTPUT SCHEMA

Return a single JSON object with the following structure:

{
  "recommendation": "keep_whole" | "decompose",
  "confidence": 0.0-1.0,
  "reasoning": "string explaining the trade-off analysis",
  "estimated_cost_current": number,
  "estimated_cost_decomposed": number,
  "estimated_latency_current_ms": number,
  "estimated_latency_decomposed_ms": number,
  "context_budget_consumed_current": number,
  "context_budget_consumed_decomposed": number,
  "sub_tasks": [
    {
      "id": "string",
      "description": "string",
      "assigned_agent": "string",
      "input_contract": {},
      "output_contract": {},
      "dependencies": ["task_id"],
      "estimated_tokens": number,
      "can_parallelize": boolean
    }
  ],
  "risks": ["string describing specific risks"],
  "escalation_triggers": ["condition that should trigger human review"]
}

## RULES

1. If the task is simple, has no internal parallelism, or decomposition overhead exceeds 30% of total cost, recommend "keep_whole".
2. If decomposition enables parallel execution that reduces latency by more than 40%, recommend "decompose".
3. If the context budget is insufficient for all sub-tasks, flag this in risks and recommend "keep_whole" or reduce sub-task count.
4. If [RISK_LEVEL] is "high" or "critical", require at least one escalation trigger.
5. Every sub-task must have a clear input/output contract and an assigned agent from [AGENT_CAPABILITIES].
6. Do not create sub-tasks that have no agent capable of executing them.
7. If confidence is below 0.7, include specific uncertainty markers in the reasoning field.

## EXAMPLES

[EXAMPLES]

Return only the JSON object. Do not include any text outside the JSON.

To adapt this template for your system, replace each square-bracket placeholder with values from your orchestration context. The [AGENT_CAPABILITIES] placeholder should contain a structured description of available agents, their skills, and current load. The [CONSTRAINTS] field is where you inject policy rules such as "never send PII to external agents" or "financial calculations require dual-agent verification." The [EXAMPLES] placeholder is optional but strongly recommended for few-shot conditioning—include at least two examples showing correct decomposition and one showing a correct keep-whole decision. After the model returns the JSON, validate it against the schema before using the sub_tasks array to dispatch work. If the confidence field is below your system's threshold, route the recommendation to a human reviewer or a supervisor agent before execution.

This prompt is designed to sit inside an agent orchestrator's planning phase, not as a standalone chat interface. It should be called before task decomposition is finalized, receiving a candidate task breakdown and returning a granularity assessment. The orchestrator should treat the output as a structured recommendation that can trigger re-decomposition, merging, or further splitting before any sub-tasks are dispatched to worker agents. Because this prompt influences resource allocation and latency, it must execute quickly and deterministically enough to avoid becoming a bottleneck in the orchestration loop.

Wire the prompt into a pre-dispatch validation step. The orchestrator sends a payload containing the proposed sub-tasks, their dependency graph, estimated token budgets, and target latency constraints. The model returns a JSON object with a granularity_score, recommendation (one of split_further, merge, keep_as_is), cost_estimate, latency_estimate, and a risks array. Validate the output schema strictly: reject any response where granularity_score is outside 0.0-1.0, where recommendation is not one of the three allowed values, or where cost_estimate and latency_estimate are missing. On validation failure, retry once with a stricter schema reminder appended to the prompt. If the retry also fails, default to keep_as_is and log the failure for offline analysis. For high-stakes workflows where over-decomposition could cause state corruption or duplicate side effects, route split_further recommendations to a human reviewer with the original task graph and the model's justification before applying the change.

Model choice matters here. Use a model with strong structured output support and low latency—this is a gating decision, not a generative task. GPT-4o or Claude 3.5 Sonnet with JSON mode enabled are good defaults. Avoid models that struggle with numeric estimates or that hallucinate schema fields. Log every granularity decision with the input task graph, the model's recommendation, the validation result, and whether the recommendation was applied or overridden. This log becomes essential for tuning the decomposition strategy over time. The most common production failure is the model recommending splits that create tasks too small to carry enough context, producing agents that fail silently. Monitor for sub-task failure rate spikes after granularity changes and set an alert if the rate exceeds a threshold within a deployment window.