Duplicate Work Detection Prompt During Task Assignment

This prompt is designed for orchestration engineers and platform operators who need to prevent duplicate work across a multi-agent system. Before dispatching a new task to an agent, this prompt compares the proposed task against a registry of tasks that are already assigned, in-progress, or recently completed. It produces a structured deduplication report with overlap scores, conflict descriptions, and a clear recommendation to proceed, merge, or cancel. Use this when your agent pool has no built-in deduplication, when tasks arrive from multiple upstream sources, or when the cost of duplicate execution is high.

The primary job-to-be-done is semantic overlap detection—catching situations where two tasks are phrased differently but target the same objective. For example, 'Analyze Q3 churn drivers in the enterprise segment' and 'Investigate why large customers left in the last quarter' describe the same work. A simple ID-based deduplication system will miss this. This prompt is most valuable when tasks originate from natural-language inputs, such as user requests, meeting notes, or cross-team tickets, where phrasing varies. It is also critical when duplicate execution carries a high cost: wasted GPU compute, conflicting side effects on shared systems, or customer-facing inconsistencies when two agents produce different answers to the same underlying question.

Do not use this prompt as a replacement for deterministic ID-based deduplication. If your system already assigns unique task IDs, request IDs, or idempotency keys, rely on those first. This prompt handles the fuzzy boundary where semantic similarity matters. Also avoid using it when tasks are intentionally parallel—such as running the same analysis across different time windows or customer segments—where the prompt might incorrectly flag legitimate work as duplicates. In those cases, include explicit differentiation fields in the task schema to prevent false positives. Finally, this prompt is not a real-time guard; it adds latency and should be placed at the task intake gate, not in the hot path of agent execution.

Before integrating this prompt, ensure you have a task registry that the prompt can query. The registry must include task descriptions, statuses, assigned agents, and timestamps. Without a current and complete registry, the deduplication report will be unreliable. If your registry is incomplete or stale, invest in registry hygiene first. Once the prompt is in place, pair it with a human review step for cases where the overlap score falls in an ambiguous range (e.g., 0.4–0.7), where the system recommends a merge but the context is unclear. This prevents both missed duplicates and incorrectly blocked work.

This prompt template is designed to be wired into your task orchestration pipeline. Before a new task is dispatched to an agent, the orchestrator calls this prompt with the proposed task and a snapshot of the current task registry. The model acts as a deduplication analyst, comparing the new task against all active, pending, and recently completed work to identify overlaps that could waste compute or create conflicting side effects. The output is a machine-readable JSON report that your orchestrator can use to decide whether to proceed, merge, or cancel the assignment.

Below is the copy-ready template. Replace every square-bracket placeholder with data from your task registry and the proposed task before sending it to the model. The [TASK_REGISTRY] should be a JSON array of existing tasks, each with an id, description, status, assigned_agent, and output_summary (if completed). The [PROPOSED_TASK] should be a JSON object with the new task's description, intended_agent, priority, and input_data schema. The [OVERLAP_THRESHOLD] is a float between 0.0 and 1.0 that sets the minimum similarity score for flagging a potential duplicate. The [FALSE_POSITIVE_CHECKS] placeholder lets you inject domain-specific rules, such as "tasks with different target environments are never duplicates."

textCopy
You are a task deduplication analyst for a multi-agent orchestration system. Your job is to compare a newly proposed task against a registry of existing tasks and determine whether the new task duplicates or overlaps with work already assigned, in-progress, or recently completed.

## INPUT

### Task Registry
[TASK_REGISTRY]

### Proposed Task
[PROPOSED_TASK]

### Overlap Threshold
[OVERLAP_THRESHOLD]

### False-Positive Checks
[FALSE_POSITIVE_CHECKS]

## INSTRUCTIONS

1. Parse the proposed task and extract its core objective, required inputs, expected outputs, and target domain.
2. For each task in the registry, compute a semantic overlap score (0.0 to 1.0) based on shared objectives, input data overlap, output similarity, and domain intersection.
3. Apply the false-positive checks to eliminate matches that are superficially similar but legitimately distinct.
4. For any task with an overlap score above the threshold, describe the nature of the conflict and recommend one of: "merge", "cancel_proposed", "cancel_existing", or "proceed_parallel".
5. If no overlaps exceed the threshold, recommend "proceed".

## OUTPUT SCHEMA

Return a valid JSON object with this exact structure:
{
  "proposed_task_id": "string",
  "recommendation": "proceed" | "merge" | "cancel_proposed" | "cancel_existing" | "proceed_parallel",
  "overlaps": [
    {
      "existing_task_id": "string",
      "overlap_score": 0.0,
      "conflict_description": "string",
      "recommendation": "merge" | "cancel_proposed" | "cancel_existing" | "proceed_parallel",
      "rationale": "string"
    }
  ],
  "false_positives_checked": ["string"],
  "confidence": 0.0
}

## CONSTRAINTS
- Do not flag tasks as duplicates solely because they share keywords. Require substantive objective overlap.
- If the proposed task has a different intended agent but the same objective, flag it as a duplicate.
- If an existing task is marked "completed" but the proposed task would produce a conflicting side effect, flag it.
- If the overlap score is below the threshold, do not include it in the overlaps array.
- Always apply every false-positive check before finalizing the recommendation.

After pasting this template into your orchestration code, validate the model's output against the JSON schema before acting on the recommendation. For high-risk workflows—such as tasks that modify production infrastructure, update customer records, or trigger financial transactions—route any overlap_score above 0.6 to a human reviewer with the full deduplication report. Log every deduplication decision with the model's confidence score, the overlap details, and the final dispatch action. This log becomes your audit trail for debugging duplicate-work incidents and tuning the overlap threshold over time. Avoid the temptation to skip deduplication for low-priority tasks; duplicate side effects at any priority level can corrupt shared state and create reconciliation debt that is far more expensive than a single model inference call.