Duplicate Agent Trace Correlation Prompt

This prompt is for observability engineers and platform operators who need to identify where multiple agents performed overlapping or identical work during a multi-agent run. The job-to-be-done is correlating execution traces from different agents by matching task fingerprints, timing windows, and resource access patterns to surface duplicate effort. You should use this prompt when you have raw agent trace logs—containing tool calls, state mutations, or retrieved context—and you need a structured report of which traces represent redundant work, why they match, and how much compute was wasted.

The ideal user has access to structured trace data including agent identifiers, timestamps, action descriptions, resource targets, and task fingerprints. Required context includes a trace window (start and end time), a similarity threshold for fingerprint matching, and a list of resources or tools that define a collision domain. Do not use this prompt for real-time duplicate prevention during agent execution—it is designed for post-hoc analysis and audit. It is also not a replacement for idempotency key enforcement or pre-execution deduplication checks; use those patterns to block duplicates before they happen, and use this prompt to find the duplicates that slipped through.

The prompt expects structured trace inputs and produces a correlation report with matched trace pairs, confidence scores, evidence for each match, and a summary of estimated duplicate compute cost. Before relying on the output, validate that the correlation recall meets your threshold by testing against a labeled dataset of known duplicate and non-duplicate trace pairs. For high-stakes audits where duplicate work could indicate a systemic orchestration bug, route the correlation report for human review before taking action on the findings. Avoid running this prompt on traces that lack consistent fingerprinting—if your agents don't produce comparable task fingerprints, the correlation quality will degrade to near-random.

This template is the core instruction set for the Duplicate Agent Trace Correlation Prompt. It is designed to be copied directly into your observability or orchestration harness. The prompt instructs a model to act as a trace correlation engine, comparing structured execution traces from multiple agents to find overlapping work. It uses square-bracket placeholders for all dynamic inputs, allowing you to swap in traces, define your output schema, and set constraints without rewriting the core logic.

textCopy
You are an agent trace correlation engine. Your task is to analyze execution traces from multiple agents and identify instances where two or more agents performed duplicate or substantially overlapping work.

## INPUT TRACES
Below are the execution traces to analyze. Each trace is a JSON object containing an agent identifier, a task fingerprint, a timestamp, and a list of actions or tool calls.

[TRACES]

## CORRELATION CRITERIA
Two traces are considered duplicates if they meet ALL of the following conditions:
1. **Task Fingerprint Match**: The semantic intent and primary entities of the tasks are identical or nearly identical, as defined by the `task_fingerprint` field in each trace.
2. **Temporal Overlap**: The execution windows (start to end time) overlap by at least [TEMPORAL_OVERLAP_THRESHOLD]%.
3. **Resource Intersection**: The traces reference at least one common resource, tool, or target object (e.g., same database record, same document ID, same API endpoint with identical parameters).

## OUTPUT SCHEMA
Return a single JSON object conforming to this exact schema. Do not include any text outside the JSON object.

[OUTPUT_SCHEMA]

## CONSTRAINTS
[CONSTRAINTS]

## EXAMPLES
[EXAMPLES]

Proceed with the analysis.

To adapt this template, replace the placeholders with your specific data and requirements. The [TRACES] placeholder should be populated with a JSON array of your agent execution traces. The [OUTPUT_SCHEMA] must be a strict JSON Schema definition that the model's output will be validated against; for example, an array of duplicate groups, each containing the correlated trace IDs and a confidence score. The [CONSTRAINTS] block is where you enforce rules like 'ignore traces from the same agent' or 'only flag duplicates with a confidence score above 0.8'. The [EXAMPLES] block should contain one or two few-shot examples of input traces and the correct, correlated output to guide the model's behavior. After copying the template, the next critical step is to build a validation harness around it to ensure the output is parseable and matches your schema before it is used to trigger alerts or deduplication logic.

This prompt is designed to be called as part of a post-execution analysis pipeline, not in the hot path of agent orchestration. It should consume structured trace logs from your agent execution framework—typically JSONL or structured log events containing agent IDs, timestamps, tool calls, state mutations, and task fingerprints. The prompt expects batched trace windows (e.g., all agent traces from a single session or a fixed time window) and returns a correlation report. Wire this into your observability stack as an asynchronous analysis job triggered by session completion events or scheduled batch windows.

Validation and retry logic is critical because the output schema is strict. Implement a JSON schema validator that checks for required fields (correlated_groups, uncorrelated_traces, correlation_confidence, evidence_links) before accepting the model's output. If validation fails, retry once with the validation error message appended to the prompt as a [PREVIOUS_ERROR] context block. After two failures, log the raw output and escalate for human review. For high-stakes debugging (e.g., production incidents where duplicate agent work caused data corruption), always route the correlation report through a human reviewer before taking corrective action on agent configurations.

Model selection matters for this task. Use a model with strong reasoning capabilities and a large context window (200K+ tokens) because trace correlation requires holding many agent execution paths in memory simultaneously. Claude 3.5 Sonnet or GPT-4o are good defaults. Avoid smaller or faster models for this prompt—the cost of missed duplicates (wasted compute, reconciliation debt) far exceeds the inference cost of a capable model. If your trace volumes exceed context limits, pre-filter traces by task fingerprint similarity before sending to the model, and note in the [CONTEXT] block which traces were excluded and why.

Observability integration should log every correlation run: input trace count, output duplicate groups found, validation pass/fail, retry count, and latency. Emit these as structured metrics to your monitoring system. Set alerts on sudden increases in duplicate detection rate (may indicate a routing bug) or correlation failures (may indicate schema drift in agent traces). Store correlation reports alongside the original traces for auditability. When the model identifies duplicate work, the report's evidence_links field should reference specific trace line numbers or span IDs, making it traceable back to the raw logs for verification.

Avoid running this prompt in real-time on every agent action—the latency and cost are inappropriate for hot-path deduplication. Instead, use the Pre-Execution Deduplication Check Prompt for inline guardrails and reserve this trace correlation prompt for debugging, audit, and optimization workflows. If you need continuous duplicate monitoring, batch traces in 5-15 minute windows and run correlation as a scheduled job. Never use the correlation output to automatically modify agent behavior without human review of at least the first N runs to establish trust in the correlation accuracy.