Duplicate Webhook Delivery Handling Prompt

This prompt is designed for integration engineers who need to handle at-least-once webhook delivery semantics reliably. The core job is to analyze an incoming webhook payload against a history of previously processed events and produce a deterministic decision: process the event as new, skip it as a duplicate, or escalate it for human review when the evidence is ambiguous. You should use this prompt when your webhook consumer cannot rely solely on idempotency keys provided by the sender, or when you need a secondary deduplication layer that fingerprints the payload content itself to catch replay attacks, retries with lost headers, or events from systems that don't support idempotency keys at all.

The ideal user is a backend or platform engineer embedding this prompt into a webhook ingestion pipeline. Required context includes the current payload, a set of recently processed event fingerprints (with timestamps), and a configurable similarity threshold. The prompt is not a replacement for database-level unique constraints or idempotency-key storage—it is a decision-support layer that handles edge cases those simpler mechanisms miss, such as near-duplicate payloads with minor timestamp drift, reordered delivery where a duplicate arrives before the original is fully committed, or partial payload changes that suggest a legitimate update rather than a retry. Do not use this prompt for real-time financial settlement or safety-critical actuation where deterministic, auditable logic is required without model inference.

Before wiring this prompt into production, define your fingerprinting strategy: which fields contribute to the fingerprint, whether you normalize timestamps, and how you handle nested objects. The prompt works best when paired with a fast pre-filter that eliminates obvious non-duplicates before invoking the model. After the model returns a decision, your application layer must enforce the action—the prompt advises, but your code commits the idempotent write. For high-throughput systems, batch recent fingerprints into the context window and set a strict time-bound on the lookback window to keep latency predictable.

The prompt below is designed to be dropped into an idempotency decision harness. It receives a candidate webhook payload alongside a set of previously processed payload fingerprints and their outcomes. The model's job is not to execute the webhook logic but to produce a structured decision—process, skip, or review—along with a fingerprint and a concise rationale. This separation keeps the AI's role narrow and testable: it classifies duplication risk while your application code enforces the decision and handles side effects.

textCopy
You are an idempotency decision engine for a webhook ingestion pipeline. Your task is to determine whether an incoming webhook delivery is a duplicate of a previously processed event.

## INPUT
- Candidate payload: [CANDIDATE_PAYLOAD]
- Previously processed fingerprints and outcomes: [PROCESSED_FINGERPRINTS]

## FINGERPRINTING RULES
1. Generate a fingerprint from the candidate payload using these fields in order of priority: [IDEMPOTENCY_KEY_FIELD], [EVENT_TYPE_FIELD], [TIMESTAMP_FIELD], [RESOURCE_ID_FIELD], [PAYLOAD_HASH_FIELDS].
2. If [IDEMPOTENCY_KEY_FIELD] is present, use it as the primary fingerprint component.
3. If [IDEMPOTENCY_KEY_FIELD] is absent, construct the fingerprint from the remaining fields concatenated with a pipe delimiter.
4. Normalize timestamps to UTC ISO 8601 before fingerprinting.
5. Ignore fields listed in [IGNORE_FIELDS] when constructing the fingerprint.

## DECISION RULES
- Return "process" if the fingerprint does not match any previously processed fingerprint.
- Return "skip" if the fingerprint matches a previously processed fingerprint AND the prior outcome was "processed" or "skipped".
- Return "review" if the fingerprint matches a previously processed fingerprint AND the prior outcome was "review" or "failed", OR if the candidate payload contains fields that differ materially from the matched prior payload beyond [IGNORE_FIELDS].
- Return "review" if the candidate payload timestamp is more than [MAX_CLOCK_SKEW_SECONDS] seconds behind the current time or ahead by more than [MAX_FUTURE_TOLERANCE_SECONDS] seconds.

## OUTPUT_SCHEMA
Return a valid JSON object with exactly these fields:
{
  "decision": "process" | "skip" | "review",
  "fingerprint": "string",
  "matched_fingerprint": "string | null",
  "rationale": "string (max 200 characters)",
  "confidence": "high" | "medium" | "low"
}

## CONSTRAINTS
- Do not invent payload fields. Use only fields present in the candidate payload.
- If the payload is malformed or missing required fingerprinting fields, set decision to "review" and explain why in the rationale.
- Do not execute any side effects. Only return the decision JSON.
- If multiple prior fingerprints match, reference the most recent one in matched_fingerprint.

## EXAMPLES
[EXAMPLES]

## RISK_LEVEL
[RISK_LEVEL]

To adapt this template, replace the square-bracket placeholders with your webhook provider's actual field names and operational thresholds. [IDEMPOTENCY_KEY_FIELD] should point to the provider's native idempotency key if one exists (e.g., stripe.event.id or shopify.webhook.id). [PAYLOAD_HASH_FIELDS] defines which payload fields contribute to the fingerprint when no explicit key is present—choose fields that uniquely identify the business event, such as order_id plus event_type. [IGNORE_FIELDS] is critical for avoiding false duplicates caused by fields that change on every delivery, such as delivery_attempt, signature, or webhook_id. [MAX_CLOCK_SKEW_SECONDS] and [MAX_FUTURE_TOLERANCE_SECONDS] should reflect your system's tolerance for clock drift; 300 seconds (5 minutes) is a common starting point. [EXAMPLES] should include at least three few-shot demonstrations: a clear duplicate with matching idempotency key, a new event with no prior fingerprint match, and a timestamp-drift case that triggers review. [RISK_LEVEL] accepts low, medium, or high and should be set based on the downstream cost of duplicate processing—payment events warrant high and stricter review thresholds.

Before wiring this prompt into production, validate that the model consistently returns parseable JSON matching the output schema. Run it against a golden test set that includes exact duplicates, near-duplicates with ignored-field changes, reordered deliveries, and payloads with missing fingerprinting fields. If the model returns review for more than 5% of non-duplicate events in shadow mode, tighten the fingerprinting rules or expand [IGNORE_FIELDS]. For high-risk use cases, route all review decisions to a human operator or a dead-letter queue rather than auto-processing. Never use this prompt to make the final processing decision without application-layer enforcement of the returned decision.

The duplicate webhook delivery prompt is designed to sit inside a webhook ingestion handler, not as a standalone decision point. When a webhook arrives, the handler must first extract the raw payload, headers, and delivery timestamp before calling the model. The prompt expects a structured input containing the current payload, a set of recently processed payload fingerprints, and the idempotency window configuration. Because webhook providers often retry deliveries with slight payload variations (reordered fields, updated timestamps, or appended metadata), the handler should normalize the payload before fingerprinting—strip provider-specific envelope fields, sort JSON keys, and remove non-deterministic values like X-Request-Id headers that change across retries. The model's decision (process, skip, or quarantine) must be treated as advisory; the handler should enforce the final idempotency check against a durable state store, not rely solely on the model's classification.

Wire the prompt into a pipeline with three stages: pre-processing, model inference, and post-decision enforcement. In pre-processing, compute a content fingerprint (SHA-256 of the normalized payload) and a semantic fingerprint (key field hash, such as order_id + event_type). Query your state store for recent deliveries matching either fingerprint within the idempotency window. Pass the current payload, the list of candidate duplicates with their timestamps and fingerprint match types, and the window configuration into the prompt's [INPUT] placeholder. For model inference, use a fast, cost-effective model (GPT-4o-mini or Claude Haiku) with temperature=0 and response_format set to the JSON schema defined in the prompt template. Set a strict timeout of 2–3 seconds; if the model call fails or times out, fall back to a deterministic rule: skip if an exact content-fingerprint match exists within the window, otherwise process. Log every model decision alongside the fingerprint evidence for auditability.

Post-decision enforcement requires atomic state updates to prevent race conditions. When the model returns process, write the fingerprint to the state store using a conditional insert (e.g., DynamoDB attribute_not_exists or Redis SETNX) before executing the business logic. If the conditional write fails, another concurrent handler already processed a duplicate—log the collision and skip. When the model returns skip, verify that a matching fingerprint actually exists in the state store; if not, the model may have hallucinated a duplicate and you should escalate to quarantine. When the model returns quarantine, store the payload in a dead-letter queue with the model's reasoning and alert an on-call engineer. Implement a retry budget for model call failures: three attempts with exponential backoff (1s, 2s, 4s), then fall back to the deterministic rule. Never retry a quarantine decision—human review is required before reprocessing.

Validation and eval checks should run continuously against this pipeline. Instrument the handler to emit metrics: model latency, decision distribution (process/skip/quarantine), fallback rate, and fingerprint collision rate. Build a regression test suite with known duplicate scenarios: exact payload retry, reordered fields, timestamp drift within the window, timestamp drift outside the window, partial payload changes (e.g., updated status field on a retry), and concurrent deliveries with sub-second arrival gaps. For each test case, assert the final processing decision (not just the model's output) and verify that no duplicate side effects occur. Run these tests in CI on every prompt change. In production, sample 5% of skip decisions and replay the original payload through a shadow pipeline to confirm idempotency holds. If the shadow pipeline produces a different decision, flag the trace for review and consider tightening the fingerprinting strategy or adjusting the idempotency window.