Dead Letter Queue Message Replay Decision Prompt

This prompt is designed for distributed systems engineers and SRE teams who need to automate the triage of messages that have landed in a dead letter queue (DLQ) after exhausting all configured retries. The core job-to-be-done is to replace manual, repetitive inspection of failed messages with a structured, consistent, and auditable decision process. The ideal user is a platform engineer building an automated recovery harness who needs the model to analyze a message's payload, its error context, and the system's current state to produce a clear, actionable classification: replay this message to its original destination, route it to a fallback handler, or permanently discard it as a poison message. This prompt is appropriate when you have a backlog of DLQ messages, need to enforce a consistent root cause taxonomy, and want to safely automate the routing of replayable messages back into processing pipelines.

You should use this prompt only when the necessary context has already been extracted and provided to the model. The prompt assumes you are feeding it a pre-fetched message payload, structured error metadata (such as the exception type, stack trace, and retry count), and relevant system state (like downstream service health or circuit breaker status). It is not designed for real-time stream processing or for making decisions without this context. Do not use this prompt when the failure mode involves a fundamental data corruption that requires upstream producer fixes, when the message payload contains sensitive data that cannot be sent to an LLM, or when the replay action itself is unsafe due to a lack of idempotency guarantees in the target handler. The prompt's value is in standardizing the triage logic, not in executing the replay, which must be handled by your application harness.

Before integrating this prompt, you must establish clear operational boundaries. The LLM's output is a decision recommendation, not an executed action. Your harness should parse the structured output, validate the decision against a set of hardcoded safety rules (e.g., never replay a message that has already been replayed more than a configured maximum), and log the decision for auditability. For high-risk systems, implement a human-in-the-loop review step for any decision classified as UNCERTAIN or for messages above a certain value threshold. The next step after reading this section is to review the prompt template, adapt the placeholders to your specific error taxonomy and system architecture, and design the validation harness that will act on the model's structured output.

The prompt below is designed to be pasted directly into your DLQ analysis harness. It instructs the model to act as a distributed systems reliability engineer, analyzing a message that has exhausted its retry budget. The model must produce a structured decision, not just a summary. All dynamic values are represented as square-bracket placeholders, such as [MESSAGE_PAYLOAD], [ERROR_CONTEXT], and [QUEUE_CONFIG]. Replace these with actual data from your message broker and monitoring systems before sending the prompt.

textCopy
You are a distributed systems reliability engineer analyzing a message that has been moved to a dead letter queue (DLQ). Your task is to determine whether the message should be replayed or discarded, and to provide a detailed justification.

## Input Data

**Message Payload:**
```json
[MESSAGE_PAYLOAD]

Error Context:

• Original Queue: [ORIGINAL_QUEUE_NAME]
• Error Message: [ERROR_MESSAGE]
• Error Code/Type: [ERROR_TYPE]
• Retry Attempts Exhausted: [RETRY_COUNT]
• Timestamp of Last Failure: [LAST_FAILURE_TIMESTAMP]

System Context:

• Consumer Group ID: [CONSUMER_GROUP_ID]
• Downstream Service Health Status: [DOWNSTREAM_SERVICE_HEALTH]
• Recent Deployment Events: [RECENT_DEPLOYMENT_EVENTS]
• Message Ordering Requirements: [ORDERING_REQUIREMENTS]

Decision Criteria

1. Poison Message Detection: Is the message itself malformed, containing invalid data that will never be processable (e.g., a missing required field, a foreign key that will never exist)?
2. Transient vs. Persistent Error: Is the error likely temporary (e.g., a network timeout, a brief service overload) or permanent (e.g., an unrecoverable data corruption, a business rule violation)?
3. Downstream Impact: What is the blast radius of replaying this message? Could it cause duplicate processing, violate ordering guarantees, or overload a struggling downstream service?
4. Business Criticality: What is the importance of the data in this message? Can it be safely discarded, or does it represent a critical business event that must be processed?

Output Schema

Respond with a single, valid JSON object conforming to this schema:

jsonCopy
{
  "decision": "REPLAY | DISCARD | MANUAL_REVIEW",
  "root_cause_classification": "POISON_MESSAGE | TRANSIENT_ERROR | PERSISTENT_ERROR | DEPENDENCY_FAILURE | UNKNOWN",
  "confidence_score": 0.0-1.0,
  "justification": "A concise, evidence-based explanation for the decision.",
  "replay_instructions": {
    "destination_queue": "[ORIGINAL_QUEUE_NAME | A_DIFFERENT_QUEUE_NAME]",
    "required_preconditions": ["A list of conditions that must be met before replaying, e.g., 'Deploy fix v1.2.3', 'Confirm downstream service health'"]
  },
  "discard_impact": "A brief statement of what data or business event will be lost if discarded.",
  "manual_review_reason": "If decision is MANUAL_REVIEW, explain why automated resolution is unsafe."
}

Constraints

• Base your decision strictly on the provided Input Data. Do not invent external facts.
• If the error context indicates a 4xx client error (excluding 429), strongly consider POISON_MESSAGE.
• If the error context indicates a 5xx server error or a 429 rate limit, strongly consider TRANSIENT_ERROR.
• If ORDERING_REQUIREMENTS is true, your replay instructions must address ordering guarantees.
• If confidence_score is below 0.8, the decision must be MANUAL_REVIEW.

To adapt this template, start by mapping your DLQ metadata fields to the placeholders. The [MESSAGE_PAYLOAD] should be the full message body, serialized as a JSON string. The [ERROR_CONTEXT] fields should be pulled from your message broker's headers or your application's error logs. The most critical adaptation is the Decision Criteria section; you should update this to reflect your organization's specific operational policies, such as known poison message signatures or business-critical event types. After adapting, test the prompt against a golden dataset of 20-30 historical DLQ messages with known correct decisions to calibrate the confidence_score threshold and ensure the model's reasoning aligns with your team's judgment.

This prompt is designed to be the decision engine within a dead letter queue (DLQ) replay harness, not a standalone chatbot. The application layer is responsible for fetching the message payload, metadata, and relevant system state, then assembling the prompt with these concrete values. The model's structured output—a replay-or-discard decision with root cause classification—must be parsed and enforced by the harness. Never allow the raw model output to directly trigger a replay action without validation against the system's own rules for ordering guarantees, idempotency, and poison message detection.

The implementation loop should follow a strict sequence: (1) Dequeue a batch of messages from the DLQ. (2) For each message, extract the message_body, error_stack, retry_count, queue_name, and timestamp. (3) Populate the prompt's [MESSAGE_PAYLOAD], [ERROR_CONTEXT], [RETRY_METADATA], and [QUEUE_CONTEXT] placeholders. (4) Call the model with temperature=0 and a structured output mode (e.g., JSON mode or function calling) to enforce the [OUTPUT_SCHEMA]. (5) Parse the decision field. If replay, validate that the destination_queue is not the original DLQ source and that the replay_priority is within acceptable bounds. If discard, log the discard_reason and root_cause_classification to your observability platform before archiving or permanently deleting the message. (6) Implement a hard circuit breaker: if the same message ID appears in the DLQ more than a configured threshold (e.g., 3 times), bypass the model and force a discard with a poison_message classification to prevent infinite replay loops.

For high-risk systems where message ordering or exactly-once processing is critical, insert a human approval step before replaying any message classified as ordering_risk or duplicate_potential. The harness should route these decisions to a review queue with the full prompt context and model reasoning attached. Additionally, instrument the harness with metrics: dlq_replay_decision_total (by decision type), dlq_model_latency_seconds, and dlq_circuit_breaker_activated_total. These metrics will surface when the model is adding latency, making poor decisions, or when the system is experiencing a poison message storm. Use a lightweight model (e.g., a fast instructor-tuned variant) to keep replay latency low; this is an operational decision, not a deep analysis task.