User Feedback Sentiment Alert Prompt Template

This prompt is designed for product managers and AI operations engineers who need to correlate unstructured user feedback signals with structured trace-level behavior anomalies. The core job-to-be-done is automating the detection of sentiment trend shifts, session abandonment correlations, and root-cause mapping between negative feedback and specific trace events. Use this prompt when you have a feedback ingestion pipeline that can supply raw feedback text and trace identifiers, and you need a repeatable, automated method for generating actionable alerts rather than manually reviewing dashboards. The ideal user has access to both a feedback database and an observability platform that exports session trace summaries, and they need to close the loop between what users report and what the system actually did.

This prompt is not a replacement for a full observability platform or a human review process for critical incidents. It is the decision layer that sits between your feedback pipeline and your alerting system. Do not use this prompt when feedback volume is extremely low (fewer than 20 records per evaluation window), when trace data is incomplete or missing span-level detail, or when the feedback text is in multiple unsupported languages without translation preprocessing. The prompt also should not be used for real-time intervention on individual sessions—it is designed for batch analysis over evaluation windows (e.g., hourly or daily batches). For real-time single-session diagnosis, use a dedicated session replay and trace review prompt instead.

Before deploying this prompt, validate that your feedback pipeline can supply the required inputs: a batch of feedback records with text, session identifiers, and timestamps, plus corresponding trace summaries that include span events, tool calls, latency data, and error flags. The output is a structured alert payload with sentiment trend classification, abandonment correlation scores, and explicit feedback-to-trace event mappings. If your trace data lacks event-level granularity, the prompt's root-cause mapping capability will degrade significantly. Start with a dry-run evaluation against a labeled dataset of known incidents to calibrate alert thresholds before routing alerts to an on-call rotation or incident management system.

The prompt below is designed to be dropped into your feedback analysis pipeline. It expects two structured inputs—a batch of recent user feedback and corresponding trace summaries—and produces a single JSON alert object. The template uses square-bracket placeholders for all variable components, including the input data, output schema, constraints, and risk level. Before using it in production, you must replace each placeholder with the actual data or configuration your system provides at runtime.

textCopy
You are an AI observability analyst. Your task is to analyze the provided user feedback batch and trace summaries, identify sentiment trend shifts, correlate negative feedback with trace-level anomalies, and generate a structured alert payload.

## INPUT DATA

### User Feedback Batch
[FEEDBACK_BATCH]

### Trace Summaries
[TRACE_SUMMARIES]

## OUTPUT SCHEMA
Return only a valid JSON object conforming to this schema:
{
  "alert_generated": boolean,
  "alert_level": "critical" | "warning" | "info" | "none",
  "sentiment_summary": {
    "overall_trend": "improving" | "stable" | "declining" | "volatile",
    "trend_confidence": number (0.0 to 1.0),
    "key_themes": [string, ...],
    "shift_detected": boolean,
    "shift_description": string | null
  },
  "correlated_anomalies": [
    {
      "feedback_theme": string,
      "trace_anomaly_type": "latency_spike" | "error_rate" | "tool_failure" | "hallucination" | "refusal" | "format_drift" | "other",
      "affected_sessions": [string, ...],
      "correlation_strength": "strong" | "moderate" | "weak",
      "evidence_summary": string
    }
  ],
  "recommended_actions": [string, ...],
  "requires_human_review": boolean
}

## CONSTRAINTS
[CONSTRAINTS]

## RISK LEVEL
[RISK_LEVEL]

## INSTRUCTIONS
1. Compare the current feedback batch against any historical trend context provided in [FEEDBACK_BATCH].
2. For each negative theme in the feedback, search [TRACE_SUMMARIES] for matching session IDs or time windows.
3. Only report correlations where the trace anomaly plausibly explains the user's reported issue.
4. If [RISK_LEVEL] is "high", set `requires_human_review` to true for any `alert_level` of "warning" or above.
5. Do not fabricate correlations. If no trace anomaly matches a complaint, do not force a match.
6. Return only the valid JSON object. Do not include any explanatory text outside the JSON.

To adapt this template, replace [FEEDBACK_BATCH] with a structured array of feedback records, each containing at minimum a session ID, timestamp, sentiment score, and verbatim comment. Replace [TRACE_SUMMARIES] with an array of trace summaries keyed by session ID, including span-level metrics like latency, error counts, and tool-call outcomes. The [CONSTRAINTS] placeholder should be replaced with any domain-specific rules, such as "ignore feedback from internal test accounts" or "only alert when at least 5 sessions exhibit the same anomaly." The [RISK_LEVEL] placeholder accepts "low," "medium," or "high" and gates the human-review flag accordingly.

Before deploying, validate that your feedback ingestion pipeline consistently populates the required fields. A common failure mode is missing session IDs in the feedback batch, which breaks the correlation step. Add a pre-processing check that rejects or quarantines feedback records without traceable session identifiers. For high-risk deployments, pair this prompt with an eval that compares the generated alert against a manually reviewed sample to measure precision and recall of the correlation logic.

This prompt is designed to run as a scheduled or event-driven job inside your feedback ingestion pipeline, not as a one-off chat interaction. When a new batch of user feedback arrives—typically from in-app surveys, support ticket CSAT scores, or NPS responses—the pipeline should enrich each feedback record with its trace session ID before invoking the model. The prompt expects a structured [FEEDBACK_BATCH] containing individual feedback items, each with a session_id, rating, text, and timestamp. Without the session ID, you lose the ability to correlate sentiment anomalies with the specific trace events that caused them, which is the primary value of this playbook.

Wire the prompt into an application worker that performs the following steps: (1) Fetch unprocessed feedback from your data warehouse or queue where session_id IS NOT NULL. (2) Assemble the [FEEDBACK_BATCH] JSON array, capping batch size at 50 items to stay within context window limits and keep latency predictable. (3) Call the model with the prompt template, setting temperature=0 for consistent alert classification and response_format to the defined JSON schema. (4) Validate the output schema strictly—reject any response missing alerts, alert_level, or correlated_sessions fields. (5) On validation failure, retry once with the same batch and an added [CONSTRAINT] instructing the model to strictly adhere to the output schema. If the retry also fails, log the raw response and escalate to a human reviewer rather than silently dropping the batch.

For high-severity alerts (alert_level: 'critical'), the harness should trigger an immediate notification to the on-call channel with the correlated session IDs and a link to the trace viewer, not wait for a dashboard refresh. For warning and info alerts, write the structured output to your observability platform as log events with dimensions for alert_type, alert_level, and affected_feature so they can be aggregated into dashboards. Avoid wiring this prompt directly to an automated rollback or model-switching action—sentiment alerts are correlational signals, not root-cause proofs. A human should confirm that the trace behavior anomaly matches the sentiment signal before taking remediation steps. The harness should also track the prompt version in the log metadata so that alert pattern changes can be traced back to prompt modifications during incident reviews.