Drift Detection Classification Prompt for Feature Pipelines

This prompt is designed for MLOps engineers and data platform teams who need to classify drift events in feature pipelines before they corrupt downstream model serving. The job-to-be-done is not to detect drift—your monitoring system should already be computing statistical distances—but to triage those raw signals into structured, actionable alerts. Use it when you have a batch of pre-computed distribution comparisons (e.g., PSI, KS statistic, Jensen-Shannon distance) against a validated baseline, alongside schema definitions and configured thresholds. The prompt consumes these summaries and produces a severity-tiered classification, identifies the specific features driving the drift, and recommends concrete investigation or mitigation steps. It acts as a classification and routing layer that sits between your monitoring system and your incident response workflow, turning a noisy stream of metrics into a prioritized queue of alerts that an on-call engineer can act on.

The ideal user is an engineer or platform operator who already has drift metrics flowing but is overwhelmed by false positives or unclear signals. The required context includes: a statistical summary per feature comparing current and baseline distributions, the feature's data type and role (e.g., numerical, categorical, primary key), pre-configured drift thresholds (warning and critical), and any known context about recent pipeline changes or upstream data source modifications. The prompt expects this context to be assembled programmatically before invocation. It does not replace a statistical testing library—you must compute the distances yourself. It does not perform root cause analysis—it classifies the symptom and points to likely investigation paths. It is not suitable for real-time model inference scoring, raw data validation, or as a substitute for a feature store's native monitoring capabilities.

Do not use this prompt when you need sub-second latency on individual records; it is designed for batch triage of aggregated drift summaries, not per-inference drift checks. Do not use it when you lack a validated baseline distribution—the prompt's recommendations are only as good as the baseline it references. Avoid using it for concept drift detection that requires ground-truth labels, as this prompt focuses on data distribution drift and schema drift, not model performance degradation. Finally, do not use this prompt as the sole decision-maker for automated model rollback or pipeline shutdown. Its output should inform a human reviewer or feed into a gated remediation workflow with explicit approval steps, especially in high-stakes or regulated environments where incorrect feature values could cause financial, safety, or compliance harm.

This prompt template is designed to be invoked by your monitoring system whenever a new batch of feature data arrives. It compares the incoming batch against a provided baseline to detect distribution, schema, and concept drift. The output is a structured drift alert that your downstream systems—such as alerting, incident management, or model retraining pipelines—can consume directly. Before using this prompt, ensure your monitoring system has already computed the necessary statistical summaries and that you have a clear definition of acceptable drift thresholds for your specific use case.

textCopy
You are a drift detection classifier for an ML feature pipeline. Your task is to analyze a new batch of feature data against a provided baseline and classify any detected drift.

## INPUT
- New Batch Summary: [NEW_BATCH_SUMMARY]
- Baseline Summary: [BASELINE_SUMMARY]
- Feature List with Types: [FEATURE_LIST]
- Drift Thresholds: [DRIFT_THRESHOLDS]

## CONSTRAINTS
- Classify drift into three categories: 'distribution_drift', 'schema_drift', 'concept_drift', or 'no_drift'.
- For each detected drift, identify the specific affected features.
- Assign a severity level: 'critical', 'warning', or 'info'.
- Provide a concise, evidence-based explanation for each finding.
- Recommend one of the following actions: 'alert', 'quarantine_batch', 'trigger_retraining', 'log_only', or 'no_action'.
- If the input data is malformed or insufficient for a confident assessment, set the overall classification to 'uncertain' and explain why.

## OUTPUT_SCHEMA
{
  "overall_classification": "string",
  "drift_details": [
    {
      "drift_type": "string",
      "affected_features": ["string"],
      "severity": "string",
      "evidence": "string",
      "recommended_action": "string"
    }
  ],
  "explanation": "string"
}

To adapt this template, replace the square-bracket placeholders with data from your feature store and monitoring system. [NEW_BATCH_SUMMARY] should contain statistical profiles (e.g., mean, variance, quantiles) of the incoming data. [BASELINE_SUMMARY] should hold the same statistics from your training or reference window. [DRIFT_THRESHOLDS] are the business rules that define acceptable deviation, such as a maximum Population Stability Index (PSI) or a percentage of null values. For high-stakes pipelines, always route outputs where severity is 'critical' or overall_classification is 'uncertain' to a human for review before any automated action like quarantine or retraining is executed.

This prompt is designed to operate as a batch classification step within a feature pipeline monitoring workflow. It should be invoked after feature computation but before alerting or automated remediation. The harness must supply a structured payload containing the current batch's statistical profile, the baseline reference distribution, and any schema metadata. The model's output is a structured drift classification record that downstream systems can parse to trigger alerts, log diagnostics, or initiate retraining workflows. Do not use this prompt for real-time, per-record decisions; it is calibrated for batch-level analysis where the cost of a false positive is a wasted investigation and the cost of a false negative is undetected model degradation.

Wiring the prompt into your pipeline requires a pre-processing layer that computes distribution summaries (mean, variance, quantile shifts, categorical frequency changes) and a post-processing layer that validates the model's JSON output against a strict schema. The harness should enforce a retry policy with exponential backoff for malformed responses, but cap retries at 3 attempts before falling back to a heuristic drift detector. Log every classification result—including confidence scores, affected features, and recommended actions—to your monitoring system for later calibration. For high-risk pipelines (e.g., financial fraud models, healthcare risk scores), route all severity: "critical" classifications to a human review queue before triggering automated model rollback. Use a drift threshold registry to store and version the baseline expectations and tolerance windows, ensuring the prompt always receives the correct reference data for the current model version.

Validation and calibration are the most critical harness components. Implement a post-inference validator that checks: (1) the output conforms to the expected JSON schema, (2) severity is one of the allowed enum values, (3) affected_features contains only features present in the input schema, and (4) confidence is a float between 0.0 and 1.0. Track the false alarm rate by comparing drift alerts against actual model performance degradation measured on a holdout set or through delayed ground-truth labels. If the false alarm rate exceeds your operational threshold (typically 5-10%), adjust the prompt's [DRIFT_THRESHOLD] parameter upward or recalibrate the baseline statistics. Conversely, if drift goes undetected before performance degrades, lower the threshold or add statistical tests (e.g., Kolmogorov-Smirnov, Chi-squared) to the pre-processing layer to surface subtle shifts the model might miss.

Model choice and latency matter at scale. For pipelines processing thousands of feature batches daily, use a fast, cost-efficient model (e.g., GPT-4o-mini, Claude Haiku) for the initial classification pass. Reserve larger models for ambiguous cases where the primary model's confidence falls below a configurable threshold (e.g., < 0.7). Implement circuit breakers that halt drift classification if the model endpoint returns 5xx errors for more than 30 seconds, falling back to a statistical drift detection method (e.g., Population Stability Index) to keep the pipeline operational. Cache the prompt prefix containing the system instructions and output schema to reduce token costs and latency on repeated calls. Finally, version your prompt template alongside your model versions in a prompt registry, and run regression tests against a golden dataset of known drift and no-drift scenarios before deploying any prompt changes.