Instruction Following Drift Detection Prompt for Multi-Step Tasks

This prompt is designed for agent platform engineers and QA leads who need to verify that a change to a system prompt, task decomposition prompt, or multi-step agent instruction set has not silently degraded the model's ability to follow complex procedures. The core job-to-be-done is automated regression testing of instruction adherence. You should use this prompt when you have a multi-step task (e.g., 'research a company, then write a brief, then format it as a memo') and a golden dataset of inputs paired with expected step-by-step compliance audits. The ideal user is someone integrating this check into a CI/CD release gate for an agent or copilot product.

This prompt is not a replacement for end-to-end integration tests or human evaluation of output quality. It specifically checks for structural compliance: omitted steps, ordering errors, and constraint violations. Do not use this prompt to judge the factual accuracy, tone, or helpfulness of the final output; pair it with separate semantic drift or factual consistency prompts for those dimensions. It is also not suitable for single-turn, non-procedural tasks where there is no explicit step sequence to audit. The prompt requires a detailed, pre-defined specification of the expected steps and constraints to function correctly.

Before deploying this prompt in a regression suite, you must have a stable, version-controlled specification of the multi-step task, a set of diverse test inputs, and a baseline compliance audit to compare against. The prompt works by decomposing the task into sub-tasks and scoring each independently, so the quality of your decomposition schema directly determines the reliability of the drift detection. Use this prompt as a gate that blocks promotion if the compliance score drops below a pre-defined threshold, and always log the per-step scores for root-cause analysis when a regression is flagged.

This prompt template is designed to be dropped into an agent evaluation harness or a CI/CD regression gate. It takes a multi-step task description, the agent's execution trace, and a set of expected constraints, then produces a structured compliance audit. The audit scores each sub-task independently, flags ordering violations, and identifies omitted or hallucinated steps. Use this when you need to verify that a prompt change hasn't degraded an agent's ability to follow complex instructions across multiple turns or tool calls.

textCopy
You are an instruction-following compliance auditor. Your job is to compare an agent's execution trace against a reference multi-step task specification and produce a structured audit report.

## INPUTS

### TASK_SPECIFICATION
[ORIGINAL_TASK_DESCRIPTION]

### EXPECTED_STEPS
[EXPECTED_STEPS_LIST]

### EXECUTION_TRACE
[AGENT_EXECUTION_TRACE]

### CONSTRAINTS
[CONSTRAINTS_LIST]

### OUTPUT_SCHEMA
{
  "audit_id": "string",
  "overall_compliance_score": 0.0-1.0,
  "step_audits": [
    {
      "step_id": "string",
      "step_description": "string",
      "status": "completed | omitted | partial | hallucinated | wrong_order",
      "evidence": "string (quote from trace)",
      "score": 0.0-1.0,
      "issues": ["string"]
    }
  ],
  "ordering_violations": [
    {
      "expected_order": ["step_id"],
      "actual_order": ["step_id"],
      "severity": "critical | major | minor"
    }
  ],
  "constraint_violations": [
    {
      "constraint": "string",
      "violation": "string",
      "evidence": "string",
      "severity": "critical | major | minor"
    }
  ],
  "omitted_steps": ["step_id"],
  "hallucinated_steps": [
    {
      "description": "string",
      "evidence": "string"
    }
  ],
  "summary": "string (2-3 sentence assessment)"
}

## INSTRUCTIONS
1. Parse the TASK_SPECIFICATION and EXPECTED_STEPS to understand the reference behavior.
2. Analyze the EXECUTION_TRACE step by step. Match each trace action to an expected step where possible.
3. For each expected step, determine status:
   - completed: step was executed correctly and in the right order
   - omitted: step was not found in the trace
   - partial: step was started but not completed
   - hallucinated: trace contains a step not in the expected list
   - wrong_order: step was executed but at the wrong position
4. Check all CONSTRAINTS against the trace. Flag any violation with evidence and severity.
5. Detect ordering violations by comparing expected step sequence to actual execution sequence.
6. Score each step 0.0-1.0 based on completeness and correctness. Overall score is the mean of step scores minus penalty for ordering and constraint violations.
7. Output ONLY valid JSON matching the OUTPUT_SCHEMA. No markdown, no commentary outside the JSON.

## SEVERITY GUIDELINES
- critical: step omission or violation that makes the task outcome invalid or unsafe
- major: significant deviation that degrades quality but doesn't invalidate the result
- minor: cosmetic or non-material deviation

Adapt this template by replacing the square-bracket placeholders with your actual data. The [ORIGINAL_TASK_DESCRIPTION] should contain the full prompt or instruction set given to the agent. [EXPECTED_STEPS_LIST] should be a numbered or bulleted list of the sub-tasks you expect the agent to perform, in the correct order. [AGENT_EXECUTION_TRACE] should include the full conversation, tool calls, and outputs. [CONSTRAINTS_LIST] should enumerate any rules the agent must follow, such as 'never expose PII', 'always cite sources', or 'confirm before deleting'. For high-risk workflows like healthcare or finance, always route outputs scoring below 0.85 to human review before accepting the audit result.

Before wiring this into production, validate that your [EXPECTED_STEPS_LIST] is exhaustive and that your trace format is parseable. A common failure mode is providing traces that are truncated or missing intermediate tool calls, which causes false omission flags. Test this prompt against a golden dataset of known compliant and non-compliant traces to calibrate your pass/fail threshold. If you're using this in a CI/CD pipeline, pair it with a JSON Schema validator that confirms the output structure before the audit scores are used as a release gate.

This prompt is designed to operate as a batch evaluation step inside a CI/CD pipeline or a scheduled regression job, not as a one-off manual check. The harness calls the prompt once per test case in your golden dataset, passing the original multi-step instruction, the expected task decomposition, and the actual output from the new prompt version. Each invocation returns a structured JSON audit with per-step compliance scores, ordering checks, and constraint violation flags. The harness aggregates these results across all test cases to produce a pass/fail decision for the release gate.

Wiring the prompt into your application requires three components: a golden dataset of multi-step tasks with human-verified decompositions, a model invocation layer that sends the prompt template with the correct variable substitutions, and a post-processing validator. For each test case, populate [ORIGINAL_INSTRUCTION] with the user-facing task, [EXPECTED_STEPS] with the reference decomposition (ordered list of sub-tasks with constraints), and [ACTUAL_OUTPUT] with the new prompt version's response. The model returns a JSON object with fields like step_compliance_scores, ordering_errors, omitted_steps, and constraint_violations. Use a JSON Schema validator to confirm the output structure before parsing scores. If the model returns malformed JSON, retry once with a stricter output format instruction appended to the prompt—beyond that, flag the test case as unevaluable and escalate for manual review.

Scoring and gating should happen at the aggregate level. Define a minimum per-step compliance threshold (e.g., 0.85 on a 0–1 scale) and a maximum allowed count of ordering errors and omitted steps across the full test suite. For high-risk agent workflows where instruction drift could cause downstream tool misuse, set stricter thresholds and require zero constraint violations on safety-critical steps. Log every evaluation result—including raw model outputs, parsed scores, and any validator rejections—to your observability platform so you can trace regressions to specific prompt changes. If the aggregate scores fall below threshold, block the release and surface the worst-performing test cases for prompt engineers to debug. For low-risk informational tasks, you may allow a small amount of drift with a warning rather than a hard block, but always track the trend over time to catch gradual degradation.