Uncertainty Propagation Tracking Prompt

This prompt is for AI engineers and platform architects building multi-step agent pipelines, RAG chains, or composite reasoning systems where uncertainty in one step can cascade and corrupt downstream decisions. The job-to-be-done is to track how confidence scores, ambiguity flags, or error margins compound across a sequence of model calls and produce a cumulative confidence estimate with a clear decision recommendation—proceed, retry, escalate, or abort. The ideal user is someone who already has per-step confidence signals (from model self-assessment, classifier scores, or validator outputs) and needs a structured way to propagate those signals through a dependency graph without losing fidelity or masking critical failure modes.

Do not use this prompt when you have only a single model call with no upstream dependencies, or when your confidence signals are binary pass/fail with no intermediate uncertainty. It is also the wrong tool when you need real-time latency guarantees—propagation analysis adds an extra inference step and should be reserved for offline evaluation, post-hoc debugging, or asynchronous guardrail pipelines. The prompt assumes you can provide a structured trace of steps, each with an input, output, confidence score, and dependency list. If your pipeline lacks per-step observability, invest in instrumentation before attempting propagation tracking.

Before wiring this into production, define your propagation rules explicitly: is uncertainty additive, multiplicative, or worst-case? Does a low-confidence retrieval step invalidate all downstream reasoning, or only reduce the final score by a configurable weight? The prompt template includes a [PROPAGATION_RULES] placeholder where you encode these domain-specific policies. Start by running this against simulated traces with known ground-truth outcomes to calibrate whether the cumulative score and recommendation align with your risk tolerance. Avoid treating the propagated score as a precise probability—it is a decision-support signal, not a calibrated measurement, and should feed into a broader evaluation harness with human review for high-risk actions.

This prompt template is designed for multi-step agent and pipeline builders who need to track how uncertainty propagates and compounds across sequential operations. Instead of treating each step's confidence in isolation, this prompt instructs the model to maintain a running uncertainty ledger, update it after each step, and produce a final cumulative confidence estimate with a clear decision recommendation. The template uses square-bracket placeholders for all variable inputs, making it straightforward to integrate into an agent harness or pipeline orchestrator.

textCopy
You are an uncertainty propagation tracker for a multi-step AI pipeline. Your job is to maintain a running ledger of confidence estimates across steps, compute how uncertainty compounds, and produce a final cumulative confidence score with a decision recommendation.

## INPUT DATA

**Step History:**
[STEP_HISTORY]
<!-- JSON array of step objects, each containing:
  - step_id: unique identifier
  - step_name: human-readable step description
  - output_summary: brief summary of what the step produced
  - step_confidence: confidence score for this step (0.0 to 1.0)
  - uncertainty_sources: array of strings describing what introduced uncertainty
  - dependencies: array of step_ids this step depends on
-->

**Propagation Rules:**
[PROPAGATION_RULES]
<!-- JSON object defining how uncertainty compounds:
  - dependency_penalty: multiplier applied when a step depends on a low-confidence predecessor
  - sequential_decay: per-step confidence decay factor for long chains
  - independence_bonus: confidence boost when independent steps agree
  - conflict_penalty: confidence reduction when dependent steps disagree
-->

**Risk Profile:**
[RISK_PROFILE]
<!-- String enum: "low", "medium", "high", "critical" -->

**Decision Thresholds:**
[DECISION_THRESHOLDS]
<!-- JSON object mapping risk levels to threshold values:
  - proceed_threshold: confidence above this means autonomous continuation
  - review_threshold: confidence between this and proceed means human review
  - abort_threshold: confidence below this means stop and escalate
-->

## OUTPUT SCHEMA

Return a valid JSON object with this exact structure:
{
  "cumulative_confidence": <float 0.0-1.0>,
  "confidence_trend": "<improving|stable|degrading|volatile>",
  "propagation_ledger": [
    {
      "step_id": "<string>",
      "input_confidence": <float>,
      "propagated_uncertainty": <float>,
      "cumulative_after_step": <float>,
      "compounding_factors": ["<string>"]
    }
  ],
  "weakest_link": {
    "step_id": "<string>",
    "reason": "<string>"
  },
  "decision": {
    "action": "<proceed|review|abort>",
    "rationale": "<string explaining why this action was chosen>",
    "escalation_priority": "<low|medium|high|critical>"
  },
  "propagation_accuracy_estimate": <float 0.0-1.0>,
  "recommended_mitigations": ["<string>"]
}

## CONSTRAINTS

1. Track uncertainty propagation through dependency chains: if step B depends on step A and step A has low confidence, step B's effective confidence must be penalized even if step B's own execution was high-confidence.
2. Account for sequential decay: longer chains accumulate more uncertainty. Apply the sequential_decay factor from propagation rules.
3. Identify the weakest link: which single step most undermines the overall confidence?
4. Produce a binary decision (proceed, review, abort) based on the cumulative confidence and the decision thresholds for the given risk profile.
5. Estimate your own propagation accuracy: how confident are you that this propagation analysis is correct given the input data quality?
6. If the step history is empty, return cumulative_confidence of 1.0 with action "proceed" and note the empty chain.
7. If any step has a null or missing confidence, treat it as 0.5 with a noted uncertainty source of "missing_confidence_estimate".
8. Never invent step data. Only use what is provided in [STEP_HISTORY].

## EXAMPLES

[EXAMPLES]
<!-- Optional: few-shot examples of step histories with known propagation outcomes for calibration -->

To adapt this template, replace each square-bracket placeholder with the corresponding data from your pipeline execution context. The [STEP_HISTORY] should be a JSON array populated by your agent harness after each step completes, capturing the step's own confidence estimate and its dependency graph. The [PROPAGATION_RULES] and [DECISION_THRESHOLDS] should be configured once per deployment based on your risk tolerance and domain requirements. The optional [EXAMPLES] placeholder can be populated with few-shot demonstrations of known propagation chains to improve calibration. Before deploying, validate that the output JSON matches the schema exactly and that the cumulative confidence correctly reflects dependency penalties by testing against simulated chains with known ground-truth outcomes.

The Uncertainty Propagation Tracking Prompt is designed to sit between steps in a multi-step agent or pipeline, not as a standalone endpoint. After each step produces an output with a confidence score, this prompt ingests the accumulated chain of step-level confidences and the final answer to produce a cumulative confidence estimate and a decision recommendation. The harness should call this prompt after every N steps or at the final step, depending on your risk tolerance. For high-stakes workflows, run it after every step to catch compounding uncertainty early. For lower-stakes pipelines, running it only at the final step may be sufficient. The prompt expects a structured input containing the chain history, so your harness must maintain a running log of each step's output, its confidence score, and any uncertainty flags before invoking this prompt.

Wire the prompt into your application as a post-step evaluation function. After each agent step completes, append the step's output and confidence metadata to a chain_history array. When you're ready to assess cumulative uncertainty, construct the prompt's [CHAIN_HISTORY] placeholder by serializing this array into a structured format—preferably JSON with fields for step_id, step_description, output_summary, confidence_score, and uncertainty_flags. Include the [FINAL_OUTPUT] and [RISK_THRESHOLD] placeholders. On the response side, validate the output against a strict schema: require cumulative_confidence (a float between 0 and 1), propagation_trend (one of stable, degrading, improving, volatile), decision (one of proceed, review, escalate, abort), and rationale (a string). If the model returns invalid JSON, missing required fields, or a cumulative_confidence outside the 0–1 range, trigger a retry with the validation error message appended to the prompt as a correction hint. After two failed retries, escalate to a human reviewer with the raw model output and validation errors logged.

Model choice matters here. This prompt requires strong instruction-following and structured output discipline. Use a model with reliable JSON mode or function-calling support, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Avoid smaller or older models that may conflate step-level and cumulative confidence or produce inconsistent decision recommendations. Log every invocation: capture the full prompt, the raw model response, the validated output, and the final decision. This audit trail is essential for debugging propagation accuracy and for compliance in regulated workflows. If your pipeline uses tool calls, include tool-call success/failure status in the chain_history entries—a failed tool call should degrade confidence more than a successful but low-confidence step. Finally, wire the decision field into your orchestration logic: proceed continues the pipeline, review flags the output for human inspection but doesn't block downstream steps, escalate pauses the pipeline and sends the full chain to a review queue, and abort terminates execution and logs the incident. Test these decision gates with simulated chains that have known propagation outcomes before deploying to production.