Inferensys

Prompt

Evidence Gap Analysis Prompt Template

A practical prompt playbook for using the Evidence Gap Analysis Prompt Template in production AI workflows to identify missing evidence before execution can proceed safely.
Product manager reviewing autonomous task execution dashboard on laptop, completed tasks visible, casual work session.
PROMPT PLAYBOOK

When to Use This Prompt

Identify when to run evidence gap analysis before committing to irreversible agent actions.

This prompt is for planning modules in autonomous and semi-autonomous agent systems that must identify what evidence is missing before execution can proceed safely. It analyzes a proposed plan against currently available evidence, identifies critical gaps, and produces a prioritized evidence collection plan with specific information requests. Use this prompt when your agent has generated a plan but needs to verify that it has sufficient evidence to execute each step without making dangerous assumptions.

This is not a plan generation prompt. It assumes a plan already exists and focuses exclusively on the gap between what the plan requires and what evidence is currently available. Run this before committing to irreversible actions, before spending tokens on multi-step execution, and whenever new context or tool outputs could invalidate prior assumptions. The prompt works best when the plan is already decomposed into discrete steps with clear preconditions, and when you have a structured inventory of currently available evidence sources. Do not use this prompt for initial plan creation, for evaluating plan quality in isolation, or for post-execution forensics—those are separate workflows with different output contracts.

The ideal user is an agent runtime or orchestration layer that needs a gating check between planning and execution. Wire this into your agent loop after plan generation but before the first tool call or state mutation. If your agent operates in a domain where incorrect actions have financial, safety, or compliance consequences, pair this prompt with a human approval step for any gaps scored above a configurable criticality threshold. For lower-risk workflows, you can use the output to drive automated evidence collection before proceeding. Avoid running this prompt on plans that are purely informational or read-only, as the overhead of gap analysis may exceed the value when no irreversible state changes are at stake.

PRACTICAL GUARDRAILS

Use Case Fit

Evidence gap analysis is a planning pre-flight check, not a runtime answer engine. It works best when the cost of missing information is high and the system has access to structured evidence sources. It fails when applied to creative tasks or when evidence sources are unavailable.

01

Good Fit: Pre-Execution Safety Gates

Use when: an agent is about to execute a multi-step plan with irreversible actions, financial impact, or compliance requirements. Evidence gap analysis prevents confident-but-wrong execution by forcing explicit evidence requirements before each critical step. Guardrail: Run gap analysis as a blocking gate before step execution, not as a parallel advisory check.

02

Bad Fit: Creative or Generative Workflows

Avoid when: the task involves content generation, brainstorming, or subjective quality assessment where there is no ground-truth evidence to check against. Evidence gap analysis will flag every creative choice as an unverified assumption, producing noise instead of safety. Guardrail: Use output review prompts instead of evidence verification for creative workflows.

03

Required Inputs: Structured Evidence Sources

Risk: running evidence gap analysis without accessible evidence sources produces a list of gaps with no path to resolution, wasting tokens and adding latency without improving safety. Guardrail: Only deploy when the agent has access to retrievable documents, API responses, database records, or explicit user-provided facts that can serve as evidence. If evidence sources are unavailable, skip gap analysis and escalate to human review directly.

04

Operational Risk: Gap Analysis Paralysis

Risk: overly strict evidence thresholds can prevent the agent from making any progress, especially when perfect evidence is unavailable. The agent loops on evidence collection without reaching a decision. Guardrail: Configure evidence sufficiency thresholds per assumption criticality. Critical assumptions require strong evidence; low-impact assumptions can proceed with weaker evidence or explicit assumptions logged for audit.

05

Operational Risk: Stale Evidence Blindness

Risk: evidence collected early in a workflow may become invalid as state changes, but the gap analysis prompt treats it as resolved. This produces false confidence in later steps. Guardrail: Include evidence freshness checks in the gap analysis harness. Re-validate assumptions before irreversible steps even if they were previously marked as evidenced. Log evidence age alongside gap status.

06

Scale Limit: Single-Agent Planning Scope

Risk: evidence gap analysis works within a single agent's plan and evidence context. In multi-agent systems, assumptions may be validated by one agent but invisible to another, creating hidden gaps at coordination boundaries. Guardrail: Use assumption communication prompts for handoffs between agents. Do not assume that evidence collected by one agent is automatically available to another without explicit transfer.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A copy-ready template for analyzing a proposed plan against available evidence, identifying critical gaps, and producing a prioritized evidence collection plan.

This template is designed to be dropped into an agent planning module before execution begins. It forces the model to treat the proposed plan as a hypothesis that must survive evidence scrutiny, not as a set of instructions to blindly follow. The prompt requires the model to enumerate what it knows, what it assumes, and what it must verify before proceeding. Use this when the cost of wrong execution is high—destructive operations, compliance-sensitive workflows, or multi-step plans where early errors compound. Do not use this for trivial, read-only, or fully deterministic tasks where evidence gaps cannot exist.

text
You are an evidence-gap analyst for an autonomous agent planning system. Your job is to examine a proposed execution plan against all available evidence and identify what critical information is missing before safe execution can proceed.

## INPUTS

[PROPOSED_PLAN]
(A structured, multi-step plan with step IDs, descriptions, dependencies, and expected outputs. Include any tool specifications or resource requirements.)

[AVAILABLE_EVIDENCE]
(A structured inventory of all evidence currently available, including source, timestamp, content summary, and reliability rating for each item. Mark any evidence that may be stale.)

[EXECUTION_CONTEXT]
(Operational context including: risk tolerance level [LOW|MEDIUM|HIGH|CRITICAL], whether any steps are irreversible, human-in-the-loop availability, and time constraints.)

[ASSUMPTION_INVENTORY]
(Optional. A pre-existing list of assumptions extracted from the goal and plan. If not provided, you must surface implicit assumptions during analysis.)

## OUTPUT SCHEMA

Return a valid JSON object with this structure:

{
  "plan_id": "string",
  "analysis_timestamp": "ISO8601",
  "overall_assessment": {
    "execution_readiness": "READY | PROCEED_WITH_CAUTION | BLOCKED",
    "critical_gap_count": "integer",
    "summary": "One-sentence assessment of whether the plan can proceed safely."
  },
  "evidence_gaps": [
    {
      "gap_id": "string",
      "affected_step_ids": ["step_id"],
      "gap_description": "What specific information is missing.",
      "criticality": "BLOCKING | HIGH | MEDIUM | LOW",
      "criticality_rationale": "Why this gap matters for safe execution.",
      "assumptions_at_risk": ["assumption_id or description"],
      "consequence_if_unresolved": "What happens if we proceed without this evidence.",
      "evidence_to_collect": "Specific information needed to close this gap.",
      "collection_method": "How to obtain this evidence (tool call, human query, retrieval, etc.).",
      "acceptable_proxy_evidence": "What partial or indirect evidence would be sufficient if direct evidence is unavailable."
    }
  ],
  "prioritized_collection_plan": [
    {
      "priority": "integer starting at 1",
      "gap_id": "string",
      "collection_action": "Concrete next action to collect this evidence.",
      "estimated_effort": "LOW | MEDIUM | HIGH",
      "blocks_other_gaps": ["gap_id"],
      "can_parallelize": "boolean"
    }
  ],
  "steps_safe_to_execute_now": ["step_id"],
  "steps_blocked": ["step_id"],
  "recommended_next_action": "PROCEED_WITH_SAFE_STEPS | COLLECT_EVIDENCE | ESCALATE_TO_HUMAN | ABORT"
}

## CONSTRAINTS

1. Every gap must be traceable to a specific plan step. No abstract or unlinked gaps.
2. Criticality scoring must reference [EXECUTION_CONTEXT] risk tolerance. A gap that is HIGH in a CRITICAL context may be BLOCKING.
3. If any BLOCKING gap exists, overall execution_readiness must be BLOCKED.
4. Do not invent evidence. If [AVAILABLE_EVIDENCE] is silent on a question, that is a gap.
5. Flag assumptions that are masquerading as evidence. If a claim in [AVAILABLE_EVIDENCE] is itself an assumption, call it out.
6. For irreversible steps, any unresolved gap of MEDIUM criticality or higher is automatically BLOCKING.
7. The prioritized_collection_plan must respect dependency order: if Gap B depends on Gap A being resolved first, Gap A gets higher priority.
8. If [ASSUMPTION_INVENTORY] is provided, cross-reference every assumption against [AVAILABLE_EVIDENCE] and flag unvalidated assumptions as gaps.
9. If evidence is older than its domain-appropriate freshness window, treat it as a gap requiring revalidation.
10. Do not recommend proceeding with steps that depend on gapped evidence, even if those steps appear low-risk in isolation.

Adaptation guidance: Replace the bracketed inputs with your system's actual data structures. The [PROPOSED_PLAN] should come from your plan generation module. The [AVAILABLE_EVIDENCE] should be assembled from your retrieval system, tool outputs, and session context. The [EXECUTION_CONTEXT] should be set by your orchestration layer based on the task's risk profile. The output schema is designed to be machine-readable by a downstream execution gate—parse overall_assessment.execution_readiness to decide whether to proceed, collect evidence, or escalate. For high-risk domains, add a human review step before acting on the recommended_next_action field. Test this prompt with plans that have deliberately missing evidence to verify it catches gaps rather than hallucinating justifications.

IMPLEMENTATION TABLE

Prompt Variables

Required inputs for the Evidence Gap Analysis prompt. Each placeholder must be populated before the prompt is sent. Validation notes describe how to programmatically verify the input before incurring inference cost.

PlaceholderPurposeExampleValidation Notes

[PLAN_STEPS]

The complete list of planned execution steps to analyze for evidence gaps

  1. Query user database
  2. Generate report
  3. Send email

Must be a non-empty array or numbered list. Reject if fewer than 1 step. Parse check: split on newlines or JSON array length > 0.

[AVAILABLE_EVIDENCE]

All evidence sources currently available to the agent, with source identifiers and retrieval timestamps

source:user_db_schema, ts:2025-01-15 source:report_template, ts:2025-01-14

Each entry must have a source identifier and timestamp. Reject if timestamp is older than [EVIDENCE_STALENESS_THRESHOLD] hours. Schema check: source field present, ts field parseable as ISO-8601.

[CRITICALITY_THRESHOLD]

The minimum gap criticality score that triggers a blocking action or escalation

0.7

Must be a float between 0.0 and 1.0. Reject if out of range. Default to 0.5 if null. Parse check: parseFloat and range validation.

[IRREVERSIBLE_ACTIONS]

List of plan step IDs or descriptions that cannot be rolled back if executed with incomplete evidence

step_3:send_email, step_5:delete_records

Each entry must match a step identifier in [PLAN_STEPS]. Reject if any irreversible action ID does not map to a known step. Cross-reference check required.

[DOMAIN_CONSTRAINTS]

Regulatory, compliance, or business rules that affect evidence requirements for specific step types

GDPR: user data access requires consent evidence SOC2: config changes require change ticket evidence

Optional. If provided, each constraint must have a rule label and description. Null allowed. If present, validate that constraint labels are non-empty strings.

[EVIDENCE_STALENESS_THRESHOLD]

Maximum age in hours before evidence is considered stale and must be re-verified

24

Must be a positive integer. Reject if <= 0. Default to 24 if null. Parse check: parseInt and > 0 validation.

[OUTPUT_FORMAT]

Desired structure for the gap analysis output, typically a JSON schema or field specification

JSON with fields: gap_id, step_id, evidence_needed, criticality_score, recommended_action

Must be a valid schema definition or field list. Reject if empty. Schema check: at minimum requires gap_id and step_id fields. Null not allowed.

[MAX_GAPS_TO_REPORT]

Upper limit on the number of evidence gaps returned to prevent output bloat in large plans

10

Must be a positive integer between 1 and 50. Reject if out of range. Default to 15 if null. Parse check: parseInt and range validation 1-50.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Evidence Gap Analysis prompt into an agent planning workflow with validation, scoring, and escalation.

The Evidence Gap Analysis prompt is designed to sit between a plan generation module and an execution engine. It receives a proposed plan and the currently available evidence, then outputs a prioritized list of missing information. To integrate this into a production agent, treat the prompt as a synchronous step in a planning pipeline: the orchestrator calls plan generation, then immediately calls evidence gap analysis on the generated plan before any execution steps are dispatched. The output of this prompt should gate execution—if critical gaps are found, the agent should not proceed to execute steps that depend on missing evidence. Instead, it should trigger evidence collection, clarification requests, or human escalation.

Wire the prompt into a function with the signature analyze_evidence_gaps(plan: Plan, evidence: EvidenceInventory, thresholds: GapThresholds) -> GapAnalysisResult. The GapThresholds object should carry configurable criticality scores that map to actions: gaps above a blocking_threshold halt execution entirely, gaps between warning_threshold and blocking_threshold allow execution with explicit warnings logged, and gaps below warning_threshold are noted but do not gate execution. The GapAnalysisResult must include a structured list of gaps, each with a criticality_score (0.0–1.0), impacted_steps (list of step IDs), evidence_required (specific information needed), and a collection_strategy (one of query_user, tool_call, retrieval, human_escalation). Validate the output against this schema before acting on it. If the model returns malformed JSON or missing required fields, retry once with a repair prompt that includes the schema and the raw output. If the retry fails, escalate to a human operator with the plan and evidence inventory attached.

For logging and observability, record every gap analysis result alongside the plan version, evidence inventory hash, and threshold configuration. This trace is essential for debugging false positives (gaps flagged that were actually covered by evidence) and false negatives (missing gaps that caused downstream execution failures). Implement a feedback loop: when an execution step fails due to missing evidence that the gap analysis should have caught, log that as a missed_gap event and feed it into your eval dataset for regression testing. For model choice, use a model with strong structured output capabilities and reasoning depth—this prompt requires the model to cross-reference plan steps against evidence sources, which is more demanding than simple classification. If latency is a concern, consider running gap analysis in parallel with other pre-execution checks, but ensure the execution engine waits for the gap analysis result before dispatching any step that touches mutable resources or has irreversible side effects.

When operating in regulated or high-risk domains, add a human review step for any gap with a criticality score above 0.7 or any gap that impacts steps marked as irreversible. The human reviewer should receive the gap analysis result, the original plan, and the evidence inventory in a structured review interface. Do not allow the agent to autonomously execute evidence collection strategies that involve external systems (tool calls, database queries) for high-criticality gaps without explicit approval. Finally, configure a maximum evidence collection budget—if the agent cannot close critical gaps within a fixed number of collection attempts or a time window, it should stop and escalate rather than loop indefinitely. The gap analysis prompt is a decision-support tool, not a replacement for operational judgment about when to proceed with uncertainty.

IMPLEMENTATION TABLE

Expected Output Contract

Validate the Evidence Gap Analysis JSON output against this contract before passing it to downstream planning or evidence-collection modules.

Field or ElementType or FormatRequiredValidation Rule

plan_id

string

Must match the [PLAN_ID] input exactly; reject on mismatch

overall_evidence_sufficiency_score

float (0.0-1.0)

Must be a number between 0.0 and 1.0 inclusive; reject if out of range or non-numeric

gaps

array of objects

Must be a non-empty array if overall_evidence_sufficiency_score < 1.0; warn if empty array with score < 1.0

gaps[].gap_id

string

Must be unique within the gaps array; reject on duplicate gap_id values

gaps[].step_id

string

Must reference a step_id present in the [PLAN_STEPS] input; reject on orphan references

gaps[].gap_description

string

Must be non-empty and describe a specific missing piece of evidence; reject on generic strings like 'missing info'

gaps[].criticality_score

float (0.0-1.0)

Must be a number between 0.0 and 1.0; reject if out of range; warn if criticality_score > 0.8 but gap_description lacks blocking language

gaps[].evidence_request

object

Must contain at least one of: specific_query, source_type, or required_format; reject if all three are null or missing

gaps[].evidence_request.specific_query

string

If present, must be a concrete, answerable question; reject on vague queries like 'get more data'

gaps[].evidence_request.source_type

string

If present, must be one of: document, api, database, human_expert, system_log, or external_service; reject on unknown source_type values

gaps[].evidence_request.required_format

string

If present, must be a valid MIME type or structured format name; reject on free-text that isn't a recognizable format

gaps[].blocking

boolean

Must be true or false; if true, criticality_score must be >= 0.7; warn on blocking=true with low criticality

gaps[].assumptions_at_risk

array of strings

Must reference assumption_ids from the [ASSUMPTION_INVENTORY] input; reject on references to non-existent assumptions

collection_plan

object

Must contain prioritized_steps array; reject if missing

collection_plan.prioritized_steps

array of objects

Must be ordered by priority descending; validate that step 0 has highest criticality_score among gaps

collection_plan.prioritized_steps[].gap_id

string

Must match a gap_id in the gaps array; reject on dangling references

collection_plan.prioritized_steps[].collection_method

string

Must be one of: query_generation, tool_call, human_escalation, retrieval_search, or direct_api; reject on unknown methods

collection_plan.prioritized_steps[].expected_resolution_time

string

If present, must be an ISO 8601 duration or a human-readable estimate with unit; warn on missing for blocking gaps

PRACTICAL GUARDRAILS

Common Failure Modes

Evidence gap analysis fails in predictable ways. These are the most common failure modes when prompting models to identify missing evidence, and the guardrails that prevent them.

01

False Confidence in Available Evidence

What to watch: The model treats low-quality or tangentially related evidence as sufficient to close a gap, producing a confident 'no gaps found' result when critical information is actually missing. This happens most often when evidence volume is high but relevance is low. Guardrail: Require the model to score each evidence item for relevance and reliability before declaring a gap closed. Add a 'minimum evidence quality threshold' that must be met per gap, and flag gaps where evidence exists but fails the quality bar.

02

Over-Identification of Trivial Gaps

What to watch: The model generates an exhaustive list of every conceivable missing piece of information, including low-impact gaps that would never change the execution decision. This floods the output with noise and obscures the gaps that actually matter. Guardrail: Add a criticality scoring step that ranks gaps by their potential to invalidate a plan step or change an outcome. Filter the final output to only include gaps above a configurable criticality threshold, with a hard cap on total gap count.

03

Gap Description Without Actionable Requests

What to watch: The model identifies that evidence is missing but describes the gap in abstract terms ('we need more information about the system') without specifying what question to ask, who to ask, or what format the answer should take. The output is diagnostically correct but operationally useless. Guardrail: Require each identified gap to include a concrete information request with: the specific question, the expected answer type, the source that could provide it, and the format the evidence should take. Validate that every gap entry has these fields populated before accepting the output.

04

Assumption Blindness in Gap Analysis

What to watch: The model analyzes evidence gaps against the explicit plan steps but fails to surface the implicit assumptions those steps depend on. A step may appear well-supported by evidence while resting on an unstated assumption that, if false, would invalidate the entire approach. Guardrail: Run an assumption extraction pass before gap analysis. Feed the extracted assumptions into the gap analysis prompt as additional items that require evidence. Cross-check each plan step's evidence against its dependency assumptions, not just its surface-level inputs.

05

Stale Evidence Treated as Current

What to watch: The model accepts evidence without checking temporal relevance, treating a six-month-old configuration snapshot or a deprecated API reference as valid support for a current plan step. This is especially dangerous in fast-changing environments where evidence decay is rapid. Guardrail: Require timestamp or freshness metadata on all evidence items. Add an explicit freshness check step that flags evidence older than a configurable threshold. For time-sensitive domains, require the model to estimate evidence half-life and downgrade confidence accordingly.

06

Circular Evidence Chains

What to watch: The model constructs a chain where Gap A is closed by Evidence B, but Evidence B's reliability depends on the same unverified assumption that created Gap A. The gap appears resolved while the underlying uncertainty is merely displaced, not eliminated. Guardrail: Require the model to trace each evidence item back to its source and check for dependency loops. Add a validation step that flags when an evidence source shares assumptions with the gap it's supposed to close. Produce a dependency graph and reject self-referential evidence chains.

IMPLEMENTATION TABLE

Evaluation Rubric

How to test output quality before shipping. Each criterion targets a specific failure mode in evidence gap analysis. Run these checks against a golden dataset of plans with known evidence gaps before deploying the prompt.

CriterionPass StandardFailure SignalTest Method

Gap Completeness

All critical evidence gaps present in the plan are identified in the output

Output misses a gap that would block safe execution

Diff output gaps against a pre-labeled gap inventory; recall must exceed 0.95

Gap Criticality Scoring

Each gap receives a criticality score with explicit rationale tied to plan step impact

Scores are assigned without justification or all gaps receive the same score

Spot-check 20 scored gaps; verify each has a non-generic rationale referencing a specific plan step

Evidence Request Specificity

Each information request names the exact data, source, or observation needed to close the gap

Requests are vague (e.g., 'more information needed') without specifying what to collect

Parse each request; flag any that lack a concrete source type, field name, or observable condition

Prioritization Order

Gaps are ordered by criticality score descending; ties are broken by dependency count

Low-criticality gaps appear before high-criticality gaps or ordering is arbitrary

Validate sort order programmatically against criticality scores; flag any inversion

Plan Step Traceability

Each gap includes a reference to the specific plan step or steps it affects

Gaps are listed without linking to any plan step, making downstream replanning impossible

Check that every gap object contains a non-empty [affected_step_ids] field matching actual plan step IDs

False Gap Rate

No gaps are reported for plan steps where evidence is already sufficient

Output flags gaps for steps that have complete, fresh evidence available in [AVAILABLE_EVIDENCE]

Cross-reference each reported gap against [AVAILABLE_EVIDENCE]; false positive rate must be below 0.05

Output Schema Compliance

Output matches the [OUTPUT_SCHEMA] exactly with all required fields present and correctly typed

Missing required fields, wrong types, or extra fields that violate the schema contract

Validate output with a JSON Schema validator against [OUTPUT_SCHEMA]; reject on any validation error

Confidence Calibration

Confidence scores on gap existence correlate with actual gap presence in held-out test cases

High-confidence gap predictions are frequently wrong or low-confidence predictions are always correct

Bucket predictions by confidence decile; compute expected calibration error against labeled test set

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Use the base prompt with a frontier model (GPT-4o, Claude 3.5 Sonnet) and minimal validation. Focus on getting the gap structure right before adding scoring rigor.

  • Remove the criticality_score field and use simple HIGH/MEDIUM/LOW labels.
  • Replace the structured evidence_requests array with a free-text "What to ask" section.
  • Skip the blocking_gaps boolean and let the human reviewer decide.
code
Analyze this plan against available evidence.

[PLAN_STEPS]
[AVAILABLE_EVIDENCE]

For each step, list:
- What evidence is missing
- Why it matters (HIGH/MEDIUM/LOW)
- What specific information would close the gap

Watch for

  • Overly broad gap descriptions that can't be actioned
  • Missing distinction between "nice to have" and "cannot proceed" gaps
  • No check that listed evidence actually exists in [AVAILABLE_EVIDENCE]
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.