Inferensys

Prompt

Span Boundary Confidence Annotation Prompt

A practical prompt playbook for annotating extracted text spans with boundary confidence markers, flagging fuzzy boundaries, and producing alternative span candidates in production extraction pipelines.
Wide-angle shot of a modern WeWork open floor plan with creative walls covered in AI system architecture diagrams, product team collaborating in standing desk area with industrial lighting.
PROMPT PLAYBOOK

When to Use This Prompt

Determines when span boundary confidence annotation is the right tool versus simpler extraction or full manual review.

Use this prompt when the exact character positions of an extracted value matter as much as the value itself. Span boundary confidence annotation is designed for extraction engineers who need to know not just what was extracted, but whether the model is certain about where the extraction starts and ends. This matters in legal clause boundaries where a single word changes obligation scope, contract dates where an adjacent number alters the effective period, clinical measurements where unit boundaries affect dosage interpretation, and financial figures where digit grouping changes magnitude. If your downstream system consumes span offsets for highlighting, citation anchoring, or record assembly, boundary errors cause cascading failures that are expensive to detect and repair after ingestion.

This prompt is not the right choice when you only need field values without positional provenance. If your pipeline treats extracted text as opaque strings and never renders source highlights, generates citation anchors, or assembles records from overlapping spans, a standard field extraction prompt with per-field confidence scores will be simpler, faster, and cheaper. Similarly, if your documents have unambiguous field delimiters—key-value pairs with colons, form fields with labels, or tabular data with consistent separators—boundary annotation adds overhead without benefit. Reserve this prompt for unstructured or semi-structured text where entity boundaries are genuinely ambiguous: run-on legal prose, clinical narratives, conversational transcripts, or documents where the same value could reasonably be parsed with different start and end positions.

Before deploying this prompt, confirm that your downstream consumers can act on boundary confidence signals. The prompt produces left-boundary and right-boundary confidence scores, ambiguity flags, and alternative boundary candidates. If your application only stores a single span per field and discards the confidence metadata, you are paying the token and latency cost of boundary annotation without using its output. Wire the boundary scores into your ingestion pipeline: route high-confidence spans directly to your system of record, flag ambiguous spans for human review, and use alternative candidates to power a disambiguation UI where reviewers select the correct boundary. For regulated domains, retain the boundary confidence trail as audit evidence that ambiguous spans were identified and resolved rather than silently ingested.

Start with a narrow scope. Pick one field type where boundary errors have caused production incidents—contract effective dates, financial amounts, or medication dosages—and run this prompt against a labeled dataset where you know the correct span boundaries. Measure boundary precision and recall separately from value accuracy. A model can extract the correct dollar amount while misaligning the span by one character, and that one-character error might break a downstream parser. Use the eval results to decide whether boundary annotation should become a standard layer in your extraction architecture or remain a targeted tool for high-risk fields.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Span Boundary Confidence Annotation Prompt delivers value and where it introduces risk. Use these cards to decide if this prompt fits your extraction pipeline before you integrate it.

01

Good Fit: High-Precision Entity Extraction

Use when: downstream systems depend on exact character offsets for legal, clinical, or financial text. Why: the prompt annotates fuzzy boundaries and provides alternative spans, reducing silent misalignment. Guardrail: always validate boundary offsets against the original source string before ingestion.

02

Bad Fit: Bulk Keyword Tagging

Avoid when: you only need presence/absence of terms without span precision. Why: boundary annotation adds token cost and latency with no benefit for simple keyword extraction. Guardrail: use a lightweight classification prompt instead and reserve this prompt for span-critical fields only.

03

Required Inputs: Source Text and Target Schema

Required: the raw source text, a list of entity types or fields to extract, and the expected output schema with span fields. Risk: without the original text, boundary offsets cannot be validated. Guardrail: always pass the exact source string the model should annotate; never pre-truncate or normalize it before extraction.

04

Operational Risk: Boundary Drift Under Paraphrase

Risk: when source text is paraphrased or summarized before extraction, boundary annotations become meaningless. Guardrail: enforce a contract that this prompt only runs on verbatim source text. Add a pre-check that detects if the input has been rewritten.

05

Operational Risk: Over-Confidence on Ambiguous Spans

Risk: the model may assign high confidence to a single span when multiple interpretations are equally valid. Guardrail: require the prompt to output alternative span candidates with confidence scores. Flag any field where the top candidate confidence is below 0.9 for human review.

06

Pipeline Fit: Pre-Ingestion Validation Gate

Use when: this prompt sits before a database insert or API call that requires exact span offsets. Guardrail: run a post-extraction validator that checks every span offset against the source text character positions. Reject records with out-of-bounds offsets before they reach downstream systems.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

Paste this prompt and replace the square-bracket placeholders with your inputs. The prompt instructs the model to extract spans, annotate boundary confidence, flag fuzzy edges, and produce alternatives.

This template is the core instruction you send to the model. It enforces a strict output contract: every extracted span must carry a boundary_confidence score, a fuzzy_boundary flag, and a list of alternative_spans when the model is uncertain. The prompt is designed to be self-contained so that you can copy it, replace the placeholders, and get structured, auditable results without additional system prompts.

text
You are a precise span extraction engine. Your task is to extract spans from the provided text that match the target entity type, and annotate each span with boundary confidence information.

## INPUT
[INPUT_TEXT]

## TARGET ENTITY
[ENTITY_TYPE]

## OUTPUT SCHEMA
Return a JSON object with an "extractions" array. Each extraction must follow this schema exactly:
{
  "span_text": "string (the exact text of the extracted span)",
  "start_char": integer,
  "end_char": integer,
  "boundary_confidence": float (0.0 to 1.0, where 1.0 means perfectly certain about the exact start and end characters),
  "fuzzy_boundary": boolean (true if the start or end character position is uncertain),
  "boundary_notes": "string (explain why the boundary is fuzzy, or 'clear' if not)",
  "alternative_spans": [
    {
      "span_text": "string",
      "start_char": integer,
      "end_char": integer,
      "rationale": "string (why this alternative is plausible)"
    }
  ]
}

## CONSTRAINTS
- Extract every instance of [ENTITY_TYPE] found in the text. Do not skip any.
- `start_char` and `end_char` must be 0-indexed character offsets into the original [INPUT_TEXT].
- If you are completely certain about the span boundaries, set `boundary_confidence` to 1.0, `fuzzy_boundary` to false, and `alternative_spans` to an empty array.
- If the boundaries are ambiguous, set `fuzzy_boundary` to true, provide a `boundary_confidence` below 1.0, and include at least one `alternative_spans` entry with a plausible different span.
- Do not invent spans that are not present in the text. If no spans are found, return an empty "extractions" array.
- Do not include any text outside the JSON object.

## EXAMPLES
[FEW_SHOT_EXAMPLES]

## RISK LEVEL
[RISK_LEVEL]

After pasting the template, replace [INPUT_TEXT] with the document or passage you are extracting from. Set [ENTITY_TYPE] to the category of span you want—such as "person name," "monetary amount," or "drug dosage." Provide [FEW_SHOT_EXAMPLES] as inline JSON examples showing correct extractions with boundary annotations for your domain. Set [RISK_LEVEL] to "low," "medium," or "high" to control the model's conservatism: at "high," the model should lower confidence and expand alternatives for any boundary ambiguity. For production use, always validate the output JSON against the schema before ingestion, and route any record where fuzzy_boundary is true to a human review queue if the downstream risk warrants it.

IMPLEMENTATION TABLE

Prompt Variables

Every placeholder the Span Boundary Confidence Annotation Prompt expects, why it matters, and how to validate it before sending.

PlaceholderPurposeExampleValidation Notes

[SOURCE_TEXT]

The raw unstructured text from which spans will be extracted and annotated.

The patient was administered 50mg of metoprolol tartrate on 2024-03-15 at approximately 08:30 AM.

Check that the string is non-empty, under the model's context limit, and has not been pre-truncated in a way that splits target spans.

[TARGET_ENTITY_TYPES]

A list of entity types to extract, each requiring boundary confidence annotation.

["MEDICATION", "DOSAGE", "DATE", "TIME"]

Validate that the list is non-empty, each type is a non-empty string, and types do not overlap semantically in a way that creates ambiguous extraction instructions.

[CONFIDENCE_THRESHOLD]

The minimum boundary confidence score (0.0-1.0) a span must meet to be auto-accepted; spans below this are flagged for review.

0.85

Must be a float between 0.0 and 1.0 inclusive. A value of 0.0 disables auto-acceptance; 1.0 requires perfect certainty. Reject non-numeric or out-of-range values.

[MAX_ALTERNATIVE_SPANS]

The maximum number of alternative boundary candidates to return for each extracted entity when the primary span is below the confidence threshold.

3

Must be an integer >= 0. A value of 0 suppresses alternatives. Validate that the number is reasonable for the expected entity density to avoid output bloat.

[OUTPUT_SCHEMA]

The exact JSON schema the output must conform to, including fields for span, confidence, alternatives, and flags.

{"entity_type": "string", "primary_span": {"text": "string", "start_char": "int", "end_char": "int"}, "boundary_confidence": "float", "alternatives": "array", "review_flag": "boolean"}

Validate the schema string is valid JSON and contains required fields: entity_type, primary_span, boundary_confidence, and review_flag. Reject malformed schemas before sending.

[AMBIGUITY_MARKERS]

A list of lexical or structural patterns that signal boundary ambiguity, used by the prompt to guide the model's attention.

["approximately", "around", "about", parenthetical asides, sentence fragments]

Ensure the list is not empty and contains concrete examples. Validate that markers are specific enough to be actionable (e.g., avoid overly generic terms like 'the').

[FEW_SHOT_EXAMPLES]

A set of 2-4 annotated examples demonstrating correct span extraction, boundary confidence scoring, and alternative generation.

[{"source": "...", "extractions": [{"entity_type": "DOSAGE", "primary_span": {"text": "50mg", ...}, "boundary_confidence": 0.98, ...}]}]

Validate that each example includes a source text and a complete extraction object matching the [OUTPUT_SCHEMA]. Check that examples cover both high-confidence and ambiguous boundary cases.

[EVAL_REFERENCE_SPANS]

A set of ground-truth spans used for post-extraction evaluation, not included in the prompt itself but used by the harness to calculate boundary accuracy.

[{"entity_type": "MEDICATION", "reference_span": {"text": "metoprolol tartrate", "start_char": 35, "end_char": 54}}]

Validate that reference spans have non-negative start_char and end_char with start < end, and that the text matches the substring at those indices in the [SOURCE_TEXT]. Used only by the eval harness, not the model.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Span Boundary Confidence Annotation Prompt into an extraction pipeline with validation, retry, logging, and human review.

The Span Boundary Confidence Annotation Prompt is designed to sit inside a production extraction pipeline where span precision is critical. Unlike field-level extraction, this prompt returns character-level boundary annotations, confidence markers, and alternative span candidates. The harness must treat the output as a structured artifact that downstream systems can validate, compare against ground truth, and route for review when boundary confidence falls below acceptable thresholds. The primary integration points are: a pre-processing step that normalizes input text and preserves original character offsets, the model call itself with strict output schema enforcement, a post-processing validator that checks span overlap, boundary consistency, and confidence score ranges, and a routing decision that sends low-confidence spans to a human review queue.

Start by wrapping the prompt in a function that accepts [INPUT_TEXT], [TARGET_ENTITY_TYPE], and [CONFIDENCE_THRESHOLD] as parameters. The model should be called with response_format set to a JSON schema that enforces the output contract: an array of span objects, each containing span_text, start_char, end_char, boundary_confidence (0.0–1.0), boundary_flags (an array of enum values like FUZZY_START, FUZZY_END, OVERLAPPING_CANDIDATE), and alternative_spans (an array of objects with span_text, start_char, end_char, and confidence). After the model responds, run a boundary validator that checks: (1) all start_char and end_char values are within the input text length, (2) span_text matches the substring at those offsets, (3) no two primary spans overlap unless explicitly flagged, (4) boundary_confidence is a float between 0 and 1, and (5) alternative spans also pass offset validation. If validation fails, retry once with the validation error message appended to the prompt as a correction hint. If the retry also fails, log the failure and escalate the entire document to human review rather than silently accepting broken spans.

For observability, log every extraction call with: the prompt version hash, model ID, input text length, number of spans returned, average boundary confidence, count of spans flagged as fuzzy, count of spans below threshold, validation pass/fail status, retry count, and latency. Emit these as structured logs or metrics so you can detect drift in boundary confidence distributions over time. For human review routing, configure a queue that receives spans where boundary_confidence < [CONFIDENCE_THRESHOLD] or where boundary_flags is non-empty. Each review item should include the original sentence, the model's primary span, all alternative spans, and a pre-filled form for the reviewer to confirm or adjust boundaries. Track review decisions to build a boundary accuracy eval dataset. Avoid wiring this prompt directly into a synchronous user-facing flow without a timeout and fallback—boundary annotation on long documents can be latent, and a stuck model call should degrade gracefully to a partial result or an escalation message rather than blocking the user.

IMPLEMENTATION TABLE

Expected Output Contract

Every field the Span Boundary Confidence Annotation Prompt must return, its type, required status, and the validation rule that determines pass or fail in production.

Field or ElementType or FormatRequiredValidation Rule

[EXTRACTED_SPAN]

string

Must be a non-empty substring of [SOURCE_TEXT]. Fail if span is absent, empty, or not found verbatim in source.

[SPAN_START_INDEX]

integer

Must be >= 0 and < len([SOURCE_TEXT]). Fail if index points outside source or does not match start of [EXTRACTED_SPAN].

[SPAN_END_INDEX]

integer

Must be > [SPAN_START_INDEX] and <= len([SOURCE_TEXT]). Fail if end precedes start or does not match end of [EXTRACTED_SPAN].

[BOUNDARY_CONFIDENCE]

float

Must be in range [0.0, 1.0]. Fail if missing, non-numeric, or outside range. Values below [CONFIDENCE_THRESHOLD] trigger review routing.

[BOUNDARY_AMBIGUITY_FLAG]

boolean

Must be true if boundary is fuzzy, false if precise. Fail if null or non-boolean. True requires at least one [ALTERNATIVE_SPAN].

[AMBIGUITY_REASON]

string

true if [BOUNDARY_AMBIGUITY_FLAG] is true

Must be one of: Lexical, Structural, Contextual, or Overlap. Fail if flag is true and reason is missing or not in controlled vocabulary.

[ALTERNATIVE_SPANS]

array of objects

If present, each object must contain span, start_index, end_index, and confidence. Fail if any alternative span is not a valid substring of [SOURCE_TEXT].

[EXTRACTION_ID]

string

Must be a non-empty UUID or unique identifier matching pattern ^[a-f0-9-]{36}$. Fail if missing, duplicate, or malformed. Used for audit trail linking.

PRACTICAL GUARDRAILS

Common Failure Modes

Span boundary confidence annotation fails in predictable ways. These are the most common production failure modes and how to guard against them before they corrupt downstream extraction pipelines.

01

Boundary Overconfidence on Ambiguous Spans

What to watch: The model assigns high confidence to a span boundary even when the text supports multiple valid interpretations—such as a person's name that could include or exclude a middle initial, or a date range with an unclear endpoint. The extraction looks authoritative but is silently wrong. Guardrail: Require the prompt to output alternative span candidates whenever boundary ambiguity exists. Add a post-extraction check that flags any span where the model did not produce at least one alternative candidate, and route single-candidate spans with confidence below 0.95 to human review.

02

Boundary Drift Across Repeated Extractions

What to watch: The same document produces slightly different span boundaries across multiple extraction runs—especially with long contexts, complex nested phrases, or when the model's sampling temperature is non-zero. This creates non-deterministic downstream records that break deduplication and audit trails. Guardrail: Run boundary extraction at temperature 0. For production pipelines, implement a stability check that extracts the same document 3 times and flags any field where span boundaries differ across runs. Route unstable spans to a boundary-resolution review queue.

03

Boundary Creep into Adjacent Content

What to watch: The extracted span bleeds into neighboring sentences, clauses, or list items—especially when punctuation is ambiguous, formatting is inconsistent, or the target entity appears near structurally similar text. The extracted value contains extra tokens that corrupt normalization and downstream joins. Guardrail: Include explicit boundary delimiter rules in the prompt (e.g., 'stop at the next sentence boundary, comma, or conjunction unless the entity explicitly spans it'). Add a post-extraction validator that checks extracted spans against sentence and clause boundaries using a lightweight NLP tokenizer, flagging spans that cross sentence boundaries without justification.

04

Confidence-Calibration Collapse on Edge Cases

What to watch: The model assigns uniformly high or uniformly low confidence to all span boundaries when encountering unfamiliar document structures, domain-specific jargon, or noisy OCR text. The confidence scores become useless for routing decisions because they fail to discriminate between genuinely certain and uncertain boundaries. Guardrail: Include few-shot examples in the prompt that demonstrate appropriate confidence variation—showing both high-confidence and low-confidence boundary annotations with justifications. Calibrate against a golden dataset of edge cases before deployment, and monitor confidence score variance per document; trigger an alert when variance drops below a threshold.

05

Alternative Candidate Suppression

What to watch: The model correctly identifies that a span boundary is ambiguous but suppresses alternative candidates in the output—either because the prompt doesn't explicitly require them, or because the output schema makes alternatives optional. Reviewers see a single boundary with moderate confidence but no visibility into what the other options were. Guardrail: Make alternative span candidates a required field in the output schema, not optional. Enforce a minimum of one alternative whenever the primary boundary confidence is below 0.9. Add a schema validation step that rejects any extraction where confidence < 0.9 and the alternatives array is empty.

06

Boundary Justification Hallucination

What to watch: The model produces plausible-sounding boundary justifications that cite evidence spans which don't actually exist in the source text, or that misrepresent what the cited span contains. This is especially dangerous in audit contexts where reviewers trust the justification without re-reading the source. Guardrail: Require the prompt to quote the exact source text substring that justifies each boundary decision. Implement a post-extraction string-match validator that verifies every quoted justification substring actually appears in the source document. Flag any extraction where a quoted substring cannot be found for immediate human review.

IMPLEMENTATION TABLE

Evaluation Rubric

Run these checks against a labeled golden dataset of at least 50 examples with known correct span boundaries. Each criterion targets a specific failure mode observed in span boundary annotation.

CriterionPass StandardFailure SignalTest Method

Boundary Exact Match Rate

= 90% of predicted span boundaries match golden start and end offsets exactly

Boundary match rate drops below 90% on golden set

Token-level comparison of predicted [start, end] against golden labels; count exact matches divided by total spans

Boundary Offset Tolerance

= 95% of predicted boundaries are within ±2 tokens of golden boundary

More than 5% of predictions deviate by 3+ tokens from golden

Measure character or token offset delta between predicted and golden boundaries; flag spans exceeding tolerance threshold

Alternative Span Recall

= 80% of golden spans appear in the alternative_candidates list when primary span is incorrect

Golden span missing from alternatives list on more than 20% of boundary errors

For each primary span mismatch, check if golden span exists in the alternatives array; compute recall over all mismatches

Confidence Calibration

Mean confidence score for correct spans >= 0.8; mean confidence for incorrect spans <= 0.5

High confidence assigned to wrong boundaries or low confidence to correct ones

Bucket predictions by correctness; compute mean confidence per bucket; flag if distributions overlap significantly

Fuzzy Boundary Flag Rate

fuzzy_boundary flag is true on >= 85% of spans where golden boundary is ambiguous or multi-interpretation

Fuzzy flag false on known-ambiguous spans or true on clear-cut spans

Label golden set with ambiguity ground truth; compute precision and recall of fuzzy_boundary boolean against that label

Null Span Handling

No span returned when text contains no mention of target entity; null_reason populated with valid enum value

Hallucinated span returned for absent entity or null_reason missing on true null

Inject negative examples with zero entity mentions; verify output is null with non-empty null_reason from allowed enum

Overlap Consistency

No two primary spans for the same entity type overlap by more than 20% of their combined length

Overlapping spans for same entity type exceed overlap threshold

Compute intersection-over-union for all same-type span pairs; flag any pair exceeding 0.2 IoU

Justification Grounding

= 90% of confidence_justification strings reference specific tokens or phrases from source text

Justification is generic, circular, or lacks source-text reference

Human or LLM-as-judge review of justification strings against source; score binary grounded/ungrounded per span

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Start with the base prompt and a flat list of spans. Remove strict schema enforcement and boundary justification requirements. Ask only for span, confidence, and boundary_flag. Use a simple JSON array output.

code
Extract spans for [ENTITY_TYPE] from [TEXT]. For each span, return:
- "span": the extracted text
- "confidence": 0.0 to 1.0
- "boundary_flag": "CLEAR" or "FUZZY"

Watch for

  • Model truncating long spans at punctuation boundaries without flagging
  • Confidence scores clustering at 0.8-1.0 with no mid-range values
  • Missing boundary flags on spans that clearly have ambiguous start/end positions
  • No alternative span candidates when boundaries are fuzzy
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.