BIO Tagging

The scheme assigns one of three tags to every token in a sentence:

• B-[TYPE]: Marks the Beginning token of an entity (e.g., B-PER for the first name).
• I-[TYPE]: Marks Inside tokens of a multi-word entity (e.g., I-PER for a surname).
• O: Marks tokens Outside any named entity.

This explicitly encodes entity boundaries, distinguishing "New York" (B-LOC, I-LOC) from two separate locations.

The primary advantage of BIO over simpler IO tagging is the resolution of adjacent entities of the same type. Without the B- prefix, the sequence I-PER I-PER could represent one entity or two.

BIO enforces that every new entity starts with a B- tag. Therefore, "President Jane Doe met John Smith" is correctly tagged as:

• Jane/B-PER, Doe/I-PER
• John/B-PER, Smith/I-PER

This prevents the model from merging distinct entities into one incorrect span.

An extended scheme called BILOU (or BIOES) adds two more tags for finer-grained span representation:

• L-[TYPE]: Last token of an entity (e.g., L-PER for a single-word entity or the final token).
• U-[TYPE]: Unit or singleton entity, a single token that is both beginning and end.

This provides explicit end-boundary signals, which can improve performance on span-level tasks but increases the total number of output classes the classifier must learn.

BIO tagging assumes a 1:1 mapping between tokens and labels, but modern subword tokenizers like Byte-Pair Encoding (BPE) or WordPiece fragment words. The word "Johansson" might become ["Johan", "##sson"].

Standard mitigation strategies include:

• Assigning the original entity label only to the first subword token and giving subsequent subwords a special ignore label during loss calculation.
• Using span-level masking to compute loss only on whole-entity reconstructions.

Misalignment here is a common source of silent evaluation errors.

BIO tags are treated as categorical labels in a many-to-many sequence prediction task. A linear-chain Conditional Random Field (CRF) is often stacked on top of neural encoders like BERT to model the transition constraints between adjacent tags.

The CRF learns valid transitions (e.g., I-PER cannot follow O without a B-PER) and prevents illegal sequences during Viterbi decoding, enforcing the structural integrity of the BIO scheme at inference time.

Performance is not evaluated on token-level accuracy due to the heavy class imbalance toward the O tag. Instead, the BIO tag sequence is decoded back into entity spans, and Mention-Level F1 is computed.

A predicted entity is considered a true positive only if both its span boundaries (start and end tokens) and its entity type match the gold standard annotation exactly. A single token shift (e.g., missing a determiner) results in a false positive and a false negative.

A discriminative probabilistic model that serves as the decoding layer on top of neural networks for sequence labeling. CRFs model the conditional probability of an entire label sequence given the input tokens, enforcing valid transitions between BIO tags.

• Prevents impossible sequences like an I-ORG tag following an O tag
• Uses Viterbi decoding to find the globally optimal tag path
• Captures dependencies between adjacent labels that independent classifiers miss

A modern NER paradigm that bypasses BIO tagging entirely by directly enumerating and classifying arbitrary text spans as entities. Instead of labeling every token, the model scores all possible start-end token pairs.

• Eliminates the sequence labeling constraint and invalid tag transition problem
• Naturally handles nested entities that BIO schemes struggle with
• Architectures like Global Pointer use multiplicative attention to score span boundaries

The standard evaluation metric for sequence labeling tasks that computes the harmonic mean of precision and recall based on exact span matching. A prediction is correct only if both the entity boundaries and the type classification match the gold standard.

• True Positive: Exact match of span and type
• False Positive: Predicted entity that does not match any gold entity
• False Negative: Gold entity missed entirely by the model

An extended tagging scheme that adds End and Single tags to the standard BIO format, providing explicit boundary information. This richer encoding helps models learn entity boundaries more effectively.

• B-PER: Beginning of a person entity
• I-PER: Inside a person entity
• E-PER: End of a person entity
• S-PER: Single-token person entity
• O: Outside any entity

The explicit end markers reduce ambiguity for multi-token entities.

A dynamic programming algorithm that efficiently finds the most probable sequence of hidden states in a linear-chain CRF. During inference, Viterbi computes the maximum over all possible BIO tag paths rather than greedily selecting the highest-scoring tag at each position.

• Guarantees globally optimal tag sequence under the model
• Enforces valid transitions like B-ORG → I-ORG → E-ORG
• Time complexity is O(n * s²) where n is sequence length and s is number of states

The benchmark dataset that established the standard evaluation methodology for NER using BIO tagging. It defines four entity types across English and German newswire text.

• PER: Person names
• ORG: Organizations
• LOC: Locations
• MISC: Miscellaneous entities

The dataset's IOB2 tagging format became the de facto standard for training and evaluating sequence labeling models.