Nil Prediction

The core logic that distinguishes an unlinkable mention from a low-confidence link. Rather than selecting the highest-scoring candidate regardless of score, the system compares the top candidate's confidence against a global or context-sensitive rejection threshold.

• Absolute Threshold: A fixed score (e.g., 0.65) below which all candidates are rejected.
• Adaptive Threshold: A dynamic cutoff calibrated per entity type or mention frequency.
• Margin-Based Rejection: Requires the top candidate to outscore the second-best by a minimum delta, preventing ambiguous selections.

If no candidate surpasses the threshold, the system returns a NIL identifier instead of a false KB entry.

The explicit classification task of recognizing that a textual mention refers to an entity not yet recorded in the target knowledge base. This is distinct from simple disambiguation failure.

• Zero-Shot Setting: The system must generalize to entirely unseen entity names without retraining.
• Open-World Assumption: The model operates under the premise that the KB is incomplete and new entities continuously emerge.
• Feature Signals: Unusual surface forms, lack of prior co-occurrence with known entities, and low semantic similarity to any KB entry vector all contribute to the detection signal.

Effective detection prevents knowledge base pollution with spurious links.

A retrieval architecture that explicitly includes a learned NIL representation alongside valid KB entity candidates during the candidate generation phase.

• NIL Embedding Vector: A trainable dense vector representing the 'no-entity' class, placed in the same semantic space as real entity embeddings.
• Similarity Scoring: The mention embedding is compared against both real entity vectors and the NIL vector; the system predicts NIL if the NIL vector yields the highest similarity.
• Contrastive Training: The model is trained with negative samples that include both hard-negative entities and explicit NIL examples to sharpen the decision boundary.

This transforms NIL prediction from a post-hoc threshold check into a first-class classification outcome.

The statistical process of ensuring that a model's predicted probability of a mention being linkable accurately reflects the true likelihood of correctness. A well-calibrated model is essential for reliable NIL prediction.

• Expected Calibration Error (ECE): A metric measuring the discrepancy between predicted confidence and empirical accuracy across bins.
• Temperature Scaling: A post-processing technique that adjusts the sharpness of the softmax distribution to improve calibration without changing predictions.
• Reliability Diagrams: Visual plots comparing average confidence against observed accuracy to diagnose overconfidence or underconfidence.

Poor calibration leads to overconfident false links or excessive, unnecessary NIL rejections.

A validation layer that uses fine-grained entity typing to verify that a candidate entity is semantically compatible with the mention's predicted type, even if the string match appears strong.

• Type Hierarchy Check: If a mention is typed as 'ORG' but the top candidate is a 'PERSON', the system can override the link and predict NIL.
• NER Signal Integration: The output of a Named Entity Recognition model provides a coarse type constraint that filters incompatible KB entries before confidence scoring.
• Ontological Consistency: Ensures that linked entities satisfy the domain and range constraints of the relations they participate in within the knowledge graph.

This prevents category errors that pure string-matching or embedding similarity might miss.

Specialized metrics beyond standard precision and recall that specifically measure a system's ability to correctly identify unlinkable mentions.

• NIL Precision: The fraction of predicted NILs that are truly unlinkable (avoiding false rejection of valid links).
• NIL Recall: The fraction of true unlinkable mentions that the system correctly identifies as NIL (avoiding false positive links).
• Link-Link-NIL Accuracy: A three-way classification metric evaluating correct KB links, correct NIL predictions, and their associated error types.
• F1-NIL: The harmonic mean of NIL precision and recall, providing a single balanced score for rejection quality.

Standard entity linking F1 scores can be misleadingly high if the system simply never predicts NIL on a dataset with few unlinkables.