Linking Confidence Score

A well-calibrated confidence score of 0.8 implies that the system expects to be correct 80% of the time across predictions with that score. Platt scaling and isotonic regression are common post-hoc calibration methods applied to raw model logits. Poor calibration leads to overconfident errors, making threshold tuning unreliable.

• Raw neural network outputs are often uncalibrated
• Expected Calibration Error (ECE) measures miscalibration
• Temperature scaling is a simple, effective fix for modern models

The confidence score serves as a decision boundary. Setting a high threshold like 0.95 maximizes precision by only accepting near-certain links, but reduces recall by discarding many correct mentions. A lower threshold like 0.5 captures more entities but introduces noise. The optimal threshold is application-specific.

• High threshold: Critical for medical or legal entity normalization
• Low threshold: Useful for exploratory search or recall-oriented tasks
• F1 score optimization finds the harmonic balance point

The confidence score is the primary mechanism for NIL prediction. If the highest candidate score falls below a predefined NIL threshold, the system predicts that the mention refers to an Out-of-KB entity (OOKB). This prevents false linking to superficially similar but incorrect knowledge base entries.

• NIL threshold is typically lower than the linking threshold
• Prevents 'forced linking' to wrong entities
• Critical for maintaining knowledge base quality in automated pipelines

In architectures like BLINK, the final confidence score is a composite. The Bi-Encoder produces a fast dot-product similarity for candidate retrieval, while the Cross-Encoder Reranker generates a high-precision relevance score through full cross-attention. The final score synthesizes both contextual similarity and prior probability.

• Bi-Encoder score: Fast, symmetric, used for top-K retrieval
• Cross-Encoder score: Slow, asymmetric, used for final ranking
• Prior probability (commonness) acts as a strong baseline feature

The confidence score resolves ambiguity by balancing prior probability (how often a surface form links to an entity) against contextual similarity (how well the surrounding text matches the entity description). For the mention 'Apple' near 'iPhone', the contextual signal overrides the high prior for the fruit company, boosting the score for Apple Inc.

• Prior dominates for unambiguous, high-commonness mentions
• Context dominates when surrounding text is highly informative
• Conflict between signals lowers overall confidence

In collective entity linking, individual mention confidence scores are iteratively updated based on the semantic coherence of the entire set. A low-confidence link may be boosted if the candidate entity shares strong knowledge graph relationships with other high-confidence entities in the document. This global normalization improves overall accuracy.

• Uses algorithms like Personalized PageRank on the knowledge graph
• Resolves ambiguous mentions by maximizing topical coherence
• Final scores reflect both local evidence and global agreement