Inferensys

Glossary

Confidence Score

A probability value output by a detection model indicating the likelihood that a predicted bounding box contains an object of a specific class and is accurately localized.
ML engineer managing model versions on laptop, version history visible, technical Git-like workflow.
PROBABILISTIC DETECTION METRIC

What is Confidence Score?

A confidence score is a probability value output by a detection model indicating the likelihood that a predicted bounding box contains an object of a specific class and is accurately localized.

In object detection in radiology, a confidence score is the model's self-assessment of a prediction's reliability, typically ranging from 0.0 to 1.0. It jointly quantifies the probability that a region contains a genuine lesion (classification certainty) and that the bounding box is precisely aligned with its spatial extent (localization certainty). This single scalar is the primary threshold for triggering clinical alerts in CADe systems.

The score is derived from the final layer activations, often via a softmax or sigmoid function, and is critical for Non-Maximum Suppression (NMS) to filter redundant detections. A high threshold (e.g., 0.9) maximizes precision by suppressing false positives, while a lower threshold increases sensitivity for small object detection. Calibration is essential, as raw scores can be overconfident, necessitating techniques like Focal Loss to handle extreme class imbalance.

PROBABILISTIC INTERPRETATION

Key Characteristics of Confidence Scores

A confidence score is not merely a threshold but a calibrated probability that quantifies a model's certainty in its own prediction, directly impacting clinical decision-making workflows.

01

Probabilistic Calibration

A well-calibrated confidence score of 0.9 implies that the prediction is correct 90% of the time. Modern architectures like DETR and YOLO output raw logits transformed via a sigmoid or softmax function. However, deep neural networks are often poorly calibrated out-of-the-box, exhibiting overconfidence.

  • Temperature Scaling: A post-hoc method that smooths probabilities without affecting accuracy.
  • Expected Calibration Error (ECE): The primary metric for measuring the gap between confidence and actual accuracy.
< 5%
Target ECE for Clinical Models
02

Threshold Tuning & ROC Analysis

The confidence score serves as a tunable dial to balance sensitivity (recall) against precision. In medical imaging, missing a lesion (false negative) carries a much higher clinical cost than a false alarm.

  • Operating Point Selection: Radiologists often prefer a high-sensitivity operating point (e.g., threshold > 0.1) for screening, even if it generates more false positives.
  • Precision-Recall Curve: Often more informative than the Receiver Operating Characteristic (ROC) curve in the context of extreme class imbalance where backgrounds dominate.
03

Confidence vs. Localization Accuracy

A high confidence score does not strictly guarantee precise spatial overlap. A model can be highly confident about a class while having a poor Intersection over Union (IoU) with the ground truth.

  • Box Uncertainty: Bayesian detection models output a variance alongside the coordinate prediction.
  • IoU-Aware Loss: Advanced loss functions penalize high-confidence predictions that have poor localization, forcing the score to reflect both classification certainty and spatial precision.
04

Non-Maximum Suppression (NMS) Dependency

The confidence score is the primary ranking criterion in Non-Maximum Suppression (NMS). When multiple overlapping bounding boxes predict the same lesion, the box with the highest confidence survives, and lower-scoring neighbors are suppressed.

  • Soft-NMS: Instead of zeroing out overlapping boxes, Soft-NMS decays their scores based on the IoU overlap, preventing the deletion of true adjacent lesions.
  • Confidence-Guided NMS: Ensures that the final output is not just the most overlapping box, but the most certain one.
05

Ensemble Confidence Fusion

Combining predictions from multiple models (e.g., Faster R-CNN and YOLO) requires sophisticated fusion of their confidence scores. Simple averaging is suboptimal if models have different calibration profiles.

  • Weighted Boxes Fusion (WBF): Merges boxes by averaging coordinates and confidence scores, producing a robust consensus score.
  • Test Time Augmentation (TTA): Averages the confidence scores of the same image under different flips and rotations to stabilize the final prediction.
06

FROC Analysis in Medical Context

The Free-Response Receiver Operating Characteristic (FROC) curve is the standard for evaluating detection confidence in radiology. Unlike standard ROC, FROC plots sensitivity against the average number of false positives per scan.

  • Confidence Threshold Sweep: The FROC curve is generated by sweeping the confidence threshold from 0 to 1.
  • Clinical Relevance: A score of 0.85 might yield 0.1 false positives per scan, while 0.65 might yield 2.0. The FROC curve visualizes this trade-off explicitly for regulatory review.
CONFIDENCE SCORE IN RADIOLOGY AI

Frequently Asked Questions

Clear, technically precise answers to the most common questions about how confidence scores function in medical object detection, their calibration, and their role in diagnostic workflows.

A confidence score is a probability value, typically ranging from 0.0 to 1.0, output by a detection model that quantifies the likelihood that a predicted bounding box contains an object of a specific class and is accurately localized. In radiology, this score represents the model's certainty that a detected region is a genuine abnormality—such as a nodule, lesion, or fracture—rather than a false positive. The score is derived from the final classification head of the detection architecture, often via a softmax or sigmoid activation function. For example, a model might output a confidence score of 0.92 for a lung nodule, indicating high certainty, while a score of 0.45 suggests ambiguity that warrants closer radiologist review. These scores are critical for tuning the sensitivity-specificity trade-off in Computer-Aided Detection (CADe) systems.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.