Precision-Recall AUC

The curve plots precision against recall at every possible decision threshold, revealing the inherent tension between these two metrics. As recall increases, precision typically decreases, and the area under this curve (AUC) provides a single scalar value summarizing this trade-off. A perfect classifier achieves an AUC of 1.0, while a random classifier achieves a value equal to the positive class ratio.

Unlike the ROC AUC, the Precision-Recall AUC is insensitive to the true negative rate, making it highly informative when the negative class vastly outnumbers the positive class. In anomaly detection for network telemetry, where failures may represent less than 0.1% of observations, a model could achieve a 99.9% ROC AUC by simply predicting 'normal' every time. The PR AUC correctly penalizes this behavior by focusing exclusively on the model's ability to find the rare positive instances.

The area under the precision-recall curve is formally known as Average Precision (AP). It approximates the weighted mean of precision values achieved at each recall threshold, with the weight being the increase in recall from the prior threshold. This calculation ensures that the metric emphasizes performance in high-recall regions, which are critical for anomaly detection use cases where missing a fault is far more costly than a false alarm.

A key advantage of the PR AUC is that it evaluates model performance independently of any single operating point. In production anomaly detection systems, the optimal threshold for flagging an alert depends on business context and cost ratios that may shift over time. The PR AUC provides a holistic measure of ranking quality across all possible thresholds, allowing teams to compare model architectures before committing to a specific decision boundary.

The baseline for the PR AUC is not a fixed value like 0.5 but is equal to the positive class prevalence in the dataset. For a dataset with 1% anomalies, a random classifier achieves a PR AUC of 0.01. When plotting, the curve is often interpolated using the 'all-points' method rather than linear interpolation to avoid over-optimistic estimates in regions where precision-recall data is sparse, providing a more conservative and realistic performance estimate.

The foundational visualization that plots precision against recall at every possible classification threshold. Unlike the ROC curve, it focuses exclusively on the positive class, making it immune to the class imbalance paradox. A model that dominates another in PR space is strictly better at identifying the minority class. The curve typically starts at (1.0, low recall) and descends toward (low precision, 1.0).

A single-number summary of the precision-recall curve, calculated as the weighted mean of precisions achieved at each threshold, with the increase in recall from the previous threshold used as the weight. This is mathematically equivalent to the area under the precision-recall curve when using the interpolation method. AP penalizes models that fail to retrieve relevant items at high ranks.

The F1 score is the harmonic mean of precision and recall at a single, fixed threshold. In contrast, PR AUC evaluates performance across all thresholds. F1 is useful when you have a specific operating point, but PR AUC is preferred for model selection because it captures the global trade-off. A model with a higher PR AUC may have a lower F1 if the chosen threshold is suboptimal.

On datasets where the positive class constitutes less than 1% of samples, ROC AUC can be deceptively high (e.g., 0.95) even for a poor model. PR AUC exposes this weakness. Key principles:

• A random classifier's PR AUC equals the positive class prevalence
• Always report PR AUC alongside ROC AUC for imbalanced problems
• Use stratified sampling to preserve class ratios in train/test splits

The standard 11-point interpolated average precision samples precision at 11 recall levels (0.0, 0.1, ..., 1.0) and averages them. Modern implementations use all-point interpolation, which examines every unique recall value. The all-point method is more precise and is the standard in benchmarks like PASCAL VOC and COCO. For network telemetry anomaly detection, all-point interpolation captures fine-grained performance differences.