Ground Truth

Labels must represent the objectively correct answer for each data point, verified through measurement or expert consensus. In manufacturing, this often means physical metrology validation—a micrometer measurement confirming a dimensional defect, not a subjective visual guess.

• Single source of truth: Labels derived from calibrated instruments, not human opinion
• Verification protocol: Multi-expert adjudication with tie-breaking mechanisms
• Measurement traceability: Labels linked to NIST-traceable standards where applicable

Ground truth must span the full distribution of real-world variation the model will encounter. A dataset labeling only common defect types will produce a model blind to rare but critical failures.

• Edge case inclusion: Rare defects, boundary conditions, and corner cases explicitly represented
• Class balance awareness: Deliberate sampling strategies to prevent majority-class dominance
• Environmental diversity: Labels covering all lighting conditions, material batches, and orientations present in production

Inter-annotator agreement must be measured and enforced. When two experts disagree on whether a surface blemish constitutes a critical defect versus a cosmetic variation, the model inherits that ambiguity.

• Annotation guidelines: Formal, documented criteria with visual reference examples
• Cohen's Kappa coefficient: Statistical measure of inter-rater agreement beyond chance
• Regular calibration sessions: Ongoing alignment meetings to prevent standard drift over time

Label granularity must match the business requirement. Bounding boxes may suffice for presence detection, but pixel-wise segmentation masks are necessary when defect boundary measurements determine pass/fail criteria.

• Classification: Image-level labels for sorting conforming vs. non-conforming parts
• Object detection: Bounding boxes for defect localization and counting
• Semantic segmentation: Pixel-level masks for precise area and shape measurement
• Instance segmentation: Distinguishing individual, overlapping defects

Ground truth is not static. As product specifications evolve and new defect modes emerge, labels must be version-controlled and updated. Training against stale ground truth produces models that reject conforming parts or pass defective ones.

• Dataset versioning: Immutable snapshots with semantic versioning (e.g., v2.1.0)
• Change logs: Documented rationale for label modifications between versions
• Reproducibility: Any model trained on a specific ground truth version can be exactly recreated

Ground truth used for evaluation must be strictly held out from the training process. Leakage—where test data influences training—produces misleadingly optimistic performance metrics that collapse in production.

• Train/validation/test splits: Enforced separation before any preprocessing
• Temporal splitting: Future production data reserved for testing to simulate real deployment
• Blind evaluation: Test set labels never inspected during model development or hyperparameter tuning

A technique to artificially expand a ground truth dataset by applying realistic transformations—rotation, scaling, shearing, color jitter—to original labeled images. The generated images inherit the original label.

• Purpose: Improves model robustness to lighting and orientation variance on the factory floor
• Constraint: Transformations must preserve the validity of the label; a rotated defect is still a defect
• Example: A scratch annotation rotated 15 degrees remains a scratch ground truth
• Risk: Over-aggressive augmentation can create unrealistic images that corrupt the objective standard

The percentage of conforming, non-defective products incorrectly classified as defective when compared to ground truth labels. This metric represents a direct operational cost.

• Calculation: False Positives / Total Actual Negatives
• Business Impact: Unnecessary scrap, rework, and line stoppage
• Root Cause: Often caused by overly sensitive thresholds or biased ground truth that labeled borderline cases as defects
• Trade-off: Reducing FRR typically increases the Escape Rate, requiring a deliberate business decision based on the cost of a miss

The percentage of actual defective products that are incorrectly classified as conforming and pass through inspection undetected. This is the most critical failure mode measured against ground truth.

• Calculation: False Negatives / Total Actual Positives
• Business Impact: Customer complaints, warranty claims, and regulatory non-compliance
• Detection Gap: Escapes often occur on rare defect types underrepresented in the ground truth dataset
• Mitigation: Requires targeted synthetic data generation and active learning to fill gaps in the labeled reference data

The degradation of a model's predictive performance over time because the statistical properties of live production data diverge from the original ground truth distribution. The objective standard becomes stale.

• Causes: New defect morphologies, gradual lighting changes, tooling wear, different raw material batches
• Detection: Monitoring the divergence between production inference distributions and the training data distribution
• Remediation: Requires re-labeling a sample of new production data to establish an updated ground truth
• Metric: A rising FRR or Escape Rate often signals that the ground truth no longer represents current reality