Semantic Segmentation

Assigns a class label to every pixel in the input image without exception. This produces a complete segmentation map of identical resolution to the input, where each pixel value corresponds to a predefined class index. For manufacturing, this means a scratch defect is not just located but precisely outlined at the pixel level, enabling accurate area measurement. Architectures like U-Net and DeepLab are purpose-built for this task, using encoder-decoder structures to capture both global context and fine spatial details.

All pixels belonging to the same semantic class are assigned the identical label, regardless of whether they represent one object or multiple disconnected regions. Two separate scratches on a metal surface both receive the 'scratch' label with no differentiation between them. This distinguishes semantic segmentation from instance segmentation, which assigns unique identifiers to each individual object. For quality inspection, this property is ideal when the total defect area matters more than counting individual defect instances.

Modern semantic segmentation models employ a symmetric encoder-decoder structure. The encoder, typically a CNN or Vision Transformer, progressively downsamples the image to extract hierarchical features and capture broad contextual understanding. The decoder upsamples these features back to the original resolution, reconstructing spatial detail. Skip connections between corresponding encoder and decoder layers preserve fine-grained boundary information that would otherwise be lost during downsampling, ensuring crisp defect edges.

Effective segmentation requires understanding both the fine texture of a defect and its relationship to the surrounding surface. Techniques like atrous spatial pyramid pooling (ASPP) and spatial pyramid pooling capture features at multiple receptive field sizes in parallel. This allows the model to simultaneously analyze a small pit defect at fine scale while recognizing that it sits within a larger machined surface region. Multi-scale processing is critical for handling defects that vary dramatically in size within the same production line.

Defect pixels often represent less than 1% of total image area, creating severe class imbalance. Standard cross-entropy loss fails here, as the model can achieve high accuracy by simply predicting 'background' everywhere. Specialized loss functions address this: Dice loss optimizes the overlap coefficient between prediction and ground truth directly. Focal loss down-weights easy background examples and forces the model to focus on hard, misclassified defect pixels. Tversky loss provides tunable control over precision-recall tradeoffs.

Raw segmentation outputs often produce coarse, blob-like boundaries that fail to capture fine defect contours. Conditional Random Fields (CRFs) serve as a post-processing step that refines label assignments by modeling pairwise pixel relationships based on color similarity and spatial proximity. This encourages pixels with similar appearance to share the same label while respecting strong intensity gradients at true boundaries. Though increasingly replaced by learned refinement modules in modern architectures, CRFs remain valuable for applications requiring sub-pixel boundary accuracy.