Superpixel Segmentation

Groups contiguous pixels into regions based on color similarity, texture homogeneity, and spatial proximity. Unlike rigid grid patches, superpixels adhere to object boundaries, preserving semantic structure. Algorithms like SLIC (Simple Linear Iterative Clustering) cluster pixels in a five-dimensional color-spatial space, balancing compactness with boundary adherence.

Superpixels replace raw pixels as the interpretable features for surrogate models in LIME. Instead of explaining which individual pixels matter—which is meaningless to humans—the explanation identifies which superpixel regions drove the prediction. Each superpixel becomes a binary feature: present or occluded during perturbation sampling.

A fundamental trade-off controlled by the compactness parameter in algorithms like SLIC:

• High compactness: Produces regularly shaped, grid-like superpixels that may cross object boundaries
• Low compactness: Produces irregular, boundary-hugging regions that better preserve semantic edges but vary wildly in size Optimal settings depend on the downstream explanation task.

During LIME's neighborhood generation, superpixels are turned on or off to create perturbed image variants. The masking replaces superpixel regions with a neutral value—typically mean pixel intensity, gray, or black—to simulate feature absence. This binary occlusion approach makes the surrogate model's training data semantically coherent rather than introducing high-frequency noise from random pixel masking.

Several algorithms produce superpixels with different characteristics:

• SLIC: Fast, simple, widely used in explanation pipelines; k-means in LabXY space
• Felzenszwalb-Huttenlocher: Graph-based approach with strong boundary adherence
• Quickshift: Mode-seeking algorithm that preserves local maxima in feature space
• Watershed: Morphological approach using gradient magnitude SLIC dominates explanation workflows due to its speed and tunable compactness.

Superpixels align with object parts rather than arbitrary pixel blocks. A superpixel might capture an entire wheel, a patch of sky, or a segment of fur—regions that carry semantic weight. This alignment ensures that when LIME identifies a superpixel as important, the explanation maps to a human-interpretable image region rather than an incomprehensible scatter of individual pixels.

The foundational k-means clustering algorithm adapted to the 5-dimensional [lab, xy] color-space and pixel-coordinate space. SLIC initializes cluster centers on a regular grid and iteratively refines them by assigning each pixel to the nearest center within a localized search region. This distance measure balances color similarity and spatial proximity, controlled by a compactness parameter. The result is a set of roughly equally-sized, compact superpixels that adhere tightly to image boundaries. SLIC is the default segmentation method in LIME for images due to its computational efficiency and predictable output.

A mode-seeking algorithm that approximates the medoid shift procedure for superpixel generation. Quickshift computes a Parzen density estimate over the joint color-spatial feature space, then connects each pixel to its nearest neighbor with a higher density within a defined kernel radius. This forms a tree structure where each branch terminates at a local density mode, producing superpixels of varying sizes that naturally adapt to image texture. Unlike SLIC, Quickshift does not require specifying the number of superpixels in advance, making it useful for images with heterogeneous detail.

A graph-theoretic approach that represents the image as an undirected graph where pixels are nodes and edges carry weights based on color dissimilarity. The algorithm merges regions using a minimum spanning tree criterion, comparing internal variation within a component to the inter-component difference. A scale parameter controls the preference for larger or smaller segments. This method produces superpixels that respect perceptual grouping principles and is particularly effective at capturing elongated, non-compact structures in natural scenes, though it is computationally heavier than SLIC.

Superpixels transform the raw pixel grid into a semantically meaningful feature representation for the local surrogate model. Each superpixel becomes a binary feature indicating its presence or absence. When LIME generates perturbed samples, it randomly turns superpixels on or off by replacing them with a neutral color or the image-wide mean. This creates a human-understandable vocabulary: the explanation states which contiguous image regions contributed most to the prediction, rather than listing individual pixel coordinates. The granularity of this representation directly controls the fidelity-interpretability trade-off.

A critical hyperparameter in SLIC that weights the relative importance of spatial proximity versus color similarity in the distance metric. A high compactness value forces superpixels into more regular, grid-like shapes that may cross object boundaries. A low value allows superpixels to follow color gradients and texture edges more faithfully, producing irregular but semantically coherent regions. Tuning this parameter is essential for explanation quality: overly compact superpixels may group semantically distinct areas, while overly flexible ones can fragment coherent objects, degrading the interpretability of the resulting saliency mask.

The final visual output of an image explanation pipeline, where the feature importance scores from the local surrogate model are mapped back onto the superpixels. Each superpixel is colored according to its contribution weight—typically using a heatmap gradient where red indicates positive contribution toward the predicted class and blue indicates negative contribution. The saliency mask overlays the original image, allowing a human auditor to instantly see which regions the model relied upon. This visualization is the primary deliverable for debugging convolutional neural network decisions in medical imaging, autonomous driving, and content moderation systems.