Explainability (XAI)

A game-theoretic approach to explain the output of any machine learning model. SHAP connects optimal credit allocation with local explanations using classic Shapley values.

• Mechanism: Uses Shapley values to compute the marginal contribution of each feature to a prediction, ensuring consistent and locally accurate attribution.
• Manufacturing Use: Pinpoints exactly which sensor readings (e.g., vibration frequency, temperature spike) contributed most to a predictive maintenance alert.
• Key Property: Additive feature attribution ensures the sum of all feature contributions equals the difference between the prediction and the average prediction.

A technique that explains individual predictions by approximating the complex model locally with an interpretable surrogate model, such as a linear regression or decision tree.

• Mechanism: Perturbs the input data around a specific instance, observes how predictions change, and fits a simple, explainable model to that local neighborhood.
• Manufacturing Use: Explains a single false-positive defect classification by highlighting the specific pixel regions in an image that led the model to incorrectly flag a scratch.
• Key Limitation: Explanations are local and may not represent the model's global behavior.

A gradient-based attribution method designed for deep neural networks that satisfies the axioms of sensitivity and implementation invariance, providing a principled way to assign importance to input features.

• Mechanism: Computes the average gradient of the model's output with respect to the input along a straight-line path from a baseline (e.g., a black image) to the actual input.
• Manufacturing Use: Generates a pixel-level heatmap overlaid on a product image, showing exactly which visual features activated the 'defect' neuron in a vision transformer.
• Axiomatic Guarantee: If a feature differs from the baseline and changes the output, it receives a non-zero attribution.

A method specific to transformer architectures that visualizes the internal attention weights to reveal which parts of an input sequence the model focuses on when making a decision.

• Mechanism: Extracts and renders the self-attention matrices from specific layers and heads of a transformer model, showing pairwise relationships between tokens or image patches.
• Manufacturing Use: Reveals that a foundation model analyzing a maintenance log focused on the phrase 'intermittent grinding noise' to generate a bearing failure diagnosis.
• Caveat: Attention weights are not a direct causal explanation but provide a useful signal for model debugging and hypothesis generation.

An explanation method that identifies the minimal changes required to an input instance to alter a model's prediction to a predefined, desired outcome.

• Mechanism: Solves an optimization problem to find the smallest perturbation to the input features that flips the classification boundary.
• Manufacturing Use: Tells a process engineer, 'If the injection pressure were reduced by 5 PSI, this part would have been classified as non-defective,' providing a direct, actionable insight.
• Key Benefit: Explanations are inherently contrastive and align with how humans naturally ask for explanations ('Why X instead of Y?').

A technique that provides explanations in terms of high-level, human-friendly concepts rather than low-level input features, enabling interpretability at the right level of abstraction.

• Mechanism: Defines a concept (e.g., 'scratched surface') using a set of example images, learns a vector in the model's activation space that represents that concept, and measures its influence on predictions.
• Manufacturing Use: Quantifies the sensitivity of a quality inspection model to the high-level concept of 'surface roughness' rather than individual pixel values.
• Key Advantage: Allows domain experts to test hypotheses about model behavior using their own vocabulary and conceptual understanding of the manufacturing process.

A game-theoretic approach to feature attribution that assigns each input variable an importance value for a particular prediction. SHAP values are mathematically grounded in cooperative game theory, ensuring consistent and locally accurate explanations.

• Computes the marginal contribution of each feature
• Unifies several explanation methods under one framework
• Critical for answering: 'Why did the model flag this weld as defective?'
• Provides both global model interpretability and per-prediction explanations

A technique that explains individual predictions by approximating the complex model locally with a simpler, interpretable surrogate model. LIME perturbs the input data and observes how predictions change to build a faithful local explanation.

• Model-agnostic: works on any black-box classifier
• Generates explanations in terms humans understand
• Useful for validating a single defect classification decision
• Limitations include instability of explanations across repeated runs

A visualization technique, primarily for computer vision models, that highlights the regions of an input image most influential to the model's decision. Saliency maps compute the gradient of the output class score with respect to the input pixels.

• Reveals where a quality inspection model is 'looking'
• Detects if the model focuses on the product or irrelevant background
• Variants include Grad-CAM and Integrated Gradients
• Essential for debugging vision-based defect classifiers

An explanation method that identifies the minimal changes required to an input to alter the model's prediction. A counterfactual answers: 'What would need to be different for the outcome to change?'

• Generates actionable feedback: 'Increase pressure by 5 PSI to pass QC'
• Directly supports process optimization and root cause analysis
• Can be constrained to produce realistic, physically possible scenarios
• Bridges the gap between explanation and prescriptive guidance

A technique specific to transformer-based foundation models that visualizes the self-attention weights between tokens or image patches. Attention maps reveal which parts of the input the model considers related when making a decision.

• Shows token-to-token relationships in maintenance logs
• Reveals cross-modal attention in multimodal models
• Helps validate that the model's reasoning path is sensible
• Caution: high attention does not always equal high importance

An interpretability paradigm that explains model decisions in terms of high-level, human-understandable concepts rather than raw input features. Concept activation vectors test whether a model has learned abstract ideas like 'roughness' or 'misalignment.'

• Aligns explanations with domain expert vocabulary
• Tests for unintended concept associations or biases
• Enables auditing at the semantic level, not just pixel level
• Key for validating that a foundation model understands manufacturing concepts correctly