Sensitivity Analysis

Sensitivity analysis operates on two distinct scales:

• Local Sensitivity Analysis (LSA): Examines output variation around a single, specific nominal input point, typically using partial derivatives or gradients. It answers, 'How does a small perturbation at this exact point change the output?'
• Global Sensitivity Analysis (GSA): Explores the entire input space, often using Monte Carlo methods, to apportion output variance across all inputs simultaneously. It answers, 'Which inputs drive output uncertainty across all plausible scenarios?' GSA is preferred for non-linear models where interactions between inputs are significant.

The most intuitive but limited approach, where a single input factor is varied while all others are held constant at their baseline values.

• Mechanism: A tornado plot is a common visualization, ranking inputs by the magnitude of output change they induce.
• Critical Limitation: OAT methods fail to detect interaction effects between inputs. A model's sensitivity to factor A might depend entirely on the level of factor B, a relationship OAT completely misses.
• Use Case: Suitable for a quick, preliminary screening of a linear or near-linear model, but insufficient for rigorous uncertainty quantification in complex systems.

A class of global methods that decomposes the total variance of the model output, V(Y), into contributions from individual inputs and their interactions.

• First-Order Sensitivity Index (S_i): Measures the main effect of input X_i alone, calculated as V(E[Y|X_i]) / V(Y). A high S_i means fixing X_i would significantly reduce output variance.
• Total-Effect Sensitivity Index (S_Ti): Measures the contribution of X_i including all its interactions with any other inputs. It is calculated as E[V(Y|X_{~i})] / V(Y).
• Sobol' Indices: The standard method for computing these indices, requiring a specific sampling design. A large difference between S_Ti and S_i indicates strong interaction effects for that input.

For differentiable models like neural networks, sensitivity is directly computed from the gradient of the output with respect to the input.

• Saliency Maps: In computer vision, the absolute value of the gradient for each pixel creates a saliency map, highlighting which pixels most influence the classification score.
• Integrated Gradients: A more robust method that addresses gradient saturation. It computes the average gradient along a straight-line path from a baseline input (e.g., a black image) to the actual input, satisfying the completeness axiom where attributions sum to the output difference.
• Limitation: These methods are inherently local and explain sensitivity around a single prediction, not global variance.

A screening method designed for models with a large number of inputs or high computational cost, where full variance decomposition is infeasible.

• Mechanism: It computes multiple elementary effects—the ratio of output change to input change—at randomly sampled points across the input space.
• Output Metrics: The mean of the absolute elementary effects (μ*) estimates the overall importance of a factor, while the standard deviation (σ) detects non-linear effects and interactions with other inputs.
• Classification: The Morris method classifies inputs into three groups: those with negligible effects, those with linear and additive effects, and those with non-linear or interaction effects, guiding further, more costly analysis.

Unlike variance-based methods that summarize uncertainty with a single moment, these techniques analyze the entire output distribution.

• Density-Based Sensitivity: Measures the shift in the output's probability density function when an input is fixed, often using the Kullback-Leibler divergence or a similar distance metric between the unconditional and conditional output distributions.
• PAWN Method: A moment-independent approach that uses Cumulative Distribution Functions (CDFs) . It measures sensitivity by the distance between the unconditional output CDF and the CDFs conditioned on fixing an input at different values.
• Advantage: These methods are more sensitive to changes in higher-order moments (skewness, tail behavior) that variance-based indices might miss, making them valuable for risk analysis where tail events are critical.