Neuron Conductance

Neuron Conductance quantifies how much a hidden neuron contributes to the model's output change from a baseline. It is defined as:

• Formula: The integral of the partial derivative of the output with respect to the neuron, multiplied by the neuron's gradient with respect to the input, along the straight-line path from baseline to input.
• Decomposition: The sum of conductance scores across all neurons in a layer equals the total Integrated Gradients attribution for that layer.
• Path Integral: Like Integrated Gradients, it satisfies the completeness axiom, ensuring attributions sum to the output difference.

Conductance is a direct extension of Integrated Gradients that shifts the focus from input features to hidden neurons:

• Input Attribution: Integrated Gradients computes the importance of each input feature directly.
• Neuron Attribution: Conductance computes the importance of each hidden neuron by measuring the gradient flow through it.
• Layer-wise Summation: Summing conductance across all neurons in a layer recovers the layer's total attribution.
• Hierarchical Analysis: Enables debugging at multiple levels—from individual neurons to entire layers—providing a granular view of internal decision-making.

Conductance inherits the completeness axiom from Integrated Gradients, ensuring mathematical consistency:

• Conservation Property: The sum of conductance values for all neurons in a network equals the difference between the model's output at the input and the baseline.
• No Attribution Leakage: Every unit of output change is fully accounted for by the neurons that contributed to it.
• Auditability: This property makes conductance suitable for compliance contexts where a complete decomposition of the prediction is required for regulatory review.

Conductance differs fundamentally from methods that rely solely on neuron activations:

• Activation Magnitude: A neuron may have a high activation value but contribute little to the final prediction if its downstream weights are near zero.
• Gradient Flow: Conductance captures the actual influence of the neuron on the output by combining activation with gradient information.
• Saturation Handling: Unlike raw gradient methods, conductance avoids the gradient saturation problem by integrating along a path, ensuring neurons that strongly activate the correct class are not incorrectly assigned zero importance.

Conductance enables systematic debugging of deep networks by isolating problematic layers:

• Bottleneck Identification: Layers with unexpectedly low total conductance may indicate vanishing gradients or dead neurons.
• Redundancy Detection: Layers where many neurons have near-zero conductance suggest over-parameterization and potential for pruning.
• Concept Attribution: When combined with Concept Activation Vectors, conductance can measure how much a neuron contributes to a high-level concept, bridging mechanistic and concept-based interpretability.
• Architecture Comparison: Engineers can compare conductance distributions across different model architectures to understand why one design outperforms another.

Practical implementation of conductance requires careful numerical approximation:

• Riemann Approximation: The path integral is approximated by summing gradients at discrete steps along the straight-line path, typically 20-300 steps.
• Baseline Selection: Like Integrated Gradients, the choice of baseline (e.g., zero embedding, black image) significantly impacts the explanation and must be chosen to represent the absence of signal.
• Computational Cost: Conductance requires a backward pass for each integration step, making it more expensive than single-pass methods like Gradient × Input.
• Batching: Multiple steps can be batched for efficient GPU computation, reducing wall-clock time.