Cross-Modal Attention Flow

Tracks how attention weights flow from one transformer layer to the next through residual connections. The raw attention matrix at layer l is combined with the identity matrix to account for the skip connection, forming the attention rollout baseline. This ensures that tokens attending to themselves in early layers are not lost in the aggregation. The propagation follows: Ã(l) = 0.5 · A(l) + 0.5 · I, where A(l) is the raw attention matrix and I is the identity matrix, before multiplying across layers to trace the complete flow.

The core mathematical operation for aggregating attention flow across layers. Starting from a modality-specific input, the rollout matrix is computed by multiplying the attention matrices from successive layers: rollout = A(l_start) · A(l_start+1) · ... · A(l_end). For cross-modal flow, this traces how information from an image patch token propagates through self-attention and cross-attention layers to influence a text token, producing a single heatmap of cross-modal influence.

Multi-head attention mechanisms compute parallel attention distributions at each layer. Cross-modal flow analysis must decide whether to average heads before propagation or track each head independently. Averaging heads (mean pooling) provides a consolidated view of information flow but may obscure specialized head functions. Independent tracking reveals that specific heads act as cross-modal bridges, funneling visual information to textual representations while other heads perform unimodal processing.

Raw cross-modal attention flow values often exhibit a long-tailed distribution, where a few token pairs dominate the signal. To produce interpretable visualizations, practitioners apply min-max normalization to scale values to [0,1] and percentile-based thresholding to filter noise. A common approach is to display only the top 10-20% of attention connections, revealing the dominant cross-modal pathways without visual clutter from spurious low-weight associations.

Cross-modal attention flow is inherently directional. The analysis distinguishes between vision-to-text flow (how image regions influence word representations) and text-to-vision flow (how linguistic queries shape visual attention). In vision-language models, these directions reveal asymmetric information transfer: text-to-vision flow often shows sharp, localized grounding, while vision-to-text flow exhibits broader, context-aggregating patterns that distribute visual information across multiple textual tokens.

Two distinct methods for aggregating attention across layers. Attention rollout linearly combines matrices assuming attention is a linear operation, which is computationally efficient but ignores non-linearities from MLP layers. Attention flow (or maximum flow) treats the network as a flow network, using max-flow min-cut algorithms to find the strongest paths through the attention graph. Flow-based methods better capture sparse, high-impact cross-modal connections but are significantly more computationally expensive.

The direct visual output of attention flow analysis. These heatmaps display the attention weights between tokens from different modalities—for instance, showing which image patches a text token like 'dog' attended to. They provide the raw, layer-by-layer visualization of vision-language grounding before aggregation techniques like rollout are applied.

A specific computational method for tracing information propagation. Rollout linearly combines attention matrices across all transformer layers to account for the mixing of attention in deeper layers. This produces a single, unified map of how information from a visual token flows to a textual token through the entire architecture, correcting for the raw attention's tendency to focus on low-level features.

A causal intervention technique that directly measures cross-modal reliance. By systematically removing or zeroing out an entire modality (e.g., blanking the image) and observing the change in prediction confidence, engineers can quantify the causal contribution of each data stream. This validates whether the attention flow patterns identified by observational methods actually drive the model's decision-making.

A backpropagation-based attribution method that conserves relevance as it decomposes the prediction score. Unlike attention, which may not directly correlate with feature importance, MLRP propagates the output signal backward through the network, assigning a relevance score to every input feature in every modality. It satisfies a conservation principle, ensuring no relevance is lost or created during the explanation process.

An advanced technique from causal inference applied to model internals. It identifies specific cross-modal neurons or attention heads that function as causal mediators. By intervening on a neuron's activation and measuring the effect on the output, this method moves beyond correlation to establish whether a particular attention flow pattern is a necessary and sufficient cause of the model's behavior.

A diagnostic scalar metric that quantifies the focus or dispersion of cross-modal attention heads. High entropy indicates a broad, unfocused gaze across the image, while low entropy signals sharp, specific grounding on a few patches. Tracking entropy across layers helps identify where the model transitions from broad perceptual processing to focused semantic grounding, diagnosing potential attention flow pathologies.