Cross-Modal Attention Maps

Cross-modal attention maps capture the asymmetric flow of information between modalities. In a typical vision-language transformer, text-to-image attention reveals which image patches a word attends to, while image-to-text attention shows which words an image region influences. This bidirectionality allows engineers to trace how a caption grounds itself in visual evidence and, conversely, how visual features are contextualized by language. Analyzing both directions is critical for diagnosing failures like visual hallucination, where a model generates text unsupported by the image.

Attention maps exhibit a characteristic coarse-to-fine progression across transformer layers:

• Early layers: Attention is broad and diffuse, covering large, semantically related regions (e.g., the entire background for 'outdoor').
• Middle layers: Attention sharpens onto specific objects and their parts (e.g., the dog's face for 'dog').
• Late layers: Attention converges on highly localized, discriminative features critical for the final prediction. This progression mirrors the hierarchical feature extraction in convolutional networks and is a key signature of effective multimodal fusion.

Individual attention heads within a cross-modal layer often learn distinct, interpretable functions. Common specializations include:

• Object grounding heads: Consistently link noun phrases to their corresponding bounding regions.
• Relational heads: Attend to spatial or semantic relationships between objects (e.g., 'on top of', 'holding').
• Contextual heads: Distribute attention broadly across scene-level features for adjectives like 'sunny' or 'crowded'.
• Syntax heads: Focus on word order and grammatical structure without strong visual grounding. Probing for these specializations helps validate that the model has learned compositional reasoning.

Beyond qualitative heatmaps, attention maps are evaluated using grounding accuracy metrics:

• Pointing Game: Measures whether the maximum attention weight for a noun falls within the ground-truth bounding box of the corresponding object.
• Attention Precision/Recall: Treats attention as a soft segmentation mask and compares it to human-annotated object masks.
• Cross-Modal Faithfulness: Quantifies the drop in model confidence when the most-attended image regions are occluded. A faithful map shows a sharp decline. These metrics provide a rigorous, reproducible framework for comparing explainability methods.

Inspecting cross-modal attention maps reveals several characteristic failure modes:

• Distractor grounding: The model attends to a visually salient but semantically incorrect object (e.g., attending to a red ball when asked about a 'red car').
• Textual bias override: Attention ignores the image entirely, defaulting to linguistic priors from pre-training, a sign of modality neglect.
• Fragmented attention: A single object is attended to by multiple disjoint heads, indicating the model has not formed a unified object representation.
• Background overfitting: Attention fixates on background textures correlated with a class, rather than the object itself, revealing a spurious correlation.

Raw attention weights from a single layer can be misleading due to the complex mixing of information across layers. Attention rollout and attention flow are post-processing techniques that linearly combine attention matrices across all layers to trace how information propagates from input tokens to the final prediction. Rollout assumes that attention weights are combined linearly through the residual connections, producing a single, holistic map of cross-modal information flow. This method often produces sharper, more semantically coherent explanations than single-layer attention.