Multi-Modal Fact-Checking

The foundational technique for verifying visual claims by querying an image against a large-scale index to find prior instances. This process uses perceptual hashing to generate a compact fingerprint of the image, which is resilient to resizing and minor color adjustments. The goal is to establish provenance—determining if an image has been repurposed from an older, unrelated event.

• Key Mechanism: Generates a unique visual signature and queries databases like Google Vision API or TinEye.
• Primary Use Case: Detecting out-of-context imagery where a real photo is paired with a false narrative.
• Limitation: Fails against heavily manipulated or synthetic images that have no prior index match.

A forensic analysis technique that distinguishes authentic recordings from synthetic media created by generative adversarial networks (GANs) or diffusion models. Detection models analyze spatio-temporal inconsistencies invisible to the human eye, such as unnatural blinking patterns, inconsistent lighting, or physiological signal artifacts in facial blood flow.

• Frequency Domain Analysis: Identifies hidden artifacts left by the generative model's up-sampling layers.
• Physiological Signal Verification: Measures heart rate variability from pixel changes in the face to detect synthetic puppetry.
• Artifact Focus: Targets boundary blending anomalies at the edges of manipulated regions.

The process of corroborating the claimed location of an image or video by cross-referencing visual clues with open-source intelligence (OSINT) and satellite imagery. This technique analyzes static features like terrain silhouettes, architectural landmarks, and vegetation against mapping platforms.

• Shadow & Sun Analysis: Uses the angle and length of shadows to calculate the time of day and approximate coordinates via sun position algorithms.
• Reference Matching: Compares visual elements against Google Street View or satellite chronologies to confirm the precise vantage point.
• Signal Verification: Cross-references visible antennae or infrastructure with radio frequency databases.

The examination of the hidden digital envelope attached to a media file to validate its authenticity. This involves parsing Exchangeable Image File Format (EXIF) data for timestamps, device fingerprints, and GPS coordinates. However, since metadata is easily spoofed, the core technique lies in structural analysis of the file's compression artifacts.

• JPEG Ghost Detection: Identifies mismatches in compression quantization tables that indicate a specific region was spliced from a different image.
• Error Level Analysis (ELA): Highlights areas of an image with different error potentials, revealing digital airbrushing or compositing.
• Sensor Pattern Noise: Matches the unique, microscopic noise pattern of a camera sensor to a specific device.

An NLP-to-Vision alignment technique that verifies if the objects, actions, and text within a visual asset logically correspond to the textual claim. This uses a Vision-Language Model (VLM) to generate a dense caption of the scene and then performs Natural Language Inference (NLI) against the claim.

• Object Co-occurrence: Flags improbable object pairings (e.g., a polar bear in a tropical street) that violate world knowledge.
• Optical Character Recognition (OCR) Verification: Extracts visible text in the image and checks it against the claim's narrative.
• Action Semantic Alignment: Confirms that the action depicted (e.g., 'shaking hands') matches the textual description.

The technique of chronologically ordering fragmented visual evidence to verify a sequence of events. This involves analyzing time-of-day indicators (shadows, clock faces) and chronological weather correlation against historical meteorological databases for the claimed location and time.

• Weather Grounding: Verifies if the rain, snow, or clear skies in a video match the historical weather record for that exact timestamp.
• Temporal Tampering Detection: Identifies dropped frames or inconsistent motion vectors that suggest a video has been sped up, slowed down, or had segments removed.
• Cross-Modal Sequencing: Aligns visual timestamps with social media posting logs to detect retroactive editing.

The forensic analysis of images, audio, and video to distinguish AI-generated or manipulated content from authentic recordings. This is a critical first-pass filter in multi-modal verification.

• GAN artifact analysis: Detects pixel-level inconsistencies left by generative adversarial networks
• Phoneme-viseme mismatch: Identifies lip-sync errors in deepfake videos
• Frequency domain analysis: Reveals hidden editing boundaries invisible to the human eye

The process of tracing an image's origin by querying visual search engines and cross-referencing EXIF metadata, landmarks, and environmental features. This establishes whether an image genuinely depicts the claimed event.

• Perceptual hashing: Generates a unique fingerprint resilient to resizing and compression
• Shadow analysis: Uses sun position to verify claimed timestamps
• Reference database matching: Compares against satellite imagery and street-level archives

The use of computer vision models to verify whether the objects, actions, and spatial relationships in an image logically support the accompanying textual claim. This goes beyond metadata to analyze scene semantics.

• Object detection: Identifies and counts entities claimed to be present
• Scene graph generation: Maps relationships between detected objects for logical consistency checks
• Zero-shot classification: Verifies novel or rare visual concepts without retraining

The retrieval architecture that searches across heterogeneous evidence corpora—text documents, image databases, and video archives—to find supporting or refuting evidence for a multi-modal claim.

• CLIP-based embeddings: Aligns text queries with visual evidence in a shared latent space
• Temporal alignment: Matches video timestamps with event timelines from textual reports
• Multi-index fusion: Merges results from vector, keyword, and structured metadata indexes

The specialized analysis of audio recordings to verify speaker identity, detect splicing, and authenticate acoustic environments. Essential for verifying claims made in podcasts, leaked recordings, or voice notes.

• Speaker diarization: Segments audio by unique speaker identity
• Acoustic environment matching: Verifies if background noise matches the claimed location
• Electrical network frequency (ENF) analysis: Matches hum against power grid logs for timestamp verification

The technique of breaking a complex claim into atomic sub-claims that span different modalities. A single news report might assert a visual event, a spoken quote, and a statistical figure—each requiring a distinct verification pipeline.

• Modality routing: Directs each sub-claim to the appropriate verification model (vision, audio, text)
• Cross-modal contradiction detection: Flags when visual evidence contradicts textual assertions
• Evidence fusion: Aggregates verification results across modalities into a unified veracity score