Tampering Localization

The fundamental output is a binary segmentation map where each pixel is classified as pristine (0) or tampered (1). This requires the model to perform dense, per-pixel predictions rather than image-level classification. Architectures like U-Net or DeepLab are commonly adapted for this task, leveraging encoder-decoder structures to capture both global context and fine-grained boundary details. The ground truth for training is a manually created, pixel-accurate mask defining the manipulated region.

Localization models often operate on noise residuals rather than raw pixels. By suppressing the image's semantic content with high-pass filters, the model focuses on the subtle, invisible traces of manipulation:

• Sensor Pattern Noise (SPN) Inconsistency: A spliced region from a different camera will lack the host image's unique Photo Response Non-Uniformity (PRNU) fingerprint.
• Re-interpolation Artifacts: Resizing or rotating a pasted object introduces periodic correlations detectable in the residual domain.
• Double JPEG Compression Ghosts: A manipulated region may exhibit a different quantization history than the background.

State-of-the-art localization relies on fully convolutional networks. The encoder progressively downsamples the input to extract high-level forensic features, while the decoder upsamples these features to the original resolution for pixel-accurate segmentation. Key architectural elements include:

• Skip Connections: Directly pass fine-grained spatial information from encoder layers to decoder layers, preserving the sharp boundaries of a spliced object.
• Atrous Spatial Pyramid Pooling (ASPP): Captures multi-scale context to detect forgeries of vastly different sizes, from a small airbrushed detail to a large inserted object.

Robust localization fuses evidence from multiple forensic domains to overcome the limitations of any single technique. A model might simultaneously analyze:

• Spatial Domain Features: Local noise patterns, texture inconsistencies, and edge discontinuities at the splice boundary.
• Frequency Domain Features: Anomalies in the Discrete Cosine Transform (DCT) or wavelet coefficients that reveal re-compression or generative model fingerprints.
• EXIF Metadata Stream: Discrepancies between the file's header information and its actual pixel data, such as a timestamp that predates the camera model's release.

Specialized convolutional layers are designed to suppress the image content and adaptively learn manipulation traces. Constrained convolutional layers force the network's initial kernels to act as high-pass filters, ensuring the model learns forensic features from noise residuals rather than overfitting to semantic image content. This improves generalization to unseen manipulation types and camera models. The learned features often correspond to detectable phenomena like CFA interpolation artifacts or local noise variance inconsistencies.

It's crucial to distinguish this task from standard deepfake detection or AIGC detection. A global detector answers 'Is this image fake?' with a confidence score. Tampering localization answers 'Where is the image fake?' with a mask. This granularity is essential for:

• Forensic Triage: An analyst can immediately focus on the 5% of an image that is manipulated.
• Evidence Integrity: In legal contexts, proving a specific object was inserted is more valuable than a general authenticity score.
• Anti-Forensics Countermeasures: Understanding the exact boundary of a manipulation helps identify the specific inpainting or splicing algorithm used.