Vision Transformer (ViT)

The foundational step where a raw image is divided into a grid of fixed-size, non-overlapping patches, typically 16x16 pixels. Each patch is linearly flattened into a 1D vector and projected into a latent embedding space. A learnable classification token is prepended to this sequence, and positional embeddings are added to retain spatial information, converting the visual input into a sequence of tokens analogous to words in NLP.

Unlike CNNs that build receptive fields hierarchically, ViTs compute relationships between all patch tokens simultaneously in every layer. The self-attention mechanism calculates attention weights that determine how much each patch influences the representation of every other patch. This allows the model to directly model long-range dependencies, such as correlating a texture anomaly on one side of a manufactured part with a structural feature on the opposite side, without information loss through sequential downsampling.

ViTs contain no built-in assumptions about translation equivariance or locality, which are hard-coded into CNNs through convolutional kernels. While this means ViTs require more data or stronger augmentation to learn spatial relationships, it also grants them greater flexibility. They can learn non-local, highly complex feature interactions that may be missed by the rigid local receptive fields of a CNN, making them powerful for detecting subtle, globally distributed manufacturing defects.

ViT architectures exhibit strong scaling properties, with performance consistently improving as model size and dataset volume increase. When pre-trained on massive generic image datasets like ImageNet-21k or JFT-300M, ViTs learn highly generalizable visual representations. These pre-trained weights can be fine-tuned on relatively small, domain-specific manufacturing defect datasets, achieving state-of-the-art accuracy that often surpasses equivalently sized CNNs.

The self-attention layers of a ViT produce explicit attention maps that can be visualized to understand which image regions the model is focusing on for its classification decision. This provides a built-in form of explainability, allowing quality assurance engineers to verify that the model is identifying the correct defect morphology rather than relying on spurious background correlations, directly supporting Explainable AI (XAI) requirements.

The core computational unit of the Vision Transformer that allows the model to weigh the importance of every image patch relative to every other patch in a single operation. Unlike the local receptive fields of CNNs, self-attention captures global contextual relationships from the very first layer. The mechanism computes query, key, and value vectors for each patch, producing attention scores that determine how much focus one patch should receive when encoding another. This enables ViTs to model long-range dependencies—such as a subtle texture anomaly spanning disconnected regions of a manufactured component—that convolutional architectures struggle to capture without significant depth.

The process of dividing an input image into a grid of fixed-size, non-overlapping patches—typically 16×16 pixels—and linearly projecting each patch into a flat vector representation. This transforms the 2D image into a 1D sequence of tokens analogous to words in a sentence, making it digestible by the transformer encoder. The patch size directly controls the granularity of the input representation: smaller patches yield finer spatial resolution at the cost of a longer sequence and higher computational demand. A learnable class token is prepended to the sequence, and positional embeddings are added to preserve spatial information lost during flattening.

An extension of single self-attention that runs multiple attention operations in parallel, each with its own learned projection matrices. Each head can specialize in different relational patterns:

• One head may attend to adjacent patches for local texture continuity
• Another may link distant patches sharing a color or structural anomaly
• A third may focus on edge boundaries between components

The outputs of all heads are concatenated and projected, giving the model a richer, multi-faceted representation of the image. This parallelism is what allows ViTs to simultaneously analyze defects at multiple spatial scales and semantic levels.

Since the transformer architecture is permutation-invariant by design—it has no inherent sense of spatial order—positional encodings inject information about the location of each patch within the original image grid. ViTs typically use learned 1D positional embeddings added directly to the patch embeddings. Without them, the model cannot distinguish between a defect in the top-left corner and one in the bottom-right. Research has shown that ViTs learn to encode spatial distance and grid topology within these embeddings, enabling the model to understand concepts like adjacency, alignment, and relative positioning critical for manufacturing inspection.

A comparative analysis of architectural trade-offs for industrial quality inspection:

• Global context: ViTs capture image-wide dependencies from layer one; CNNs require many layers to expand their receptive field
• Inductive bias: CNNs have built-in translation equivariance and locality, making them more data-efficient; ViTs must learn these priors from larger datasets
• Scalability: ViTs scale more effectively with massive datasets and compute, often surpassing CNNs when pre-trained on large corpora like JFT-300M
• Fine-grained localization: CNNs retain an edge in precise spatial localization due to their hierarchical feature maps; hybrid architectures like Swin Transformer aim to bridge this gap

The dominant paradigm for deploying ViTs in manufacturing involves a two-stage process:

• Pre-training: The model is first trained on a massive general-purpose dataset like ImageNet-21k or JFT-300M using supervised classification or self-supervised methods like Masked Autoencoding (MAE)
• Fine-tuning: The pre-trained backbone is then adapted to a specific defect classification task using a smaller, domain-specific dataset of labeled manufacturing images

This transfer learning approach mitigates ViTs' hunger for data, enabling high accuracy even with limited factory-floor examples. The pre-trained model provides rich, generalizable visual features that transfer effectively to industrial domains.