Siamese Network

The defining feature of a Siamese network is that its subnetworks are identical clones—they share the exact same weights, biases, and parameters. This weight tying ensures that two similar inputs will be processed by the same function, mapping them to nearby points in the embedding space. If the weights were not shared, the network could arbitrarily map two identical inputs to different locations, breaking the symmetry required for a valid distance metric. During backpropagation, the gradient updates from both branches are summed and applied to the single shared parameter set.

Unlike classification networks that output class probabilities, Siamese networks are trained to output a distance or similarity score between two inputs. The final layer typically computes a metric such as:

• L1 (Manhattan) distance: Sum of absolute differences between embedding components
• L2 (Euclidean) distance: Straight-line distance in the embedding space
• Cosine similarity: Angle between the two embedding vectors, ignoring magnitude This distance is then fed into a loss function like contrastive loss or triplet loss to pull similar pairs together and push dissimilar pairs apart.

Siamese networks operate on pairs of inputs rather than single samples. During training, the data loader must construct pairs with explicit labels indicating whether they are semantically similar (positive pair) or dissimilar (negative pair). Common pairing strategies include:

• Genuine/Impostor pairs in face verification: two photos of the same person vs. different people
• Paraphrase detection: two sentences with the same meaning vs. different meanings
• Signature verification: two signatures from the same author vs. forgeries This pairwise paradigm transforms a classification problem into a verification or ranking problem.

A well-designed Siamese network enforces functional symmetry: the distance computed between input A and input B must equal the distance between input B and input A. This is guaranteed by the shared-weight architecture combined with a symmetric distance function. Formally, for a Siamese network f and distance metric d:

• d(f(A), f(B)) = d(f(B), f(A)) This property is critical for applications like one-shot learning and retrieval, where the query and candidate can be swapped without changing the relevance score. Asymmetric architectures like Cross-Encoders sacrifice this property for higher accuracy.

Siamese networks excel at one-shot and few-shot learning—classifying instances from classes never seen during training. Because the network learns a similarity function rather than class-specific decision boundaries, it can compare a new query sample against a single reference example at inference time. This is the foundation of:

• Face verification systems that enroll a user with a single photo
• Signature verification that authenticates against one reference signature
• Omniglot character recognition, where new alphabets are introduced with just one example per character The network generalizes to novel classes by asking 'is this similar?' rather than 'which class is this?'

After training, the Siamese architecture can be decoupled: the shared subnetwork is extracted and used as a standalone embedding function. This encoder maps any single input to a fixed-dimensional dense vector that captures its semantic features. These pre-computed embeddings can be:

• Indexed in a vector database for efficient approximate nearest neighbor (ANN) search
• Cached offline to avoid recomputing representations for static document collections
• Transferred to downstream tasks like clustering, visualization, or as input features for a classifier This decoupling is the basis for modern Bi-Encoder retrieval systems used in RAG pipelines.

The foundational objective function for Siamese architectures that explicitly defines a distance metric between paired outputs. Given a pair of inputs processed by the twin subnetworks, contrastive loss minimizes the Euclidean distance for positive pairs (same class) while enforcing a margin-based penalty that pushes negative pairs (different classes) beyond a specified threshold. The loss is defined as L = Y*D² + (1-Y)*max(0, margin - D)², where Y indicates pair similarity and D is the distance between output vectors. This margin-based formulation prevents the network from arbitrarily distorting the embedding space for already well-separated negatives.

A metric learning objective that extends pairwise comparison to triplet-based ranking. Each training sample consists of an anchor, a positive (same class as anchor), and a negative (different class). The loss enforces the constraint that the anchor-positive distance plus a margin must be less than the anchor-negative distance: L = max(0, d(a,p) - d(a,n) + margin). This triplet formulation allows the network to learn relative similarity rankings rather than absolute distances.

• Hard triplets: Negatives closer to the anchor than the positive, producing the strongest gradient signal
• Semi-hard triplets: Negatives farther than the positive but within the margin boundary
• Triplet mining strategies critically impact convergence speed and final embedding quality

The direct architectural descendant of Siamese networks optimized for asymmetric retrieval tasks. A Bi-Encoder independently encodes two separate inputs—typically a query and a document—into dense vector representations using identical or type-specific encoders. Unlike the symmetric Siamese design, Bi-Encoders often employ separate towers for queries and candidates, allowing the document index to be pre-computed offline. At inference time, relevance scoring reduces to a fast dot product or cosine similarity between the query vector and pre-indexed document vectors, enabling sub-linear search over millions of items via approximate nearest neighbor algorithms.

Information Noise-Contrastive Estimation, the dominant loss function in modern self-supervised contrastive learning. Unlike classic contrastive loss that operates on explicit pairs, InfoNCE frames representation learning as a categorical cross-entropy problem: given one positive pair and K negative samples, the model must correctly identify which candidate is the true positive. This formulation maximizes the mutual information between different views of the same underlying data point.

• The temperature parameter τ controls the concentration of the similarity distribution
• Lower temperatures increase the penalty on hard negatives
• Forms the theoretical basis for SimCLR, CLIP, and most modern contrastive frameworks

The precision-oriented counterpart to the Siamese Bi-Encoder that sacrifices inference speed for full pairwise attention. Rather than encoding inputs independently, a Cross-Encoder concatenates the query-document pair and processes them jointly through full self-attention layers. This allows the model to capture fine-grained token-level interactions and subtle semantic relationships that independent encoding misses.

• Produces a single relevance score rather than comparable embeddings
• Typically used as a re-ranking stage after Bi-Encoder retrieval narrows the candidate set
• Architecturally identical to standard Transformer classifiers, repurposed for pairwise scoring

The strategic selection of negative samples that are deceptively similar to the anchor but belong to a different class. In Siamese and contrastive training, random negatives provide weak gradient signals because the model easily distinguishes obviously dissimilar pairs. Hard negatives—samples that map close to the anchor in the current embedding space despite being semantically different—force the network to learn fine-grained discriminative features.

• Offline mining: Hard negatives selected from the full dataset after each epoch
• Online mining: Hard negatives identified within the current mini-batch
• Semi-hard mining: Negatives that satisfy d(a,p) < d(a,n) < d(a,p) + margin
• Critical for preventing representation collapse in deep metric learning