Hubness Reduction

A post-hoc distance metric that corrects for hubness by penalizing the similarity score of a candidate translation based on the average similarity of the source and target vectors to their respective neighborhoods.

• Mechanism: Computes a corrected score: CSLS(x, y) = 2 cos(x, y) - r_T(x) - r_S(y), where r_T(x) is the mean cosine similarity of source vector x to its k target neighbors.
• Effect: Increases the isolation of hubs by explicitly subtracting their high average similarity, making genuine mutual nearest neighbors more prominent.
• Origin: Introduced by Conneau et al. (2018) as a critical fix for aligning monolingual word embedding spaces.

A retrieval formulation that re-normalizes similarity scores from the perspective of the target space, effectively dampening the influence of hub vectors that dominate standard nearest neighbor searches.

• Mechanism: Instead of retrieving the target word y that maximizes P(y|x), ISF retrieves the y that maximizes P(x|y) using a softmax over all source vectors for each target candidate.
• Intuition: A hub y that is a neighbor to many source words will have a low P(x|y) for any specific x, because its probability mass is diluted across the entire source vocabulary.
• Trade-off: Computationally more expensive than CSLS as it requires normalization over the entire source embedding matrix.

A pre-processing normalization technique that centers and projects embedding vectors onto the unit hypersphere to reduce the intrinsic dimensionality that exacerbates hubness.

• Centering: Subtracts the mean vector from all embeddings to remove the global bias that creates hubs in the first place.
• L2 Normalization: Projects all vectors onto the unit hypersphere, ensuring that similarity comparisons are based purely on angular distance rather than vector magnitude.
• All-but-the-Top: A complementary method that removes the top d principal components from the embedding matrix, which often encode dataset-wide noise rather than semantic information.

A retrieval constraint that only accepts a translation pair (x, y) if x is a nearest neighbor of y and y is a nearest neighbor of x, enforcing strict reciprocity.

• Hub Rejection: Hubs are naturally filtered out because while many source words consider a hub their nearest neighbor, the hub rarely considers any single source word its own nearest neighbor.
• Precision-Recall Trade-off: Dramatically increases precision at the cost of recall, as many valid but asymmetric translation pairs are discarded.
• Application: Often used as a high-confidence seed dictionary generator for iterative self-learning methods.

A class of methods that explicitly model the hubness distribution and use it to re-weight or re-rank candidate neighbors, rather than treating all vectors as equally reliable.

• Hubness Score: Each target vector is assigned a hubness score based on its k-occurrence—the number of times it appears in the k-nearest neighbor lists of source vectors.
• Anti-Hub Penalization: Vectors with extremely low k-occurrence (anti-hubs) are also problematic; regression methods can up-weight their influence to ensure they are not completely ignored.
• Local Scaling: Applies a hubness-dependent scaling factor to the similarity scores, effectively shrinking the neighborhood radius around hubs and expanding it around anti-hubs.

A self-learning bootstrapping procedure that alternates between building a synthetic bilingual dictionary using current alignment and refining the orthogonal mapping matrix using that dictionary.

• Procrustes Problem: Finds the optimal orthogonal transformation W that minimizes the distance between mapped source vectors and target vectors in a seed dictionary.
• Iterative Loop: Uses the current W to generate new high-confidence translation pairs via CSLS, then re-solves the Procrustes problem with the expanded dictionary.
• Hubness Mitigation: By iteratively expanding the dictionary with mutual nearest neighbors, the alignment gradually covers more of the embedding space, reducing the concentration of hubs in untranslated regions.