Data Augmentation

A technique where text is translated to an intermediate language and then back to the original language, generating paraphrased versions that preserve semantic meaning while introducing lexical diversity.

• Mechanism: Source text → Target language → Back to source language
• Example: 'The cat sat on the mat' → French → 'The cat was seated on the rug'
• Best for: Text classification, sentiment analysis, and machine translation training
• Key benefit: Produces grammatically fluent variations that reflect natural linguistic patterns

Randomly substitutes words with their synonyms from lexical databases like WordNet to create semantically equivalent text variants while preserving the original label.

• Mechanism: Select words at random → Replace with synonyms from a thesaurus
• Example: 'The movie was excellent' → 'The film was outstanding'
• Best for: Short-text classification and sentiment tasks
• Caution: Context-blind replacement can alter nuance; use with word sense disambiguation

Introduces noise by randomly inserting non-essential words or deleting random words from the text, forcing models to focus on the most salient tokens rather than overfitting to specific phrasing.

• Insertion: Adds synonyms of random words at random positions
• Deletion: Removes words with a probability threshold (typically 0.1-0.2)
• Example: 'The quick brown fox' → 'The quick brown fox jumps' or 'The brown fox'
• Best for: Improving model resilience to incomplete or noisy inputs

Replaces named entities—such as person names, locations, or organizations—with alternative entities of the same type, preserving the sentence structure while teaching the model that the entity category, not the specific instance, drives the label.

• Mechanism: Detect entities via NER → Replace with same-type entity from a knowledge base
• Example: 'Apple opened a store in Tokyo' → 'Google opened a store in London'
• Best for: Relation extraction and entity-agnostic classification
• Requires: A curated entity dictionary or knowledge graph for valid substitutions

Leverages pre-trained language models like BERT or RoBERTa to generate contextually appropriate word substitutions, producing more semantically coherent augmentations than static synonym replacement.

• Mechanism: Mask a token → Predict top-k replacements using a masked language model
• Example: 'The [MASK] was delicious' → 'The meal was delicious' or 'The dessert was delicious'
• Best for: Tasks requiring high semantic fidelity
• Advantage: Captures polysemy and context-dependent meaning that static thesauruses miss

Adapts the image-domain Mixup technique to NLP by creating synthetic training examples through linear interpolation of input embeddings and their corresponding labels, encouraging smoother decision boundaries.

• Mechanism: Interpolate two sentence embeddings and their labels with a mixing coefficient λ
• Formula: x̃ = λxᵢ + (1-λ)xⱼ; ỹ = λyᵢ + (1-λ)yⱼ
• Best for: Regularization in deep neural classifiers
• Variants: SeqMix and word-level embedding interpolation

A label-preserving augmentation technique that replaces named entities in training text with alternative entities of the same type. For example, substituting 'Paris' with 'Berlin' in a sentence about capital cities maintains the semantic structure while increasing lexical diversity.

• Preserves syntactic and semantic labels
• Reduces overfitting to specific entity surface forms
• Commonly used with NER and relation extraction tasks

A data augmentation method where text is machine-translated to an intermediate language and then back to the source language, producing paraphrased variants. This introduces lexical diversity while preserving the original meaning and entity labels.

• Generates syntactically diverse training examples
• Effective for multilingual entity linking
• Can introduce subtle semantic drift requiring validation

A lexical augmentation strategy that randomly substitutes words with their synonyms using resources like WordNet or contextual embeddings. When applied to entity contexts, it strengthens the model's ability to recognize entities regardless of surrounding word choice.

• Improves contextual robustness
• Must avoid replacing named entities themselves
• Works well with keyphrase extraction and salience scoring

A weak supervision paradigm that automatically generates labeled training data by aligning text corpora with existing knowledge bases like Wikidata. This creates augmented datasets for entity extraction without manual annotation, though labels may be noisy.

• Scales entity recognition training to millions of examples
• Relies on entity linking for alignment
• Often combined with noise-reduction techniques

A self-supervised paradigm where models learn representations by distinguishing positive pairs (augmented versions of the same entity mention) from negative pairs (different entities). Uses loss functions like InfoNCE to pull similar entity representations together in embedding space.

• Enhances entity resolution accuracy
• Builds robust knowledge graph embeddings
• Reduces sensitivity to surface form variation