EDA (Easy Data Augmentation)

This operation randomly selects n non-stop words in a sentence and replaces each with one of its synonyms. It leverages lexical databases like WordNet to find appropriate substitutes.

• Purpose: Introduces lexical diversity while preserving the sentence's core semantic meaning.
• Implementation: Requires a synonym lookup. Words are typically chosen based on part-of-speech tags to ensure grammatical correctness.
• Example: Original: "The quick brown fox jumps over the lazy dog." Augmented: "The fast brown fox leaps over the idle dog."

This operation finds synonyms for n random non-stop words in the sentence and inserts one of these synonyms into a random position.

• Purpose: Increases sentence length and complexity, forcing the model to handle slightly more verbose but semantically redundant input.
• Key Consideration: The insertion point is random, which can subtly alter sentence flow but not its fundamental label.
• Example: Original: "This is a good example." Augmented: "This is a fine good solid example."

This operation randomly chooses two words in the sentence and swaps their positions. This is repeated n times.

• Purpose: Disrupts word order to improve model robustness to syntactic variations and minor grammatical errors.
• Limitation: Excessive swapping can destroy sentence coherence, so n is kept small (e.g., 1 or 2 swaps).
• Example: Original: "I love machine learning." Augmented: "Learning love machine I." (with n=2, demonstrating potential incoherence).

With probability p, each word in the sentence is randomly deleted.

• Purpose: Forces the model to infer meaning from an incomplete context, improving resilience to missing words (e.g., from speech recognition errors).
• Mechanism: A uniform random number is generated for each word. If the number is less than p (typically ~0.1), the word is removed.
• Example: Original: "The cat sat on the mat near the window." Augmented: "The cat sat on mat near window."

EDA's operations are governed by a single hyperparameter, α, which controls the percentage of words in a sentence that are altered.

• Function: For a sentence with n_words, the number of words to modify is α * n_words. This value dictates the intensity for Synonym Replacement, Random Insertion, and Random Swap.
• Typical Range: α is usually set between 0.05 and 0.1. A value of 0.1 means approximately 10% of words will be changed.
• Impact: A higher α creates more diverse but potentially noisier augmented data. It is a critical lever for balancing augmentation strength.

EDA is particularly valuable in low-data regimes, such as few-shot learning or working with specialized domain texts where labeled data is scarce.

• Empirical Result: In the original EDA paper, applying these four operations to small text classification datasets (e.g., 500-2000 samples) yielded average accuracy improvements of ~0.8% to 3.0%.
• Cost-Effectiveness: It provides a substantial boost in model performance with minimal computational overhead compared to more complex methods like back-translation.
• Primary Use Case: Augmenting training sets for tasks like sentiment analysis, topic classification, and intent detection where label preservation is paramount.

The overarching family of techniques that artificially expands a training dataset by applying random, label-preserving transformations to existing data samples. Its primary goals are to improve model generalization, increase robustness to input variations, and act as a regularizer to prevent overfitting.

• Core Principle: Create new training examples without collecting new data.
• Modality-Specific: Techniques differ for images (geometric/photometric transforms), text (EDA, back translation), audio (SpecAugment), and tabular data.
• Application: Standard practice in modern deep learning pipelines to combat data scarcity.

A text augmentation technique that generates paraphrased sentences by translating a source sentence into an intermediate language and then back to the original language.

• Process: English → French → English.
• Outcome: Produces a semantically equivalent sentence with different syntactic structure and word choice.
• Use Case: Effective for tasks like question answering, text classification, and machine translation where sentence diversity is crucial.
• Comparison to EDA: More computationally intensive (requires a translation model) but can produce more grammatically complex and diverse rephrasings than EDA's simple operations.

An automated, policy-based data augmentation strategy for images that simplifies the search for effective transformations. It uses only two hyperparameters: N (the number of transformations to apply sequentially) and M (a shared magnitude for all transformations).

• How it works: Randomly selects N operations from a predefined set (e.g., rotation, shear, color adjustments) and applies each with magnitude M.
• Advantage: Eliminates the need for a separate, costly search phase (unlike AutoAugment), making it efficient and easy to tune.
• Relation to EDA: Represents the automated policy counterpart in computer vision to EDA's fixed set of simple operations in NLP.

Synthetic Minority Oversampling Technique is an augmentation method for tabular data designed to address class imbalance. It generates synthetic examples for the minority class by interpolating between existing instances in feature space.

• Algorithm: For a minority class sample, find its k-nearest neighbors, then create new samples along the line segments joining the original sample and its neighbors.
• Key Difference from EDA: Operates in the feature space of a dataset (e.g., after vectorization), not on raw data like text strings or pixels.
• Primary Use: Imbalanced classification problems where one class has significantly fewer examples.

A direct spectrogram augmentation method for automatic speech recognition (ASR) and audio processing. It applies simple, policy-based distortions directly to the log-mel spectrogram representation of audio.

• Core Operations:
  • Time Warping: Slightly distorts the spectrogram along the time axis.
  • Frequency Masking: Masks blocks of consecutive frequency channels.
  • Time Masking: Masks blocks of consecutive time steps.
• Philosophical Kinship with EDA: Employs a similarly straightforward set of input-space operations (warp, mask, delete) tailored to its specific data modality (spectrograms) to dramatically improve model robustness.

Refers to the application of data transformations dynamically during the training loop, typically within the data loader. This ensures each epoch presents a unique, freshly augmented view of the dataset to the model.

• Implementation: Transformations are applied on-the-fly as batches are fetched for training.
• Benefits:
  • Infinite Dataset: Effectively creates an endlessly varying training set from a finite pool of original data.
  • Memory Efficiency: Does not require storing all augmented versions on disk.
• Contrast with Offline Augmentation: EDA is often applied offline (pre-processing), but its operations can be integrated into an online pipeline for NLP, where sentences are randomly altered each time they are loaded.