TextFooler

TextFooler operates under a black-box threat model, requiring no access to the target model's architecture, gradients, or training data. It relies solely on input-output queries to observe classification confidence scores. This makes it highly practical for evaluating real-world deployed NLP systems where internal model details are proprietary. The framework systematically probes the model's decision boundary by manipulating text inputs and observing prediction changes, simulating a realistic external adversary with no insider knowledge.

The attack begins by calculating word importance scores for every token in the input text. TextFooler temporarily removes each word and measures the resulting drop in the target model's confidence for the correct class. Words whose deletion causes the largest confidence decrease are ranked as most critical. This step identifies the minimal set of words that dominate the model's classification decision, enabling efficient adversarial perturbation without exhaustive search over all possible token combinations.

TextFooler enforces a semantic similarity threshold using Universal Sentence Encoder embeddings. For each candidate synonym substitution, the framework computes the cosine similarity between the original and perturbed sentence embeddings. Replacements that fall below a predefined similarity threshold (typically 0.8) are rejected. This constraint ensures the adversarial text remains semantically equivalent to the original, preserving human readability and preventing the attack from producing nonsensical or obviously altered text that would be trivially detectable.

For each high-importance word, TextFooler generates a list of counter-fitted synonym candidates using pre-trained word embeddings. The framework prioritizes synonyms that are:

• Grammatically compatible with the surrounding context (matching part-of-speech tags)
• Semantically similar to the original word
• Likely to flip the classification when substituted

The attack iteratively replaces words in descending order of importance, querying the model after each substitution. The process stops as soon as the model's prediction changes to any incorrect class, minimizing the number of perturbations.

TextFooler achieves high attack success rates across diverse NLP tasks and model architectures. In the original paper, it fooled BERT-based classifiers on sentiment analysis and textual entailment datasets with success rates exceeding 90% while maintaining semantic similarity above 0.8. The framework is particularly effective against fine-tuned transformer models that rely heavily on specific keywords for classification. Its success demonstrates that even state-of-the-art NLP models remain vulnerable to simple, semantically-preserving word substitutions.

TextFooler exposes a fundamental vulnerability in NLP classifiers: over-reliance on superficial lexical cues rather than deep semantic understanding. Defenses against this attack include:

• Adversarial training with TextFooler-generated examples to harden decision boundaries
• Interval Bound Propagation (IBP) for certified robustness to synonym substitutions
• Synonym encoding that maps synonyms to shared embeddings before classification
• Randomized smoothing over synonym sets to provide probabilistic robustness guarantees

The framework has become a standard benchmark in the adversarial robustness evaluation of NLP systems.