Paraphrase Generation

The generated paraphrase must preserve the core meaning of the original query while altering its surface form. This distinguishes it from synonym expansion, which operates at the word level. A successful paraphrase of 'how to fix a leaky faucet' might be 'repairing a dripping tap'—different lexically but identical in intent. Semantic drift is the primary failure mode, where the model introduces or removes constraints, changing retrieval precision.

Effective paraphrase generation introduces structural variation to bridge the gap between how users ask questions and how documents are written. Techniques include:

• Passivization: 'The engineer configured the server' → 'The server was configured by the engineer'
• Clause reordering: Moving subordinate clauses to alter emphasis
• Nominalization: Converting verbs to nouns ('decide' → 'decision') This diversity ensures coverage of different writing styles in the target corpus.

Modern paraphrase generation relies on sequence-to-sequence models fine-tuned for the task:

• T5 and BART: Encoder-decoder transformers trained on paraphrase corpora like PARA-NMT or Quora Question Pairs
• Back-translation: Translating the query to an intermediate language and back (e.g., English → German → English) to induce natural variation
• Prompted LLMs: Using few-shot prompts with large models to generate diverse alternatives without task-specific fine-tuning Each approach trades off latency, diversity, and semantic fidelity.

A central tension in paraphrase generation is balancing lexical diversity against semantic fidelity. Techniques to control this include:

• Temperature sampling: Higher temperatures increase diversity but risk hallucination
• Nucleus sampling (top-p): Truncates the low-probability tail to maintain coherence
• Diversity-promoting beam search: Adds penalties for similar tokens across beams Production systems often generate multiple candidates and apply a semantic similarity filter using a cross-encoder to discard paraphrases below a cosine similarity threshold.

Paraphrase generation can be applied at two distinct points in the retrieval pipeline:

• Query-time expansion: The user's query is paraphrased into multiple variants, each executed against the index. This increases query latency but requires no index modification.
• Index-time document expansion: Documents are augmented with generated paraphrases of their key passages (as in doc2query). This shifts the compute cost offline, keeping query latency low at the expense of a larger index. The choice depends on latency budgets and index update frequency.

Paraphrase quality is assessed through both intrinsic and extrinsic metrics:

• BLEU and ROUGE: Measure n-gram overlap with reference paraphrases, but penalize valid lexical diversity
• BERTScore: Uses contextual embeddings to evaluate semantic similarity without exact match requirements
• iBLEU: Penalizes both insufficient and excessive diversity relative to references
• Retrieval effectiveness: The ultimate extrinsic measure—does adding paraphrases improve Recall@K or NDCG on a held-out query set? Human evaluation remains critical for detecting subtle semantic drift.