Synthetic Query Generation

A generative language model is prompted with a legal document chunk and instructed to produce plausible queries that a lawyer might ask to retrieve that specific text. This creates a positive training pair (query, document) without human annotation.

• Input: A passage from a contract or case law
• Output: 5-10 synthetically generated natural language questions
• Goal: Teach the retriever to map legal jargon to layperson queries

Generic prompts produce generic queries. Effective synthetic generation requires carefully engineered prompts that encode legal domain knowledge, such as:

• Specifying the jurisdiction (e.g., 'Generate queries a Delaware corporate lawyer would ask')
• Defining the task (e.g., 'Generate queries about indemnification clauses')
• Providing few-shot examples of realistic legal research questions

This ensures the synthetic queries mirror the terminological precision and fact-pattern complexity of real legal search.

Beyond positive pairs, advanced pipelines generate hard negatives—documents that are lexically similar but semantically irrelevant. The LLM is prompted to produce a query for which a specific document is a near-miss.

• Example: A query about 'termination for convenience' paired with a document about 'termination for cause'
• Impact: Dramatically improves the embedding model's discriminative power
• Training: Used with Multiple Negatives Ranking Loss for fine-tuning

Synthetic generation transforms an unsupervised corpus into a supervised training set at scale. The primary cost is LLM inference, not human expert time.

• Scale: 1 million unlabeled documents can yield 5-10 million training pairs
• Cost: Orders of magnitude cheaper than manual annotation by lawyers
• Quality Control: Requires a filtration step to remove hallucinated or nonsensical queries before training

Synthetic queries are the fuel for domain-adapting embedding models like Legal-BERT or BGE. The generated pairs are fed into a contrastive fine-tuning pipeline:

1. Generate: Create query-document pairs from unlabeled legal corpus
2. Filter: Remove low-quality or off-topic generations
3. Fine-tune: Train a bi-encoder using Contrastive Loss or Multiple Negatives Ranking Loss
4. Evaluate: Measure improvement on NDCG and Recall@K against a held-out legal benchmark

If the generating LLM lacks sufficient legal diversity, the synthetic queries may represent a narrow subset of possible search behaviors. The retriever overfits to the generator's biases.

• Symptom: High benchmark scores but poor performance on real user queries
• Mitigation: Use diverse generators, temperature sampling, and periodically validate against real query logs
• Monitoring: Track Embedding Drift in production to detect semantic narrowing over time

The foundational training objective that pulls semantically similar pairs closer in embedding space while pushing dissimilar pairs apart. In synthetic query generation, the generated query and its source document form a positive pair, while all other documents in the batch serve as negatives. This loss function teaches the model to distinguish relevant from irrelevant legal documents based on vector proximity.

An efficient training paradigm where every other document in the batch acts as a negative example for a given query-document pair. This eliminates the need for explicit negative sampling during fine-tuning. For legal embedding models, this loss scales effectively with synthetic query generation because each generated query is paired only with its source document, and all other in-batch documents are treated as negatives automatically.

A curation strategy that identifies superficially similar but irrelevant documents to improve model discrimination. In legal contexts, this might involve finding cases with overlapping citations but opposing holdings. Synthetic query generation can be extended to produce hard negatives by prompting the LLM to generate queries that are deliberately misleading or adjacent to the document's actual content, forcing the retriever to learn finer distinctions.

A technique that augments original search queries with related terms or synonyms to improve recall. Synthetic query generation serves as a form of offline query expansion, creating diverse reformulations of potential user questions. For legal retrieval, this captures the variability in how attorneys phrase research questions, including jurisdictional terminology differences and alternative statutory citations.

A retrieval paradigm addressing the length imbalance between short queries and long documents. Legal queries are typically concise, while case law and contracts span thousands of tokens. Synthetic query generation creates training pairs where a short generated query maps to a lengthy document passage, teaching bi-encoders to bridge this length gap by encoding queries and documents into a shared dense space despite their structural asymmetry.

A compression technique where a smaller student model learns to replicate the embedding behavior of a larger teacher model. In synthetic query generation pipelines, a powerful LLM (teacher) generates high-quality queries, and the resulting training pairs are used to fine-tune a lightweight retriever (student). This transfers the teacher's semantic understanding of legal language into an efficient model suitable for production deployment.