DoLa (Decoding by Contrasting Layers)

DoLa operates by computing the logit difference between a later, mature transformer layer and an earlier, premature layer. The core insight is that factual knowledge tends to emerge in later layers, while earlier layers contain more generic linguistic patterns. By subtracting the early-layer logits from the late-layer logits, DoLa amplifies the signal for factually grounded tokens and suppresses hallucinatory ones. This contrastive approach requires no external model, no fine-tuning, and no additional training data.

Rather than using a fixed early layer for all tokens, DoLa employs dynamic premature layer selection. For each decoding step, it evaluates multiple candidate premature layers and selects the one that maximizes the Jensen-Shannon divergence from the mature layer's output distribution. This adaptive approach ensures the contrast is most effective when the model is uncertain or prone to hallucination, while minimizing interference during straightforward, factual token predictions.

Unlike Retrieval-Augmented Generation (RAG) which relies on external knowledge bases, DoLa reduces hallucinations purely through internal model mechanics. It exploits the hierarchical nature of transformer representations:

• Early layers: Encode surface-level syntax and common patterns
• Later layers: Encode deeper semantic and factual knowledge By contrasting these, DoLa surfaces latent factual knowledge already present in the model's parameters but often suppressed during standard decoding. This makes it complementary to RAG, not competitive.

DoLa is particularly effective at surfacing rare or long-tail factual knowledge that standard greedy decoding or nucleus sampling might miss. The contrastive signal amplifies tokens associated with specific entities, dates, and verifiable facts. Research shows DoLa significantly improves performance on:

• TruthfulQA: Benchmark for truthful responses to adversarial questions
• FACTOR: Factual correctness evaluation
• StrategyQA: Multi-hop reasoning requiring implicit facts This makes it valuable for applications requiring high factual precision without latency overhead.

DoLa functions as a drop-in replacement for standard decoding strategies. It can be combined with:

• Greedy decoding: Simple contrastive amplification
• Nucleus sampling: Contrast before top-p filtering
• Beam search: Apply contrastive scoring to beam candidates The method adds minimal computational overhead since it reuses intermediate layer outputs already computed during the forward pass. No additional model calls or architectural changes are required, making it practical for production deployment.

DoLa belongs to the broader family of contrastive decoding methods that improve generation quality through distributional comparison:

• Standard Contrastive Decoding: Contrasts an expert model against an amateur model
• DoLa: Contrasts layers within a single model (no external amateur needed)
• Context-aware Decoding: Contrasts with and without context DoLa's key advantage is self-containment—it requires only one model and leverages its internal representations, eliminating the need to train or maintain a separate amateur model while achieving comparable hallucination reduction.