Self-Consistency Decoding

Instead of taking the single most probable token at each step, Self-Consistency marginalizes over multiple latent reasoning paths. The model samples k diverse chain-of-thought trajectories (e.g., k=10), each with a non-zero temperature to encourage variation. The final answer is selected by majority voting on the conclusion, not the path. This exploits the empirical finding that a correct reasoning process is more likely to reach a consensus answer than an erroneous one, which tends to scatter across the output space.

The efficacy of Self-Consistency is directly proportional to the entropy of the sampled reasoning paths. If all paths are identical, the vote is meaningless. Diversity is induced by:

• Sampling Temperature: Setting T > 0 (commonly 0.7) to flatten the probability distribution.
• Top-k/Top-p Sampling: Truncating the token vocabulary to avoid degenerate repetition.
• Prompt Variation: Injecting minor phrasing differences. In legal multi-document reasoning, this diversity allows the model to explore different statutory interpretations or factual connections before converging on the most legally sound conclusion.

The final output is not a raw generation but a statistically aggregated result. Common strategies include:

• Majority Vote: The most frequent exact-match answer. Works well for classification and short-form QA.
• Normalized Weighted Sum: Weights each answer by the inverse perplexity of its reasoning path, favoring more confident chains.
• Semantic Clustering: For long-form answers, embeddings of the k outputs are clustered, and the medoid of the largest cluster is selected. This prevents a single verbose but wrong answer from dominating. In legal citation tasks, this ensures the most frequently retrieved and validated precedent is surfaced.

Self-Consistency is an inference-time compute multiplier. Generating k sequences requires k times the decoding cost of a single greedy generation. This creates a direct trade-off:

• Latency: A 10x increase in generation time, making it unsuitable for sub-second real-time applications without parallelization.
• Throughput: Reduced batch size capacity on a single accelerator. The cost is justified in high-stakes legal analysis where factual precision is paramount. Optimizations include early stopping when m out of k paths agree on an answer, and speculative majority voting that prunes divergent paths mid-generation.

Self-Consistency directly counters random hallucination by exploiting the stochastic nature of the error. A hallucinated fact is typically a low-probability, idiosyncratic artifact of a single sampling path. By requiring consensus across multiple independent samples, these spurious artifacts are washed out statistically. The technique is particularly effective against:

• Factual Fabrication: Inventing case citations or statutes.
• Logical Contradiction: One path contradicting another. It is less effective against systematic hallucinations (errors learned during pre-training that appear consistently across all paths), which require RAG grounding or knowledge editing to resolve.

The foundational technique that elicits step-by-step reasoning from a language model before it delivers a final answer. Self-consistency decoding operates by sampling multiple diverse chain-of-thought trajectories and identifying the most consistent conclusion.

• Mechanism: Prompts the model to break down complex legal problems into intermediate reasoning steps
• Role in Self-Consistency: Provides the raw reasoning paths that are aggregated to determine the majority outcome
• Example: For a multi-document contract dispute, CoT generates separate analyses of each clause before synthesizing a ruling

A quantitative evaluation framework that measures the factual consistency of a generated summary or answer relative to the source material. It directly complements self-consistency by verifying that the most frequent conclusion is not just popular but also grounded.

• Core Function: Identifies contradictions and unsupported fabrications in model outputs
• Synergy: Applies an additional verification layer to the majority-vote answer selected by self-consistency
• Legal Application: Ensures a synthesized case law conclusion does not contradict the primary holding of any cited precedent

A set of statistical techniques that enable a model to estimate the confidence of its own predictions. When self-consistency reveals a split vote across reasoning paths, uncertainty quantification flags the output for mandatory human review.

• Key Insight: High agreement among paths signals high confidence; low agreement signals ambiguity
• Techniques: Includes conformal prediction and Bayesian methods to provide formal error guarantees
• Risk Mitigation: Prevents the system from confidently asserting a conclusion when the underlying legal reasoning is genuinely contested

A secondary, often smaller, language model trained to act as a critic that checks the primary model's output for factual errors and logical inconsistencies. It serves as an orthogonal validation step to the internal consistency check provided by self-consistency decoding.

• Architecture: Operates as an independent fact-checker, not just a reranker
• Complementary Role: Catches errors that might be consistently repeated across multiple reasoning paths
• Example: A verifier model cross-references every legal citation in a generated brief against a trusted database to detect hallucinated precedents

The fundamental generation strategies that determine the diversity of reasoning paths available for self-consistency. Greedy decoding selects the single highest-probability token at each step, while temperature sampling introduces controlled randomness.

• Greedy Decoding: Deterministic but produces only one reasoning path, making self-consistency impossible
• Temperature Sampling: A hyperparameter (typically > 0) that flattens the probability distribution to generate diverse CoT trajectories
• Trade-off: Higher temperature increases path diversity for better self-consistency but risks introducing noise and incoherence

A collaboratively constructed benchmark for evaluating legal reasoning in large language models. It provides a standardized testbed to measure how effectively self-consistency decoding improves accuracy on specific legal tasks.

• Task Diversity: Includes issue spotting, rule application, and statutory interpretation
• Evaluation Role: Quantifies the reduction in hallucination rates when self-consistency is applied versus single-path decoding
• Metric: Measures the exact-match accuracy of the majority-vote conclusion against a ground-truth legal determination