Self-Consistency

Instead of taking the single most probable token at each step, Self-Consistency uses a non-zero temperature and nucleus sampling to generate a fixed set of independent Chain-of-Thought sequences. This explores the model's latent space of plausible legal arguments.

• Mechanism: Generates 5-40 distinct reasoning traces for the same query.
• Diversity Driver: Stochastic sampling ensures each path may cite different precedents or apply a different logical structure.
• Key Distinction: This is not an ensemble of different models; it is an ensemble of different reasoning paths from a single model.

The core aggregation mechanism treats each sampled reasoning path as a 'vote' for a final answer. The algorithm extracts the terminal conclusion from each path and selects the most frequent one. This marginalizes out the randomness of individual reasoning failures.

• Process: Extract final answer span -> Count frequencies -> Select mode.
• Effect: A single faulty logical leap in one path is statistically drowned out by multiple correct paths.
• Legal Application: For a question like 'Is this clause a material adverse change?', the model samples 10 analyses; if 7 conclude 'Yes', that becomes the final output.

An advanced variant that moves beyond simple majority voting. When the model outputs a confidence score or a log-probability for its generated sequence, these values can be used to weight each vote. A path generated with high model confidence carries more weight than a low-confidence one.

• Weighting Schema: w_i = exp(1/N * sum(log P(t_i))) where N is the path length.
• Advantage: Corrects for cases where a wrong answer appears frequently but with low internal consistency.
• Use Case: Critical in high-stakes legal analysis where a bare majority is insufficient for auditability.

Self-Consistency directly attacks the stochastic hallucination problem in legal AI. While a single greedy decode might confidently fabricate a citation, the probability of multiple independent paths fabricating the same wrong citation is exponentially lower. Accuracy scales positively with the number of samples.

• Empirical Finding: Performance on arithmetic and factual legal benchmarks plateaus after ~40 samples.
• Cost Trade-off: Compute cost scales linearly with sample count, requiring a balance between accuracy gain and inference budget.
• Result: Drastically reduces the 'long tail' of random factual errors in case law synthesis.

Self-Consistency is not a standalone prompting technique; it is a decoding wrapper applied specifically to Chain-of-Thought (CoT) prompts. The underlying prompt must explicitly instruct the model to generate a step-by-step legal analysis before stating the conclusion.

• Dependency: Useless without a CoT prompt. It amplifies the reasoning quality of CoT, not replaces it.
• Prompt Structure: 'Let's think step by step. First, identify the governing law...'
• Synergy: CoT provides the depth; Self-Consistency provides the breadth and statistical robustness.

This strategy is optimal for legal tasks with a single, verifiable ground truth. It is less suited for open-ended creative drafting or subjective argument generation where there is no 'correct' answer to converge upon.

• High Suitability: Statutory interpretation, contract clause classification, deadline calculation, case outcome prediction.
• Low Suitability: Persuasive brief drafting, client counseling narratives, negotiation strategy.
• Heuristic: If the output can be validated against a database or a binary legal rule, apply Self-Consistency.