Self-Consistency

The core mechanism replaces naive single-path decoding with a probabilistic aggregation of multiple reasoning chains. For a given problem, the model samples k independent chain-of-thought traces using a non-zero temperature. The final answer is selected via majority voting over the extracted conclusions. This approximates marginalizing out the latent reasoning variable, reducing variance from any single flawed chain. The technique is most effective when reasoning paths are diverse—different intermediate steps leading to the same correct answer—rather than merely paraphrasing the same logic.

Diversity among sampled reasoning paths is the critical success factor. It is controlled by the sampling temperature parameter. A higher temperature flattens the token probability distribution, encouraging the model to explore less-likely but potentially valid reasoning trajectories. Without sufficient diversity, self-consistency degenerates into repeated sampling of the same (potentially incorrect) chain. The optimal temperature balances exploration of alternative logical decompositions against the risk of introducing nonsensical tokens that derail the reasoning trace entirely.

After generating multiple reasoning traces, a lightweight answer extraction function parses the final line or a structured format to isolate the conclusive answer. The aggregation step then performs majority voting (hard voting) or weighted averaging for numerical outputs. This process is agnostic to the intermediate reasoning's validity—it only evaluates the consistency of the terminal output. For tasks with high aleatoric uncertainty, the distribution of answers itself provides a confidence score, where a fragmented vote signals ambiguity that a single greedy decode would mask.

Self-consistency provides a safeguard against the brittleness of greedy decoding. A single chain-of-thought can fail due to a localized arithmetic error, a missed logical step, or an unlucky token sample that cascades into a hallucination. By generating independent samples, the method isolates these failures. As long as the correct reasoning path is sampled more frequently than any single incorrect pattern, the majority vote recovers the correct answer. This makes the technique particularly valuable for multi-hop reasoning where error propagation in a single chain is a primary failure mode.

The reliability gain comes at a linear increase in inference compute. Generating k samples requires k times the tokens and latency of a single greedy decode. However, this cost is embarrassingly parallel—all k chains can be generated simultaneously in a single batch, minimizing wall-clock overhead. The technique is a practical demonstration of inference-time compute scaling, where additional computation substitutes for larger model size or fine-tuning. For production systems, the sample count is a tunable parameter balancing cost against the required accuracy threshold for a given task.

Standard majority voting can be skewed by biased answer formats. Normalized self-consistency addresses this by applying length normalization or inverse frequency weighting to counter the model's tendency to produce shorter or more common answers. For generative tasks without a discrete answer set, bidirectional entailment can be used: the model evaluates the logical consistency between each generated answer and the original question, selecting the response with the highest average entailment score across all sampled chains.