Self-Consistency Improves Chain of Thought Reasoning in Language Models
Abstract
Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. In this paper, we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting. It first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths. Self-consistency leverages the intuition that a complex reasoning problem typically admits multiple different ways of thinking leading to its unique correct answer. Our extensive empirical evaluation shows that self-consistency boosts the performance of chain-of-thought prompting with a striking margin on a range of popular arithmetic and commonsense reasoning benchmarks, including GSM8K (+17.9%), SVAMP (+11.0%), AQuA (+12.2%), StrategyQA (+6.4%) and ARC-challenge (+3.9%).
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces a simple yet transformative decoding strategy that replaces greedy chain-of-thought generation with sampled reasoning paths aggregated via majority voting, establishing inference-time consistency as a foundational paradigm for improving LLM reasoning. Through rigorous multi-model, multi-benchmark evaluation, it demonstrates that leveraging diverse reasoning trajectories significantly outperforms single-path decoding and prior ensemble methods, fundamentally shifting how practitioners approach LLM inference scaling and reasoning reliability.
Methodology
The proposed method is conceptually elegant and computationally straightforward: replace greedy decoding in Chain-of-Thought (CoT) prompting with stochastic sampling (temperature/top-k/nucleus) to generate multiple diverse reasoning paths, then aggregate final answers via majority voting. The core hypothesis—that correct reasoning paths converge to the same answer while incorrect ones diverge—is well-motivated and empirically validated. The paper thoughtfully explores alternative aggregation strategies (unweighted vs. probability-weighted) and correctly identifies that LLMs are poorly calibrated, making simple majority voting surprisingly effective. While not introducing a new architecture or training objective, the method represents a clever and highly practical inference-time scaling strategy that shifts the paradigm from single-path optimality to ensemble-based consistency.
Results & evaluation
The experimental design is rigorous and comprehensive. The authors evaluate across four models spanning two orders of magnitude in scale (20B to 540B), covering arithmetic, commonsense, and symbolic reasoning benchmarks. They systematically compare against greedy CoT, sample-and-rank, beam search, and prompt ensembles, demonstrating consistent and substantial gains. The ablation studies on the number of sampled paths, sampling hyperparameters, robustness to imperfect prompts, and applicability to zero-shot CoT are particularly strong. The scaling curves clearly show diminishing but meaningful returns, providing practitioners with actionable guidance on compute-accuracy trade-offs.
Reproducibility
High. The method requires no fine-tuning, auxiliary models, or human annotations, relying solely on off-the-shelf LLMs and inference-time sampling. The authors provide exact prompts, sampling configurations, and parsing rules. While LaMDA and PaLM are closed, the inclusion of public UL2 and GPT-3 (via API) ensures the approach can be readily replicated by the community. The simplicity of the pipeline inherently lowers the barrier to reproduction.
Limitations
The primary limitation is the linear increase in inference compute and latency, which may be prohibitive for real-time applications. The method also assumes a fixed, discrete answer space amenable to majority voting, limiting direct applicability to open-ended generation tasks. Additionally, the paper acknowledges that models can still produce logically flawed or factually incorrect intermediate steps, and the approach does not inherently ground or verify the reasoning process itself. Finally, the reliance on prompt formatting for answer parsing introduces minor fragility in highly unconstrained settings.
Broader impact
This work fundamentally shaped the trajectory of LLM reasoning research by establishing inference-time compute scaling as a critical lever for performance. It directly inspired subsequent paradigms like Best-of-N, Tree of Thoughts, and self-reflection mechanisms, and became a standard baseline in virtually all reasoning-focused LLM papers. By demonstrating that diversity in reasoning paths outweighs single-path optimization, it shifted community focus toward decoding strategies and ensemble methods. The work also highlights important considerations around compute efficiency, model calibration, and the need for better rationale grounding in production systems. The paper introduces a simple yet transformative decoding strategy that replaces greedy chain-of-thought generation with sampled reasoning paths aggregated via majority voting, establishing inference-time consistency as a foundational paradigm for improving LLM reasoning. Through rigorous multi-model, multi-benchmark evaluation, it demonstrates that leveraging diverse reasoning trajectories significantly outperforms single-path decoding and prior ensemble methods, fundamentally shifting how practitioners approach LLM inference scaling and reasoning reliability.
Curator note
Wang et al.; majority-vote sampling over CoT paths
Author of this paper? Send a correction.