Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Abstract
We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain of thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking. For instance, prompting a 540B-parameter language model with just eight chain of thought exemplars achieves state of the art accuracy on the GSM8K benchmark of math word problems, surpassing even finetuned GPT-3 with a verifier.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper introduces chain-of-thought prompting, a simple yet transformative technique that elicits multi-step reasoning in large language models by providing intermediate reasoning exemplars in few-shot prompts. Through rigorous cross-model, cross-task, and cross-scale evaluations, the authors demonstrate that reasoning is an emergent capability that unlocks dramatically improved performance on arithmetic, commonsense, and symbolic benchmarks, fundamentally reshaping prompt engineering practices and establishing a new paradigm for eliciting complex cognitive behaviors from foundation models without parameter updates.
Methodology
The methodology is elegantly simple yet profoundly effective: augmenting few-shot in-context learning prompts with intermediate natural language reasoning steps ("chain of thought") before the final answer. The authors systematically isolate the mechanism through rigorous ablations (equation-only, variable compute via dot sequences, post-answer CoT), demonstrating that the performance gains stem specifically from sequential, language-mediated reasoning rather than mere compute allocation or knowledge retrieval. The approach requires no architectural changes or parameter updates, relying entirely on prompt design. While conceptually straightforward, the systematic framing of CoT as a scale-dependent emergent capability is methodologically rigorous and well-motivated.
Results & evaluation
The experimental design is comprehensive and strategically chosen. The authors evaluate across three distinct reasoning paradigms: arithmetic (GSM8K, SVAMP, ASDiv, AQuA, MAWPS), commonsense (CSQA, StrategyQA, BIG-bench subsets, SayCan), and symbolic manipulation (last-letter concatenation, coin flip). The inclusion of multiple model families (GPT-3, LaMDA, PaLM, UL2, Codex) and scales (from 350M to 540B) provides compelling evidence for the emergent nature of CoT, clearly showing flat or negative scaling curves for smaller models and steep gains past ~100B parameters. Error analysis on model-generated chains of thought adds valuable qualitative insight, categorizing failure modes (semantic misunderstanding, missing steps, arithmetic errors) and showing how scaling mitigates them. The robustness checks across annotators, exemplar orders, and independent datasets are particularly strong, mitigating concerns about prompt overfitting.
Reproducibility
High for inference and prompt replication. The paper provides exact prompt templates, exemplar sets, and decoding parameters (greedy sampling). Results across proprietary API models (GPT-3, PaLM) are fully reproducible by any researcher with API access. However, full reproducibility of the underlying model training is impossible due to the use of closed-weight models, and total compute budgets are not disclosed (standard for industry-scale LLM research). The authors mitigate this by providing all inputs, outputs, and targets in supplementary materials, and by demonstrating that CoT works robustly across different prompt permutations and annotator styles.
Limitations
The primary limitation is strict scale dependency: CoT only emerges reliably in models exceeding ~100B parameters, making it computationally expensive and inaccessible for resource-constrained settings. The method relies on manual prompt engineering to craft high-quality reasoning exemplars, which introduces annotation overhead and potential bias. Furthermore, the generated chains of thought are not guaranteed to be faithful or logically sound; models can produce plausible-sounding but flawed reasoning that coincidentally yields correct answers, raising interpretability and reliability concerns. The paper also notes that CoT does not universally improve all tasks (e.g., minimal gains on single-step arithmetic or certain classification benchmarks), limiting its applicability to strictly multi-step, reasoning-heavy problems.
Broader impact
This work fundamentally shifted the paradigm of LLM interaction, establishing prompting as a viable alternative to task-specific fine-tuning for complex reasoning. It catalyzed an entire subfield of reasoning-augmented prompting (e.g., Self-Consistency, Tree of Thoughts, ReAct, Program-of-Thought), directly influencing how practitioners deploy LLMs in mathematics, code generation, scientific reasoning, and agent-based systems. By demonstrating that reasoning can be elicited purely through natural language scaffolding, it democratizes access to advanced model capabilities without requiring labeled training data. However, it also amplifies concerns around hallucination, compute inequality, and the opacity of emergent capabilities, necessitating future work on verifiable reasoning, smaller-model distillation, and mechanistic interpretability. This paper introduces chain-of-thought prompting, a simple yet transformative technique that elicits multi-step reasoning in large language models by providing intermediate reasoning exemplars in few-shot prompts. Through rigorous cross-model, cross-task, and cross-scale evaluations, the authors demonstrate that reasoning is an emergent capability that unlocks dramatically improved performance on arithmetic, commonsense, and symbolic benchmarks, fundamentally reshaping prompt engineering practices and establishing a new paradigm for eliciting complex cognitive behaviors from foundation models without parameter updates.
Curator note
Wei et al.; showed reasoning emerges with step-by-step prompting
Related later work
- Self-Consistency (majority vote CoT) 2022
- Tree of Thoughts 2023
- Program-of-Thought 2022
- ReAct (reasoning + acting) 2022
- DeepSeek-R1 (RL for reasoning) 2025
- OpenAI o1/o3 series 2024
Author of this paper? Send a correction.