Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Abstract
Language models are increasingly being deployed for general problem solving across a wide range of tasks, but are still confined to token-level, left-to-right decision-making processes during inference. This means they can fall short in tasks that require exploration, strategic lookahead, or where initial decisions play a pivotal role. To surmount these challenges, we introduce a new framework for language model inference, Tree of Thoughts (ToT), which generalizes over the popular Chain of Thought approach to prompting language models, and enables exploration over coherent units of text (thoughts) that serve as intermediate steps toward problem solving. ToT allows LMs to perform deliberate decision making by considering multiple different reasoning paths and self-evaluating choices to decide the next course of action, as well as looking ahead or backtracking when necessary to make global choices. Our experiments show that ToT significantly enhances language models' problem-solving abilities on three novel tasks requiring non-trivial planning or search: Game of 24, Creative Writing, and Mini Crosswords. For instance, in Game of 24, while GPT-4 with chain-of-thought prompting only solved 4% of tasks, our method achieved a success rate of 74%. Code repo with all prompts: https://github.com/princeton-nlp/tree-of-thought-llm.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The Tree of Thoughts framework represents a significant advancement in the capabilities of language models for problem-solving. By integrating deliberate reasoning and exploration into the decision-making process, it opens new avenues for research and application in natural language processing and artificial intelligence.
Methodology
The Tree of Thoughts (ToT) framework introduces a novel approach to problem-solving with language models by enabling exploration of multiple reasoning paths and self-evaluation. This is a significant advancement over traditional methods that rely on linear, token-level decision-making. The methodology is well-structured, leveraging insights from cognitive science and classical AI, and it effectively integrates search algorithms with the language model's reasoning capabilities. The modularity of the approach allows for flexibility in adapting to various problem types, which is a strong point of the methodology.
Results & evaluation
The experiments conducted across three distinct tasks—Game of 24, Creative Writing, and Mini Crosswords—demonstrate the effectiveness of the ToT framework. The results show substantial improvements over baseline methods, particularly in the Game of 24, where the success rate jumped from 4% to 74%. The experiments are well-designed, with clear baselines and thorough evaluations, including both quantitative metrics and qualitative assessments through human judgments. This rigorous evaluation strengthens the paper's claims about the framework's efficacy.
Reproducibility
The paper provides a GitHub repository with all prompts and code, which enhances reproducibility. However, the paper could benefit from more detailed descriptions of the experimental setups, such as the specific configurations used in the experiments and any hyperparameters that were tuned. The availability of code is a positive aspect, but the clarity of documentation will be crucial for others to replicate the results.
Limitations
The paper acknowledges that the ToT framework may not be necessary for all tasks, particularly those where existing models already perform well. Additionally, the computational cost of using ToT is significantly higher than simpler prompting methods, which could limit its practical applicability in resource-constrained environments. The authors also note that the framework was tested on only three tasks, suggesting that further exploration of its capabilities across a broader range of problems is needed.
Broader impact
The ToT framework has the potential to enhance the decision-making capabilities of language models, making them more effective in complex problem-solving scenarios. This could lead to advancements in various applications, such as coding, data analysis, and creative writing. However, the authors also caution about the potential for misuse in applications involving interaction with external environments, highlighting the need for careful consideration of ethical implications. The Tree of Thoughts framework represents a significant advancement in the capabilities of language models for problem-solving. By integrating deliberate reasoning and exploration into the decision-making process, it opens new avenues for research and application in natural language processing and artificial intelligence.
Curator note
Yao et al.; systematic search over reasoning chains
Author of this paper? Send a correction.