Strategist: Evolving Reasoning with Agents
Abstract
A model is only half of an AI agent system. The other half is the inference strategy: the reasoning procedure that turns model calls into a response. Today that second half is written by hand, and the experience of building one strategy never carries to the next. We introduce Strategist, a meta-agent that evolves inference strategies across tasks. Each strategy is written as a blueprint,executable code that other strategies can reuse, and together the blueprints form a library. Given a task and a small development set, Strategist retrieves related blueprints, writes candidate strategies, tests them on real samples, and revises them from the results. Every candidate then enters the library, so later tasks build on earlier discoveries. On 30 benchmarks, Strategist beats the strongest baseline on 24 and matches its average accuracy (74.7% vs.74.0%) at 39% lower inference cost. Its strategies also transfer to unseen tasks and other models without further search. The part of an AI system that is not improved on its own now does.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a novel meta-agent framework that evolves reusable inference strategies via a growing library of executable code blueprints. By automating the design of reasoning procedures and enabling cross-task knowledge accumulation, it achieves superior accuracy and cost-efficiency compared to hand-designed and existing automated baselines, marking a significant step towards self-improving agentic systems.
Methodology
The paper introduces "Strategist," a meta-agent framework that automates the design of inference strategies (reasoning procedures) for LLM agents. The core novelty lies in the representation of strategies as "Strategy Blueprints"—executable Python code with a standardized interface that allows for composition and reuse. Unlike previous methods that treat strategy optimization as a per-task black-box search or rely on fixed component libraries, Strategist maintains a growing library of blueprints that accumulates knowledge across tasks. The methodology involves a four-step loop: retrieval of relevant blueprints, proposal of new candidates (balancing exploration and exploitation), evaluation on real samples, and accumulation of results back into the library. The use of multi-tier model routing (fast/balanced/complex) within a single strategy is a practical and effective addition, allowing for cost-aware reasoning.
Results & evaluation
The experimental setup is rigorous, evaluating 30 benchmarks across math, code, commonsense, and QA. The comparison against 18 baselines (including standard prompting, CoT, and agentic frameworks) is extensive. The results show that Strategist beats the strongest baseline on 24/30 benchmarks while reducing inference cost by 39%. The ablation studies clearly demonstrate the contribution of the library and iterative search. The cross-model transfer experiments (applying strategies discovered on Qwen3 to GPT-5 and Gemini) are a strong validation of the method's generalizability, showing that the learned strategies capture task structure rather than model-specific idiosyncrasies.
Reproducibility
The paper provides high reproducibility. It releases code on GitHub, details the exact model configurations (Qwen3 variants), and provides pseudocode for the design loop. The prompt templates and library retrieval mechanisms are described in detail. The use of open-source models for the main experiments further aids reproducibility.
Limitations
The primary limitation is the reliance on a powerful meta-LLM (GPT-6) for the strategist, which may not be accessible to all researchers. The cost of the search process itself (10 iterations per benchmark) is significant, though the paper argues this is a one-time cost per task type. The method is evaluated on a specific set of benchmarks, and its performance on highly dynamic or real-time tasks is not explored.
Broader impact
This work has significant implications for the field of agentic AI by shifting the focus from static prompt engineering to dynamic, cumulative strategy evolution. It provides a framework for building AI systems that improve their own inference efficiency and accuracy over time without retraining the underlying models. This could lead to more robust and cost-effective AI deployments in production environments. The paper presents a novel meta-agent framework that evolves reusable inference strategies via a growing library of executable code blueprints. By automating the design of reasoning procedures and enabling cross-task knowledge accumulation, it achieves superior accuracy and cost-efficiency compared to hand-designed and existing automated baselines, marking a significant step towards self-improving agentic systems.
Author of this paper? Send a correction.