DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck
Abstract
Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fundamental bottleneck: the cost of evaluating progress. In real settings, a single true evaluation, such as a full benchmark or serving trace, can take hours, so a search can afford only a handful of true evaluations. We present DuoDiscover, a search system that jointly discovers what to optimize and how to evaluate progress efficiently, through three mechanisms. First, each worker in a fleet of coding agents co-discovers an optimization target and a fast, tailored proxy evaluator, which we call a yardstick. The agent then hill-climbs the yardstick rather than the true evaluation. Second, since search can exploit gaps between a yardstick and the true objective, fidelity tests continually recheck each yardstick's ranking and invalidate any that drift. Third, the fleet is bottom-up, with no central decomposition of the problem. A lightweight orchestrator spawns and terminates agents, briefs running ones, and seeds new agents with others' confirmed work. DuoDiscover enables autonomous optimization from software to chip architecture. We demonstrate DuoDiscover on two repository-level benchmarks and four GPU and TPU kernel suites covering hundreds of kernels. On SWE-fficiency, it doubles the average speedup of human experts. On SOL-Exec, it ranks first on 121 of 235 kernels and in the top three on 212. On the heavily optimized vLLM and SQLite engines, DuoDiscover achieves 1.92$\times$ and 1.38$\times$ gains over the base code. On a CPU microarchitecture optimization task with evaluations of up to 28 hours, DuoDiscover surpasses both AlphaEvolve-based ArchAgent v2 and the best human design, with 1.65$\times$ ArchAgent v2's gain in 16.7 days versus more than 40 days, advancing the frontier of autonomous chip design.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
DuoDiscover introduces a co-discovery mechanism for optimization targets and proxy evaluators to overcome the evaluation bottleneck in autonomous search. The paper demonstrates significant improvements in code and hardware optimization tasks, establishing a new paradigm for scaling LLM-based discovery in high-cost domains.
Methodology
The paper introduces DuoDiscover, a search system that addresses the "evaluation bottleneck" in autonomous code optimization. The core novelty is the "co-discovery" of optimization targets and cheap proxy evaluators ("yardsticks") by LLM agents. Unlike prior work (e.g., AlphaEvolve) that treats the evaluator as fixed or user-defined, DuoDiscover allows agents to construct their own fast metrics to guide hill-climbing, with a rigorous "fidelity test" (Kendall's tau rank correlation) to invalidate proxies that drift from the true objective. The bottom-up fleet architecture, where a lightweight orchestrator manages expensive true evaluations while agents exploit cheap proxies, is a well-motivated and technically sound approach to scaling search in high-cost domains.
Results & evaluation
The experimental evaluation is extensive and covers a wide range of domains, from repository-level Python optimization (SWE-fficiency, FormulaCode) to GPU/TPU kernel optimization (SOL-Exec, JAXBench) and CPU microarchitecture design (DPC4). The results are strong, with DuoDiscover outperforming baselines like Claude Code and AlphaEvolve-based agents. Notably, the paper reports state-of-the-art results on the SOL-Exec leaderboard and surpasses human experts on SWE-fficiency. The inclusion of a 28-hour evaluation task (CPU prefetcher) demonstrates the method's ability to handle extreme evaluation costs, a key claim of the paper.
Reproducibility
The paper provides detailed descriptions of the fidelity test, noise estimation, and search loops. However, as is common with large-scale agentic systems, the exact prompts and internal logic of the "orchestrator" LLM are not fully disclosed, which may limit exact reproduction. The use of specific, potentially proprietary, LLM backbones (Claude Opus 5, Gemini 3.8 Flash) also poses a reproducibility challenge for external researchers without access to these models.
Limitations
The method relies heavily on the ability of LLMs to construct valid proxy evaluators. If an agent cannot construct a faithful yardstick, the system falls back to expensive true evaluations, negating the efficiency gains. Additionally, the upfront cost of building and validating yardsticks may be prohibitive for very short search budgets. The paper also notes that the method is less effective when the true objective is highly non-linear or when the gap between proxy and true objective is too large to bridge via hill-climbing.
Broader impact
This work has significant implications for the field of autonomous software engineering and hardware design. By solving the evaluation bottleneck, it enables the application of LLM-based search to real-world engineering problems where feedback is slow and expensive. This could accelerate progress in areas like chip design, database optimization, and large-scale system tuning, potentially reducing the time and cost of manual optimization efforts. DuoDiscover introduces a co-discovery mechanism for optimization targets and proxy evaluators to overcome the evaluation bottleneck in autonomous search. The paper demonstrates significant improvements in code and hardware optimization tasks, establishing a new paradigm for scaling LLM-based discovery in high-cost domains.
Author of this paper? Send a correction.