Last 7 Days (October 10 – October 16, 2026)
Significant efforts are currently being directed toward the implementation of large-scale quantum information processors, but realizing useful quantum advantage also requires the development of efficient quantum algorithms. While for many practical problems the dominant resource costs are associated with a small set of computational primitives, optimizing them in the quantum setting presents unique challenges, including a reversibility constraint absent from ordinary classical computation. Here we introduce an evolutionary framework in which large language models (LLMs) propose circuit designs and an independent evaluator verifies and scores them. We apply this framework to tasks spanning numerical computation, chemistry, optimization, quantum simulation, and number theory. In seven of the eight tasks, the discovered circuits significantly reduce both Toffoli and qubit counts relative to the state of the art. In the eighth, the search discovers a new reversible architecture for prime-field modular inversion. By handling reversibility differently, the architecture makes divsteps -- a classical algorithm previously impractical for reversible integer arithmetic -- viable and substantially improves the Toffoli-qubit trade-off for modular inversion. These results open the door for evaluator-guided LLM co-design of new algorithmic structures for quantum computation.
Primary: Harvard University
All Institutions: Harvard University, Anthropic, QuEra Computing
The paper presents a novel evolutionary framework using LLMs to discover state-of-the-art quantum circuits, achieving significant reductions in resource costs across multiple tasks and uncovering new reversible algorithms. This work bridges the gap between AI-driven code generation and rigorous quantum algorithm design, offering a powerful tool for future quantum computing research.
The paper introduces a robust evolutionary framework where Large Language Models (LLMs) propose high-level Python circuit designs, which are then verified by an independent, programmatic evaluator. This approach effectively addresses the "reversibility constraint" of quantum computation by allowing the LLM to reason about algorithmic structure rather than just gate sequences. The methodology is sound, leveraging the fact that coherent circuits on computational basis states can be simulated efficiently (tracking bits and phases), enabling rapid feedback loops. The use of MAP-Elites to maintain diversity in the search space is a strong technical choice that prevents premature convergence to local optima.
The experimental results are highly impressive, demonstrating state-of-the-art improvements in Toffoli and qubit counts across eight diverse tasks, including Reed-Solomon decoding and modular inversion. The discovery of a new reversible architecture for prime-field modular inversion using divsteps is a significant algorithmic contribution, not just an optimization of existing circuits. The benchmarks are rigorous, comparing against established baselines and providing detailed resource accounting.
The paper provides sufficient detail on the circuit model, cost metrics, and search strategy to allow for reproduction of the framework. However, the specific LLM prompts and the exact "OpenEvolve" implementation details are not fully open-sourced in the text, which may limit immediate reproducibility for those without access to the specific tooling or LLM versions (Claude Mythos 5).
The framework is bounded by the task specification and the computational basis representation, excluding optimizations that rely on changing the encoding itself (e.g., coset representations). The reliance on a specific LLM (Claude Mythos 5) and the proprietary "OpenEvolve" framework may limit accessibility. Additionally, the TQ-cost metric is a simplification that may not capture all hardware-specific constraints like depth or routing.
This work has significant implications for the field of quantum computing, offering a scalable path to discovering efficient quantum algorithms. It demonstrates that LLMs can contribute to fundamental algorithmic design in domains with complex constraints, potentially accelerating the development of fault-tolerant quantum computers. The approach could be extended to other areas of AI for Science where programmatic evaluation is feasible. The paper presents a novel evolutionary framework using LLMs to discover state-of-the-art quantum circuits, achieving significant reductions in resource costs across multiple tasks and uncovering new reversible algorithms. This work bridges the gap between AI-driven code generation and rigorous quantum algorithm design, offering a powerful tool for future quantum computing research.
Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fundamental bottleneck: the cost of evaluating progress. In real settings, a single true evaluation, such as a full benchmark or serving trace, can take hours, so a search can afford only a handful of true evaluations. We present DuoDiscover, a search system that jointly discovers what to optimize and how to evaluate progress efficiently, through three mechanisms. First, each worker in a fleet of coding agents co-discovers an optimization target and a fast, tailored proxy evaluator, which we call a yardstick. The agent then hill-climbs the yardstick rather than the true evaluation. Second, since search can exploit gaps between a yardstick and the true objective, fidelity tests continually recheck each yardstick's ranking and invalidate any that drift. Third, the fleet is bottom-up, with no central decomposition of the problem. A lightweight orchestrator spawns and terminates agents, briefs running ones, and seeds new agents with others' confirmed work. DuoDiscover enables autonomous optimization from software to chip architecture. We demonstrate DuoDiscover on two repository-level benchmarks and four GPU and TPU kernel suites covering hundreds of kernels. On SWE-fficiency, it doubles the average speedup of human experts. On SOL-Exec, it ranks first on 121 of 235 kernels and in the top three on 212. On the heavily optimized vLLM and SQLite engines, DuoDiscover achieves 1.92$\times$ and 1.38$\times$ gains over the base code. On a CPU microarchitecture optimization task with evaluations of up to 28 hours, DuoDiscover surpasses both AlphaEvolve-based ArchAgent v2 and the best human design, with 1.65$\times$ ArchAgent v2's gain in 16.7 days versus more than 40 days, advancing the frontier of autonomous chip design.
Primary: Google
All Institutions: Google
DuoDiscover introduces a co-discovery mechanism for optimization targets and proxy evaluators to overcome the evaluation bottleneck in autonomous search. The paper demonstrates significant improvements in code and hardware optimization tasks, establishing a new paradigm for scaling LLM-based discovery in high-cost domains.
The paper introduces DuoDiscover, a search system that addresses the "evaluation bottleneck" in autonomous code optimization. The core novelty is the "co-discovery" of optimization targets and cheap proxy evaluators ("yardsticks") by LLM agents. Unlike prior work (e.g., AlphaEvolve) that treats the evaluator as fixed or user-defined, DuoDiscover allows agents to construct their own fast metrics to guide hill-climbing, with a rigorous "fidelity test" (Kendall's tau rank correlation) to invalidate proxies that drift from the true objective. The bottom-up fleet architecture, where a lightweight orchestrator manages expensive true evaluations while agents exploit cheap proxies, is a well-motivated and technically sound approach to scaling search in high-cost domains.
The experimental evaluation is extensive and covers a wide range of domains, from repository-level Python optimization (SWE-fficiency, FormulaCode) to GPU/TPU kernel optimization (SOL-Exec, JAXBench) and CPU microarchitecture design (DPC4). The results are strong, with DuoDiscover outperforming baselines like Claude Code and AlphaEvolve-based agents. Notably, the paper reports state-of-the-art results on the SOL-Exec leaderboard and surpasses human experts on SWE-fficiency. The inclusion of a 28-hour evaluation task (CPU prefetcher) demonstrates the method's ability to handle extreme evaluation costs, a key claim of the paper.
The paper provides detailed descriptions of the fidelity test, noise estimation, and search loops. However, as is common with large-scale agentic systems, the exact prompts and internal logic of the "orchestrator" LLM are not fully disclosed, which may limit exact reproduction. The use of specific, potentially proprietary, LLM backbones (Claude Opus 5, Gemini 3.8 Flash) also poses a reproducibility challenge for external researchers without access to these models.
The method relies heavily on the ability of LLMs to construct valid proxy evaluators. If an agent cannot construct a faithful yardstick, the system falls back to expensive true evaluations, negating the efficiency gains. Additionally, the upfront cost of building and validating yardsticks may be prohibitive for very short search budgets. The paper also notes that the method is less effective when the true objective is highly non-linear or when the gap between proxy and true objective is too large to bridge via hill-climbing.
This work has significant implications for the field of autonomous software engineering and hardware design. By solving the evaluation bottleneck, it enables the application of LLM-based search to real-world engineering problems where feedback is slow and expensive. This could accelerate progress in areas like chip design, database optimization, and large-scale system tuning, potentially reducing the time and cost of manual optimization efforts. DuoDiscover introduces a co-discovery mechanism for optimization targets and proxy evaluators to overcome the evaluation bottleneck in autonomous search. The paper demonstrates significant improvements in code and hardware optimization tasks, establishing a new paradigm for scaling LLM-based discovery in high-cost domains.
Language models are often evaluated alone but increasingly deployed together, routed between, made to debate, ensembled, or collaboratively trained. For a model inside a system, what matters most is not how well it performs on its own, but how much the system gains from its presence. We call this a model's compositional strength and formalize it using the Shapley value of a cooperative game over coalitions of a model pool, and use a model's leave-one-out contribution to the full coalition as a tractable proxy, requiring n+1 rather than 2^n system evaluations. Evaluating individual vs. compositional strengths across five model pools, twelve collaboration methods, and twelve datasets, we find them largely uncorrelated: in 12 of 15 (pool, method-family) settings, individual strength does not translate into compositional advantages, and the coupling is weakest among general-purpose models that practitioners are most likely to deploy. Standalone benchmarks are therefore often poor proxies for models' value in collaborative systems. We then propose compositional reinforcement learning (C-RL), which trains a policy inside a live multi-model system and rewards the outcome produced by the system as a whole, rather than the policy's correctness in isolation. The leave-one-out baseline in Shapley values cancels under a group policy gradient, so this needs no additional rollout cost beyond RL on the collaborative system's outcome and never requires running a coalition without the policy. Across three collaboration protocols and eight tasks spanning reasoning, coding, and scientific discovery, C-RL produces the strongest collaborator in 21 of 24 settings, lifting collaborative performance by +4.0 to +5.4 pts on average (~3x the gain of baselines), up to +13.1 pts.
Primary: University of Washington
All Institutions: University of Washington
The paper makes a significant contribution by formalizing compositional strength via Shapley values and demonstrating that standard individual benchmarks are poor predictors of a model's value in collaborative systems. By proposing Compositional Reinforcement Learning (C-RL), which efficiently trains models to maximize their contribution to a multi-agent system's outcome, the authors provide a novel and effective method for developing specialized collaborator models that generalize across tasks and protocols.
The paper introduces a rigorous formalization of "compositional strength" using Shapley values from cooperative game theory, adapted for LLM collaborations. The key methodological contribution is the derivation of a tractable Leave-One-Out (LOO) estimator that approximates the Shapley value with linear complexity ($n+1$) rather than exponential ($2^n$). The proposed training method, Compositional Reinforcement Learning (C-RL), is elegant in its use of group-relative policy gradients (GRPO) to cancel out the baseline cost of the LOO marginal, allowing the policy to be trained on system-level rewards without extra rollouts. The credit assignment mechanism, which attributes system-level rewards to individual policy generations within a multi-turn collaboration, is a significant technical challenge addressed effectively.
The evaluation is extensive, covering 5 model pools, 12 collaboration methods, and 12 datasets. The finding that individual strength and compositional strength are largely uncorrelated is a strong empirical contribution. The training experiments demonstrate that C-RL outperforms baselines (including standard RL and collaborative fine-tuning) in 21/24 settings. The generalization tests to unseen datasets and collaboration protocols are convincing, showing that C-RL teaches transferable collaboration skills rather than overfitting to a specific setup. The inclusion of proprietary frontier models as collaborators adds practical relevance.
The paper promises code release upon acceptance. The use of proprietary APIs (GPT-5.5, etc.) introduces a reproducibility caveat noted by the authors, as these models may change over time. However, the open-source components (Qwen, open-weight collaborators) allow for partial reproduction. The detailed description of the MoCo toolkit and specific hyperparameters aids reproducibility.
The reliance on proprietary models for some experiments limits full reproducibility. The computational cost of running multi-model collaborations for RL training is high, potentially limiting adoption to well-resourced labs. The credit assignment is uniform across turns, which the authors acknowledge as a simplification that could be improved. The study focuses on a specific set of collaboration protocols; while generalization is tested, the scope is not exhaustive of all possible multi-agent architectures.
This work shifts the focus of LLM evaluation and training from individual capability to system-level contribution. It provides a theoretical foundation for understanding multi-agent dynamics and a practical method for training better collaborators. This is highly relevant as LLMs are increasingly deployed in compound systems. The finding that standard benchmarks do not predict collaborative value is a critical insight for practitioners building multi-agent systems. The paper makes a significant contribution by formalizing compositional strength via Shapley values and demonstrating that standard individual benchmarks are poor predictors of a model's value in collaborative systems. By proposing Compositional Reinforcement Learning (C-RL), which efficiently trains models to maximize their contribution to a multi-agent system's outcome, the authors provide a novel and effective method for developing specialized collaborator models that generalize across tasks and protocols.
Significant efforts are currently being directed toward the implementation of large-scale quantum information processors, but realizing useful quantum advantage also requires the development of efficient quantum algorithms. While for many practical problems the dominant resource costs are associated with a small set of computational primitives, optimizing them in the quantum setting presents unique challenges, including a reversibility constraint absent from ordinary classical computation. Here we introduce an evolutionary framework in which large language models (LLMs) propose circuit designs and an independent evaluator verifies and scores them. We apply this framework to tasks spanning numerical computation, chemistry, optimization, quantum simulation, and number theory. In seven of the eight tasks, the discovered circuits significantly reduce both Toffoli and qubit counts relative to the state of the art. In the eighth, the search discovers a new reversible architecture for prime-field modular inversion. By handling reversibility differently, the architecture makes divsteps -- a classical algorithm previously impractical for reversible integer arithmetic -- viable and substantially improves the Toffoli-qubit trade-off for modular inversion. These results open the door for evaluator-guided LLM co-design of new algorithmic structures for quantum computation.
Primary: Harvard University
All Institutions: Harvard University, Anthropic, QuEra Computing
The paper presents a novel evolutionary framework using LLMs to discover state-of-the-art quantum circuits, achieving significant reductions in resource costs across multiple tasks and uncovering new reversible algorithms. This work bridges the gap between AI-driven code generation and rigorous quantum algorithm design, offering a powerful tool for future quantum computing research.
The paper introduces a robust evolutionary framework where Large Language Models (LLMs) propose high-level Python circuit designs, which are then verified by an independent, programmatic evaluator. This approach effectively addresses the "reversibility constraint" of quantum computation by allowing the LLM to reason about algorithmic structure rather than just gate sequences. The methodology is sound, leveraging the fact that coherent circuits on computational basis states can be simulated efficiently (tracking bits and phases), enabling rapid feedback loops. The use of MAP-Elites to maintain diversity in the search space is a strong technical choice that prevents premature convergence to local optima.
The experimental results are highly impressive, demonstrating state-of-the-art improvements in Toffoli and qubit counts across eight diverse tasks, including Reed-Solomon decoding and modular inversion. The discovery of a new reversible architecture for prime-field modular inversion using divsteps is a significant algorithmic contribution, not just an optimization of existing circuits. The benchmarks are rigorous, comparing against established baselines and providing detailed resource accounting.
The paper provides sufficient detail on the circuit model, cost metrics, and search strategy to allow for reproduction of the framework. However, the specific LLM prompts and the exact "OpenEvolve" implementation details are not fully open-sourced in the text, which may limit immediate reproducibility for those without access to the specific tooling or LLM versions (Claude Mythos 5).
The framework is bounded by the task specification and the computational basis representation, excluding optimizations that rely on changing the encoding itself (e.g., coset representations). The reliance on a specific LLM (Claude Mythos 5) and the proprietary "OpenEvolve" framework may limit accessibility. Additionally, the TQ-cost metric is a simplification that may not capture all hardware-specific constraints like depth or routing.
This work has significant implications for the field of quantum computing, offering a scalable path to discovering efficient quantum algorithms. It demonstrates that LLMs can contribute to fundamental algorithmic design in domains with complex constraints, potentially accelerating the development of fault-tolerant quantum computers. The approach could be extended to other areas of AI for Science where programmatic evaluation is feasible. The paper presents a novel evolutionary framework using LLMs to discover state-of-the-art quantum circuits, achieving significant reductions in resource costs across multiple tasks and uncovering new reversible algorithms. This work bridges the gap between AI-driven code generation and rigorous quantum algorithm design, offering a powerful tool for future quantum computing research.
Few-step autoregressive video generators enable streaming generation, yet they run every token through the final denoising step, regardless of how much refinement it needs. We find this uniform computation is often unnecessary: many tokens already closely match their eventual full-step outputs in the initial steps, while others require refinement until the end. We introduce Speculative Forcing, a framework that treats intermediate clean estimates as drafts and predicts when individual tokens can be safely accepted early. Once accepted, a token is frozen as its final output and undergoes no further denoising, while its cached keys and values remain available as context for the remaining tokens. We explore two alternative verifier designs: a training-free one based on similarity to clean content and stability across steps, and a learned one that reads the frozen generator's intermediate features to estimate each token's remaining gap to its full-step output. Across short-video, long-video, and world generation, Speculative Forcing preserves the quality of full-step sampling while substantially reducing computation. On LongLive, it matches the four-step generator with ~1.7 denoising steps per token, running over 1.5x faster at 41.7 and 37.8 fps. On Lingbot-World 2.0, where camera motion continually changes the visible content, it maintains quality with ~2.5 denoising steps per token and runs 1.35x faster at 24.5 fps.
Primary: NAVER AI Lab
All Institutions: NAVER AI Lab
Speculative Forcing enables efficient autoregressive video generation by dynamically reducing denoising steps for tokens that require less refinement. The paper presents a novel and effective approach to optimizing the inference cost of video diffusion models, demonstrating substantial speedups while preserving quality across various video generation tasks.
The paper introduces "Speculative Forcing," a framework that adapts the concept of speculative decoding to autoregressive video generation. The core insight is that not all tokens in a video sequence require the full computational budget of the denoising process. The method treats intermediate clean estimates as "drafts" and employs a verifier to determine if a token can be safely accepted early. Two verifier designs are proposed: a training-free heuristic based on similarity and stability, and a learned verifier that predicts the remaining gap to the full-step output. This approach allows tokens to be frozen early, reducing the average number of denoising steps per token while maintaining quality. The methodology is sound, leveraging the structure of autoregressive diffusion models where early steps often converge for static or predictable content.
The experiments are conducted on recent, high-performance video generation models (LongLive and Lingbot-World 2.0). The results demonstrate significant speedups (1.5x to 1.35x) with minimal loss in quality, matching the performance of full-step sampling. The evaluation covers short-video, long-video, and world generation tasks, showing the method's versatility. The use of state-of-the-art baselines and the focus on practical metrics like FPS and steps-per-token makes the evaluation relevant and compelling.
The paper provides sufficient detail on the verifier designs and the integration with existing generators. However, as it relies on specific proprietary or recently released models (LongLive, Lingbot-World), full reproducibility may depend on the availability of these base models and their code. The training-free verifier is highly reproducible, while the learned verifier requires training data and specific feature extraction from the frozen generator.
The method is specific to autoregressive video generators that use a few-step denoising process. It may not apply directly to non-autoregressive or single-step video models. The speedup is dependent on the content; highly dynamic scenes may offer less benefit than static ones. The learned verifier adds a small overhead, which may negate gains in very low-latency scenarios.
This work has significant implications for the deployment of video generation models in real-time applications, such as interactive media, gaming, and live streaming. By reducing computational cost without sacrificing quality, it makes high-fidelity video generation more accessible and energy-efficient. It also opens avenues for further research into adaptive computation in generative models. Speculative Forcing enables efficient autoregressive video generation by dynamically reducing denoising steps for tokens that require less refinement. The paper presents a novel and effective approach to optimizing the inference cost of video diffusion models, demonstrating substantial speedups while preserving quality across various video generation tasks.
Configuring distributed LLM training across hundreds of GPUs requires jointly optimizing tensor, pipeline, expert, context, and data parallelism alongside micro-batch size, recomputation, sharding, and overlap, a combinatorial space where memory legality can hinge on a single byte of activation accounting. Today, teams navigate it through expensive multi-node trial and error, burning GPU-hours on crashes and suboptimal recipes. We present Primus Projection, which closes this loop with two coupled components. First, a hybrid projection engine that measures what it can and simulates what it cannot: GPU microbenchmarks on as few as a single device capture compute kernels and any in-scope communication; a sub-node downscale-measure-upscale workflow analytically restores out-of-scope collectives, FSDP overlap, and the full pipeline schedule at the multi-node target. A pure simulation mode runs entirely on CPU, enabling pre-silicon planning without any GPU. Second, the Primus Tuning Agent treats the projection engine as a fast scoring oracle (seconds per candidate instead of tens of minutes) and uses it to drive an intelligent search over the configuration space. A deterministic seed planner sweeps single axes; an LLM investigation loop proposes cross-axis combinations the planner cannot reach, with an analytic memory pre-filter that rejects infeasible candidates before any GPU time is spent. Validated on Llama 3.1 (dense) and Mixtral 8x22B (MoE) across two GPU generations, all projections stay within ~10% of measured multi-node throughput, and fidelity holds at production scale: 4.7% error on DeepSeek V3 671B at 1,024 GPUs and 1.9% on Qwen3-235B-A22B at 256 GPUs. In a Mixtral 8x22B case study, the Primus Tuning Agent discovers a configuration delivering +27% throughput over the published 4-node BF16 reference at the same scale, in under 30 minutes of single-node exploration.
Primary: Advanced Micro Devices (AMD)
All Institutions: Advanced Micro Devices (AMD)
The paper presents a highly effective hybrid system for optimizing distributed LLM training configurations by combining single-node micro-benchmarks with analytical simulation and LLM-driven search. Its rigorous validation on production-scale models and significant throughput improvements demonstrate substantial practical value for large-scale AI infrastructure, though its impact is somewhat constrained by its specific validation on AMD hardware.
The paper introduces "Primus Projection," a hybrid performance modeling framework that combines empirical micro-benchmarks on limited hardware (down to a single GPU) with analytical simulation for out-of-scope communication and pipeline scheduling. The methodology is rigorous, addressing the "combinatorial explosion" of distributed training configurations (TP, PP, EP, CP, DP, etc.) by creating a fast scoring oracle. The "downscale-measure-upscale" workflow is a clever engineering solution to the problem of requiring full-cluster access for profiling. The integration of an LLM-based tuning agent (using DSPy and ReAct-style reasoning) to navigate this configuration space is a modern and effective approach, leveraging the projection engine as a cheap surrogate to avoid expensive trial-and-error on real clusters. The memory pre-filtering mechanism is particularly strong, preventing OOM errors before any GPU time is spent.
The evaluation is extensive and convincing. The authors validate their projection engine against measured multi-node throughput on AMD MI325X and MI355X GPUs, achieving within ~10% error for standard models and as low as 1.9-4.7% error for production-scale models (DeepSeek V3, Qwen3-235B). The case study on Mixtral 8x22B demonstrates a +27% throughput improvement over published references using only single-node exploration. The comparison against analytical baselines (Calculon, Galvatron) clearly shows the superiority of the hybrid benchmark-anchored approach, particularly for MoE models where analytical models fail. The robustness analysis regarding MoE load balancing is a valuable addition, showing how the model handles imbalance.
The paper provides detailed algorithmic descriptions (Algorithm 1), hardware profile formats (YAML), and clear definitions of the search space. However, as an industry paper from AMD, the specific code implementation and hardware profiles are likely proprietary or not fully open-sourced, which may limit immediate reproducibility by external researchers without access to similar AMD hardware. The methodology is described well enough for replication in principle, but the "black box" nature of the LLM agent's prompts and the specific hardware calibration data are potential barriers.
The primary limitation is the vendor-specific validation; all experiments are on AMD Instinct GPUs and the ROCm stack. While the authors argue the method is vendor-agnostic, the lack of validation on NVIDIA or TPU hardware is a significant gap for a general ML audience. Additionally, the LLM agent's performance depends on the quality of the projection oracle; if the oracle has systematic biases (as noted with the optimistic/pessimistic offsets), the agent might converge to suboptimal local minima if the ranking is distorted, although the paper argues rank preservation is robust. Network contention is not modeled, which could be an issue in highly congested clusters.
This work has high practical impact for organizations training large-scale LLMs, as it reduces the cost and time of cluster configuration. It bridges the gap between theoretical performance modeling and practical engineering constraints. The use of LLMs for systems optimization is a growing trend, and this paper provides a concrete, high-fidelity example of its application. It could influence how future distributed training frameworks are designed, potentially integrating such projection oracles natively. The paper presents a highly effective hybrid system for optimizing distributed LLM training configurations by combining single-node micro-benchmarks with analytical simulation and LLM-driven search. Its rigorous validation on production-scale models and significant throughput improvements demonstrate substantial practical value for large-scale AI infrastructure, though its impact is somewhat constrained by its specific validation on AMD hardware.
Language captures only a subset of the perceptions, intuitions and intents that form meaning between speakers. What, then, are the limits of language? We take steps towards measuring this in LLMs as follows: for a set of concepts, we measure how well each concept can be transmitted between two LLMs as their capabilities scale. This requires novel measurement tools: (1) optimal messages: we introduce a prompt optimization algorithm that beats existing algorithms by up to 14% on benchmarks and accurately estimates prompt upper bounds. (2) A transmission measure: the sender is asked to label text; the receiver then tries to recover the sender's prompt, and recovery is measured; we find this correlates strongly with human label error. With these measures, we use observational scaling laws to study how transmissibility of a concept, conveyed with optimal messages, increases as transmitter and receiver capacity grows: does it asymptote below or above a threshold? Measuring 1.2k concepts in eight domains, we find concepts that (a) can be fully articulated; these concepts usually depend on local patterns, e.g. "joke ends with surprise"; (b) might be articulated by future, stronger models; these rely on whole-text structural patterns, e.g. "story's worldbuilding coherence"; (c) concepts that face articulability limits; these concepts' meaning, we find, is tied to identity and social-group awareness, e.g. "comic voice". Interesting, these same concepts are not all challenging for humans: humans have moderate-to-high levels of agreement in 76% of concepts that LLMs struggle with; articulation limits, we find, are executor relative. Finally, we characterize the nature of improvement in each case: while recoverable concepts can be articulated through definitions and explanations, irrecoverable concepts cannot be transmitted, even with examples, and might require novel learning mechanisms.
Primary: Stanford University
All Institutions: Stanford University
The paper introduces a rigorous framework for measuring the limits of language articulation in LLMs, combining novel prompt optimization with a transmission metric to categorize concepts by their articulability. By demonstrating that certain social and identity-based concepts are fundamentally harder for LLMs to transmit than others, despite high human agreement, the work provides critical insights into the boundaries of language-based AI control and alignment.
The paper introduces a novel framework for measuring the "articulability" of concepts in LLMs, decomposing the problem into message optimization and transmission measurement. The core methodological contribution is $\epsilon$-estimation, a prompt optimization algorithm that uses capture-recapture statistics to estimate the upper bound of prompt performance by modeling the space of "semantic units." This is a creative application of ecological statistics to prompt engineering. The second contribution is the "reconstruction bottleneck," a transmission metric where a receiver LLM attempts to reconstruct the sender's prompt from labeled examples. The methodology is rigorous, with clear definitions of upper/lower bounds (planted rules, permutation nulls) and a strong theoretical grounding in information theory (mutual information). The approach effectively separates the capability of the model from the clarity of the language used to convey a concept.
The experiments are extensive, covering 1.2k concepts across eight diverse domains (humor, math, code, etc.). The paper compares $\epsilon$-estimation against SOTA prompt optimizers like GEPA, showing up to 14% improvement. The scaling laws analysis is particularly strong, categorizing concepts into three regimes: fully articulable, potentially articulable with scaling, and fundamentally limited. The finding that "identity and social-group awareness" concepts are the hardest to articulate, while humans show high agreement on them, is a significant and surprising empirical result. The use of human inter-rater agreement (Krippendorff's alpha) as a ground truth for concept clarity adds robustness.
The paper provides detailed descriptions of the unit mining process, the capture-recapture estimator, and the reconstruction protocol. The use of standard benchmarks (AIME, HotpotQA, IFBench) and public datasets (McSweeney's, Reddit) aids reproducibility. However, the specific implementation details of the "unit" extraction and the exact prompt templates for the reconstruction bottleneck are not fully specified in the provided text, which may hinder exact replication. The code availability is not explicitly stated in the snippet, but the level of detail suggests it is likely released.
The primary limitation is the reliance on LLMs as both transmitters and receivers, which may not fully capture human-like articulation limits. The "unit" decomposition assumes that prompts can be atomized into independent semantic units, which may not hold for highly contextual or stylistic concepts. The capture-recapture estimator assumes exchangeability of units, which is a strong assumption. Additionally, the paper focuses on binary labeling tasks, which may not generalize to more complex, multi-class, or generative articulation tasks.
This paper has significant implications for AI alignment and interpretability. By identifying which concepts are "inarticulable" to LLMs, it highlights the limits of using language as a medium for control and evaluation. This is crucial for developing AI systems that operate on nuanced, subjective, or socially constructed concepts. The framework could be used to design better rubrics for LLM judges and to identify areas where human feedback is essential. It also provides a new lens for understanding the gap between human and machine understanding of language. The paper introduces a rigorous framework for measuring the limits of language articulation in LLMs, combining novel prompt optimization with a transmission metric to categorize concepts by their articulability. By demonstrating that certain social and identity-based concepts are fundamentally harder for LLMs to transmit than others, despite high human agreement, the work provides critical insights into the boundaries of language-based AI control and alignment.
Recent work has sought to transfer first-order nonconvex optimization guarantees to zeroth-order methods through gradient estimators, typically incurring a factor linear in $d$. \citet{kornowski2024} obtained an $\mathcal{O}(dδ^{-1}ε^{-3})$ oracle complexity for nonsmooth nonconvex optimization and asked whether its dependence on all three parameters is jointly optimal. Although the dimension dependence and the first-order dependence on $(δ,ε)$ are separately known to be unavoidable, a matching zeroth-order lower bound has remained open. We answer this question in the unrestricted full-space setting. For smooth stochastic nonconvex optimization, we establish a scalar-oracle lower bound of $Ω\!\left(dL_0^2\min\{ΔH,L_0^2\}/ε^4\right)$ assuming a smooth, globally lower-bounded population objective and a second-moment bound on sample Lipschitz constants. A smooth reduction then yields a lower bound for Goldstein stationarity, proving the joint minimax optimality of the dependence on $(d,δ,ε)$ under explicit high-dimensional and small-accuracy conditions. Crucially, our results allow algorithms to query arbitrary points in $\mathbb{R}^d$, adaptively select sample-function labels, reuse the same sampled function at any number of locations, and output points without spatial restrictions. Thus, unlike lower bounds based on bounded query regions or fresh evaluation noise, our guarantees apply to a fully adaptive, reusable-sample scalar oracle. We also establish a separate noiseless exact-value lower bound against randomized algorithms, and hence also against deterministic methods. Our proof combines a globally defined sequential nonconvex construction with an exact Gaussian scalar channel, multistage random-field estimates, and an antithetic finite-sample realization that preserves information hardness under adaptive reuse.
Primary: Nanjing University
All Institutions: Nanjing University, State Key Laboratory for Novel Software Technology
The paper establishes a joint minimax lower bound for stochastic zeroth-order nonconvex optimization, proving that the dependence on dimension, smoothness, and accuracy is jointly optimal under a fully adaptive, reusable-sample scalar oracle model. By constructing a global zero-chain with hidden workspace directions and employing information-theoretic tools to control adaptive sample reuse, the authors demonstrate that the oracle complexity scales as $\Omega(d L_0^2 \min\{\Delta H, L_0^2\} / \epsilon^4)$, resolving a key open question in the field and providing a rigorous theoretical limit for gradient-free optimization methods.
The paper addresses a significant gap in zeroth-order optimization theory: establishing a joint lower bound for the dependence on dimension ($d$), smoothness ($H$), and accuracy ($\epsilon$) in the stochastic nonconvex setting. The authors propose a novel construction that combines a global zero-chain geometry with a "moving well" mechanism to force sequential information acquisition. The core technical contribution is the proof that even with a fully adaptive, reusable-sample scalar oracle (where algorithms can query any point and reuse sample functions), the complexity scales as $\Omega(d L_0^2 \min\{\Delta H, L_0^2\} / \epsilon^4)$. The methodology is rigorous, utilizing Gaussian process representations, antithetic sampling to create a finite static oracle, and information-theoretic arguments (specifically spherical concentration and log-Sobolev inequalities) to bound the information gain per query. The construction is complex, involving careful management of global Lipschitz moments and spatial cutoffs to ensure the hardness persists globally rather than just locally.
As a pure theory paper, there are no empirical experiments. The "evaluation" is the mathematical proof of the lower bound. The rigor of the proof is the primary metric of quality here. The paper provides detailed appendices (referenced but not fully visible in the snippet, though the main text outlines the structure) to support the claims. The lack of empirical validation is standard for this subcategory of optimization theory.
The paper is highly reproducible in the sense that the mathematical results are deterministic and verifiable by other theorists. There is no code to reproduce. The constants and conditions are explicitly stated, allowing for independent verification of the bounds.
The lower bound holds under specific high-dimensional and small-accuracy conditions (explicitly stated as $d \ge C(B/\epsilon^2)(2+BL_0^2/\epsilon^4)$). The bound is for the smooth stochastic case, and the extension to nonsmooth Goldstein stationarity is via a reduction, which may not be tight in all regimes. The construction is quite intricate, making it difficult to generalize to other oracle models without significant modification. The constants involved are conservative (e.g., $2^{128}$), which limits immediate practical utility for tight algorithm design but is typical for lower bound constructions.
This paper settles an open question regarding the joint optimality of zeroth-order optimization complexities. It provides a definitive answer to whether the linear dependence on dimension $d$ is unavoidable in the stochastic nonconvex setting when using scalar oracles. This has implications for the design of gradient-free optimization algorithms, confirming that one cannot hope for sub-linear dimension dependence in the worst case under these assumptions. It strengthens the theoretical foundation for understanding the limitations of zeroth-order methods compared to first-order methods. The paper establishes a joint minimax lower bound for stochastic zeroth-order nonconvex optimization, proving that the dependence on dimension, smoothness, and accuracy is jointly optimal under a fully adaptive, reusable-sample scalar oracle model. By constructing a global zero-chain with hidden workspace directions and employing information-theoretic tools to control adaptive sample reuse, the authors demonstrate that the oracle complexity scales as $\Omega(d L_0^2 \min\{\Delta H, L_0^2\} / \epsilon^4)$, resolving a key open question in the field and providing a rigorous theoretical limit for gradient-free optimization methods.
We introduce an approach for self-supervised representation learning designed to be used as a tokenizer for generative modeling. Our objective supervises a representation loss by factorizing the pointwise mutual information of the data, which we estimate by taking information differences between conditional predictions from a learned generative model. We justify our algorithm theoretically and empirically, demonstrating that our generative approach to representation learning avoids dimensional collapse and feature suppression in cases where commonly used methods like DINO and LeJEPA fail. Moreover, we show our representation learning technique is scalable; expending more compute leads to predictably better latent tokenization. For the first time, we perform a careful analysis of the combined flop efficiency of tokenizer and generative model and demonstrate that our approach is a compute-optimal improvement over direct diffusion for pixel-space generation, obtaining an FID-50K of 2.18 with JiT-L. Applied to ImageNet and the text-to-image dataset GPIC, we show tokenizer encoder scaling laws for diffusability, where extrapolating our approach achieves representations in 7x fewer FLOPs than DINOv2 on ImageNet and 8x fewer FLOPs than WebSSL on GPIC.
Primary: Massachusetts Institute of Technology
All Institutions: Massachusetts Institute of Technology
[One sentence main contribution]. The paper introduces RePR, a generative representation learning method that factorizes Pointwise Mutual Information to create scalable, feature-suppression-robust tokenizers, demonstrating that this approach yields compute-optimal improvements over direct pixel diffusion for image generation.
The paper proposes RePR (Relative Prediction Representations), a self-supervised representation learning method that explicitly factorizes the Pointwise Mutual Information (PMI) of the data using a generative model to estimate PMI. The core innovation is replacing the discriminative InfoNCE loss (which relies on negative sampling and is prone to feature suppression) with a generative objective that estimates PMI via the ELBO of a diffusion model. The method introduces "Timestep Weighting" to bias the PMI estimation toward semantically relevant noise scales and "Bootstrapped PMI Estimation" to make the likelihood estimation tractable for high-dimensional inputs by decomposing the PMI into incremental token predictions. The architecture uses a Masked Autoencoder (MAE) variant with a diffusion head for the prediction model and a separate representation encoder. The theoretical grounding is strong, linking the method directly to the information-theoretic properties of contrastive learning while avoiding its practical pitfalls.
The experiments are rigorous and address a critical gap in the field: the compute efficiency of tokenizers. The authors demonstrate that RePR encoders exhibit predictable scaling laws for "diffusability" (how well a generative model can learn from the latent space). They show that RePR achieves DINOv2-level performance with 7x fewer FLOPs on ImageNet and 8x fewer than WebSSL on GPIC. A key result is the demonstration that using RePR with Latent Forcing yields a better FID-50K (2.18) than direct pixel diffusion (JiT-L) at equal total training FLOPs, proving that representation learning is a compute-optimal component of the generative pipeline. The ablation studies on masking strategies, loss weights, and iterative feature prediction are thorough. The feature suppression experiment using watermarked images effectively highlights the robustness of the generative PMI objective compared to contrastive baselines like DINOv3 and LeJEPA.
The paper provides a GitHub link and detailed implementation notes in the appendix, including architectural specifics (RoPE, register tokens, layer splits) and training hyperparameters. The use of standard components (ViT, MAE, Diffusion) aids reproducibility. However, the specific "Latent Forcing" method is cited as a separate work (Baade et al.), which may introduce dependency on external codebases not fully detailed here. The FLOP calculations are explicitly defined, which is crucial for the claims made.
The method relies on a diffusion model for PMI estimation, which adds computational overhead during the representation learning phase, although the paper argues this is offset by downstream efficiency. The scaling laws are extrapolated, and the "7x/8x" claims depend on the accuracy of these power-law fits. The method is primarily evaluated on image generation; applicability to other modalities (audio, video) is mentioned but not extensively tested. The reliance on a specific masking strategy and timestep weighting scheme requires careful tuning.
This paper has high potential impact by providing a theoretically grounded, scalable alternative to contrastive representation learning for generative tasks. It challenges the prevailing assumption that discriminative objectives are sufficient for tokenizers and offers a path to more compute-efficient generative models. The demonstration of compute-optimal generation via learned representations could influence the design of future foundation models, encouraging a shift towards generative self-supervision for representation learning. [One sentence main contribution]. The paper introduces RePR, a generative representation learning method that factorizes Pointwise Mutual Information to create scalable, feature-suppression-robust tokenizers, demonstrating that this approach yields compute-optimal improvements over direct pixel diffusion for image generation.
Textual reasoning before acting has improved vision-language-action (VLA) models, but at the cost of latency: every action conditioned on the reasoning text must wait for autoregressive decoding. We show that actions do not need to wait for or condition on the generated reasoning to benefit from it, because the reasoning supervision already improves the backbone representation. We introduce TacitVLA, a reasoning VLA whose action expert never attends to the reasoning it generates for the current call. On long-horizon tasks where the reasoning text also serves as episodic memory, it can generate each entry during action execution rather than before acting. We evaluate it in a controlled study on four benchmarks spanning manipulation and autonomous driving, memory and non-memory tasks: RoboMME, RMBench, LIBERO, and PhysicalAI-AV. On all four, TacitVLA matches the performance of its explicit-reasoning counterpart at up to 3.4x lower latency, and matches the latency of the same architecture trained without reasoning at up to +17.1% success. In absolute terms, it surpasses the best previously published results on the RoboMME Counting suite (82.7% vs. 66.8%) and RMBench (77.5% vs. 56.5%), and reaches 72.9% on LIBERO-Plus and a 52.5% scene score on the public split of the PhysicalAI-AV closed-loop driving challenge. Project page: https://tacitvla.github.io.
Primary: NVIDIA
All Institutions: NVIDIA
TacitVLA demonstrates that reasoning supervision improves VLA performance through representation learning rather than explicit inference-time conditioning, enabling real-time deployment with up to 3.4x lower latency. This paper provides a rigorous, controlled study across manipulation and driving benchmarks, showing that masking the action expert's attention to reasoning tokens retains the performance benefits of reasoning while eliminating the autoregressive decoding bottleneck, a significant contribution to the efficiency of embodied AI systems.
The paper proposes TacitVLA, a method that decouples the *generation* of reasoning text from the *conditioning* of actions on that text. The core insight is that the benefit of reasoning in Vision-Language-Action (VLA) models primarily stems from the representation learning induced by the reasoning supervision signal (cross-entropy loss on text tokens) during training, rather than the explicit attention to the generated text tokens during inference. By masking the action expert's attention to the reasoning tokens during both training and inference, the model retains the improved backbone representations without incurring the autoregressive decoding latency. For memory tasks, the paper introduces an asynchronous generation strategy where the reasoning text (ledger entry) is generated during the execution of the current action chunk, ensuring it is available for the next call without blocking the current action. This is a clever architectural optimization that leverages the existing Mixture-of-Transformers structure.
The evaluation is rigorous and comprehensive, covering four distinct benchmarks: RoboMME (memory/counting), RMBench (long-horizon manipulation), LIBERO/LIBERO-Plus (standard manipulation/robustness), and PhysicalAI-AV (autonomous driving). The controlled study comparing Explicit Reasoning, TacitVLA, and No Reasoning is well-designed, isolating the effect of attention masking. The results show that TacitVLA matches the performance of Explicit Reasoning while achieving the latency of the No Reasoning baseline (or slightly higher due to deferred generation on memory tasks). The latency improvements are significant (up to 3.4x), which is critical for real-time deployment. The performance gains over the No Reasoning baseline (up to +17.1% success) confirm the value of the reasoning supervision.
The paper provides detailed implementation details in the appendix, including architecture specifics, training hyperparameters, data sources, and evaluation protocols. The use of standard benchmarks (LIBERO, RoboMME, RMBench) and public datasets (PhysicalAI-AV public split) enhances reproducibility. The code and project page are available. However, the driving dataset is proprietary (3.8M events), which limits full reproducibility for that specific benchmark, though the methodology is clear.
The primary limitation is the reliance on high-quality reasoning annotations for training. If the reasoning data is noisy or incorrect, the representation learning benefit may diminish. Additionally, the asynchronous generation on memory tasks assumes that the generation time fits within the action execution window; if the reasoning text becomes very long or the hardware is slower, this assumption may break, leading to latency spikes. The paper also notes that on non-memory tasks, the reasoning text is discarded, which might miss opportunities for interpretability or safety checks that could be performed post-hoc.
This work has significant implications for the deployment of VLA models in real-world robotics and autonomous driving, where latency is a critical constraint. By demonstrating that reasoning can be "internalized" into the model's representations without the inference cost, it paves the way for more efficient and responsive embodied AI systems. The approach is generalizable to other architectures that use reasoning as a training target, potentially influencing future designs of foundation models for robotics. TacitVLA demonstrates that reasoning supervision improves VLA performance through representation learning rather than explicit inference-time conditioning, enabling real-time deployment with up to 3.4x lower latency. This paper provides a rigorous, controlled study across manipulation and driving benchmarks, showing that masking the action expert's attention to reasoning tokens retains the performance benefits of reasoning while eliminating the autoregressive decoding bottleneck, a significant contribution to the efficiency of embodied AI systems.
Generating information-rich visual artifacts such as slides, posters, and dashboards as code is appealing because the outputs are structured, editable, and verifiable. Yet code models cannot see how their choices of layout and color come together on the canvas, often limiting spatial planning and aesthetic quality. Image generators excel at visual design but can misrender requested content, making their outputs unreliable targets for direct reproduction. We therefore introduce AesCode, which generates code grounded in the content prompt while using an image generated from the same prompt as an aesthetic guidance. To do so, AesCode represents the target as a design graph that describes the whole canvas and makes each property attributable. From this graph, we derive complementary cross-modal rewards: deterministic verifiers score properties parsable from code and rendering, while a VLM judge assesses the non-parsable ones with a sample-specific Visual Graph Rubric (VGR) tied to graph elements and relations. We optimize these decoupled rewards through reinforcement learning with per-channel normalization to jointly improve spatial planning, aesthetic quality, and verifiable correctness. To evaluate this setting, we build a benchmark that scores verifiable correctness and visual quality separately, with visual scores highly consistent with human judgments. We find that reference images substantially improve the visual quality of code generated by existing models, and ablations show that AesCode benefits from references both as model input and as a source of visual rewards. Relative to same-scale Qwen3-VL baselines given the same references, AesCode-8B and AesCode-32B improve visual quality by 31.1 and 22.4 percentage points, reaching scores comparable to those of closed-source frontier models such as GPT-5.5 and Claude Opus 4.8.
Primary: Microsoft
All Institutions: Microsoft, Yale University, Nanyang Technological University, University of Pittsburgh, Shanghai Jiao Tong University, Fudan University
AesCode introduces a decoupled cross-modal reward system using a Design Graph representation to optimize aesthetic code generation via reinforcement learning. The paper presents a rigorous methodology that effectively combines deterministic verification with VLM-based aesthetic judgment, achieving state-of-the-art results that rival frontier closed-source models, thereby validating the potential of code-based approaches for high-fidelity visual design.
The paper proposes AesCode, a framework for generating visual artifacts (slides, posters) as code. The core innovation is the "Design Graph" representation, which decomposes the visual canvas into attributable properties. This allows for a decoupled reward system: deterministic verifiers check parsable properties (e.g., text presence, layout constraints), while a Vision-Language Model (VLM) judge assesses non-parsable aesthetic qualities using a sample-specific Visual Graph Rubric (VGR). The training utilizes Reinforcement Learning (RL) with per-channel normalization to optimize these distinct reward signals simultaneously. This approach effectively bridges the gap between the structural reliability of code generation and the aesthetic fidelity of image generation.
The authors introduce a new benchmark that separates verifiable correctness from visual quality, noting high consistency with human judgments. Experiments show that AesCode-8B and AesCode-32B outperform same-scale Qwen3-VL baselines by 31.1 and 22.4 percentage points in visual quality, respectively, when given reference images. The results claim parity with closed-source frontier models like GPT-5.5 and Claude Opus 4.8. The ablation studies confirm the utility of reference images both as input context and as a source for visual rewards.
The paper provides a project page and likely code release (implied by the GitHub URL structure). The methodology is detailed enough to understand the RL setup and reward decomposition. However, the reliance on a VLM judge for aesthetic scoring introduces a dependency on the specific VLM used, which may vary in robustness.
The primary limitation is the dependency on high-quality reference images for optimal performance; without them, the aesthetic guidance is lost. Additionally, the "Design Graph" construction may be complex for highly unstructured or free-form artistic layouts, potentially limiting generalizability to strict UI/dashboard design. The evaluation relies heavily on VLM-based scoring, which, while consistent with humans, is not a ground-truth metric.
This work has significant implications for the "code-as-design" paradigm, making it viable for professional-grade visual content creation. By decoupling aesthetic and functional rewards, it offers a template for other multimodal generation tasks where structural integrity and perceptual quality are distinct objectives. It could drive adoption of code-based design tools in enterprise settings where editability is crucial. AesCode introduces a decoupled cross-modal reward system using a Design Graph representation to optimize aesthetic code generation via reinforcement learning. The paper presents a rigorous methodology that effectively combines deterministic verification with VLM-based aesthetic judgment, achieving state-of-the-art results that rival frontier closed-source models, thereby validating the potential of code-based approaches for high-fidelity visual design.
Terminal agents have shown strong abilities in complex coding tasks. However, they often struggle to recover from errors during long-running execution (e.g., command errors, faulty implementations), eventually leading to task failures. To this end, we present ReTerm, a self-improving terminal agent trained by learning how to recover from its own failed terminal tasks. To support its training, we construct a customized data pipeline and produce ReTerm-10K, a high-quality dataset that provides effective recovery demonstrations for SFT cold start. Furthermore, we propose ReTerm-GRPO, an online RL algorithm that forks from the agent's own failure states to generate multiple independent self-recovery attempts. By learning from whether these attempts successfully complete the task, the agent self-improves through its own recovery experience. Built on Qwen3.5-9B, our ReTerm-9B achieves 34.8% Pass@1 on Terminal-Bench 2.0, improving over the base model by 14.6 points. Further analysis shows that our method achieves consistently higher recovery rates across diverse error categories compared with standard GRPO training. All code, data and model are released at https://github.com/twothsnd/ReTerm.
Primary: Shanghai Jiao Tong University
All Institutions: Fudan University, MMLab, Shanghai Jiao Tong University, The Chinese University of Hong Kong
ReTerm introduces a recovery-oriented training framework for terminal agents, combining a curated dataset of recovery trajectories with an online RL algorithm that learns from self-generated failure states. The paper demonstrates that explicitly training agents to recover from errors leads to substantial performance gains on terminal and coding benchmarks, offering a new perspective on agent self-improvement beyond simple task completion.
The paper proposes ReTerm, a framework for training terminal agents to recover from their own execution errors. The methodology consists of two main components: (1) ReTerm-10K, a dataset constructed by collecting failed trajectories from a base model (Qwen3.5-9B), restoring the environment state, and using a stronger teacher model (DeepSeek-V4-Pro) to generate recovery continuations, which are then filtered for quality; and (2) ReTerm-GRPO, an online reinforcement learning algorithm that forks from the agent's own failure states during training to generate multiple independent self-recovery attempts. The use of a "recovery task queue" to manage the distribution of failure states across training steps is a practical engineering contribution that addresses the sparsity of failure events in RL. The approach is logically sound, addressing the specific gap of "recovery" rather than just "task completion," and the distinction between SFT cold-start and online RL self-improvement is well-motivated.
The experiments are comprehensive, evaluating the model on Terminal-Bench 2.0, OpenThoughts-TBLite, SWE-bench Verified, and out-of-domain tasks like AIME and SpreadsheetBench. The results show significant improvements over the base model (Qwen3.5-9B) and standard GRPO baselines. The ablation study clearly isolates the contribution of the recovery-specific RL component. The error analysis section, which categorizes errors into planning, tool use, implementation, verification, and resource management, provides valuable insight into where the model improves. The case studies are detailed and illustrative of the recovery process. The performance gains are substantial (e.g., +14.6 points on Terminal-Bench 2.0), suggesting a meaningful impact on the state of the art for 9B-scale models.
The authors release code, data, and models, which is a strong positive for reproducibility. The paper provides detailed descriptions of the data pipeline, filtering criteria, and training hyperparameters. The use of specific model versions (Qwen3.5-9B, DeepSeek-V4-Pro) and benchmarks (Terminal-Bench 2.0) allows for clear replication. The prompt templates for data curation and error analysis are included in the appendix, further enhancing transparency.
The method relies on a stronger teacher model (DeepSeek-V4-Pro) for the SFT cold-start phase, which may limit scalability if such a teacher is not available. The recovery task queue mechanism adds complexity to the training loop. The evaluation is primarily on terminal/coding tasks; while out-of-domain generalization is tested, the core contribution is specific to interactive terminal environments. The paper does not extensively discuss the computational cost of maintaining and sampling from the recovery task queue compared to standard RL.
This work has significant implications for the development of autonomous coding agents. By focusing on recovery, it addresses a critical failure mode in long-horizon tasks. The techniques could be extended to other interactive domains where agents must correct their own mistakes, such as robotics or GUI agents. The release of the ReTerm-10K dataset and the ReTerm-GRPO algorithm provides a valuable resource for the community. ReTerm introduces a recovery-oriented training framework for terminal agents, combining a curated dataset of recovery trajectories with an online RL algorithm that learns from self-generated failure states. The paper demonstrates that explicitly training agents to recover from errors leads to substantial performance gains on terminal and coding benchmarks, offering a new perspective on agent self-improvement beyond simple task completion.
The success of autoregressive language models trained with Chain-of-Thought~(CoT) supervision raises a fundamental question: How simple can a next-token predictor be while still enabling the efficient simulation of general computation through its autoregressively generated CoT? Surprisingly, we show that a single neuron suffices. For a broad family of activations, including threshold and ReLU, a single autoregressive neuron can simulate any feedforward circuit with the same activation. Specifically, a size-$s$ circuit on $n$ inputs can be simulated by a neuron of dimension $O(ns+s^2)$ using $O(s^2)$ generation steps, with both bounds independent of circuit depth and tight under natural assumptions. For threshold activations, this improves upon the simulation of Joshi et al. (COLT 2025), whose bounds scale exponentially with circuit depth. Consequently, autoregressive threshold neurons can simulate polynomial-time Turing machines, while remaining efficiently learnable from CoT supervision. Our main expressivity result relies on a novel connection to Sidon sets from additive combinatorics. As an experimental illustration, we efficiently train a single autoregressive ReLU neuron to mimic a neural network teacher with nontrivial test accuracy on CIFAR-10.
Primary: Weizmann Institute of Science
All Institutions: Weizmann Institute of Science, Apple
The paper demonstrates that a single autoregressive neuron can efficiently simulate general computation via Chain-of-Thought, leveraging Sidon sets to achieve depth-independent polynomial bounds. This theoretical breakthrough clarifies the role of CoT in enabling simple models to perform complex tasks, with significant implications for understanding the efficiency and learnability of autoregressive language models.
The paper introduces a rigorous theoretical framework for the expressivity of autoregressive single-neuron models. The core innovation is the use of Sidon sets (additive combinatorics) to construct collision-free encodings of circuit weights within a single shared parameter vector. This allows a single neuron to simulate feedforward circuits of arbitrary depth with polynomial bounds on dimension and generation steps, independent of circuit depth. This is a significant improvement over prior work (Joshi et al., 2025) which suffered from exponential scaling with depth. The methodology is mathematically sound, providing tight upper and lower bounds under natural assumptions.
The experimental section is minimal but illustrative. It demonstrates that a single autoregressive ReLU neuron can be trained to mimic a CNN teacher on CIFAR-10 using activation traces. While the results (99% agreement with teacher) are strong, the experiment is a toy setup designed to validate the theory rather than a competitive benchmark. It does not compare against standard baselines or evaluate on unseen tasks, limiting its practical impact.
The theoretical proofs are detailed and self-contained. The experimental setup is described with sufficient detail (Adam optimizer, sparse implementation, 4000 trajectories) to be reproducible, though the code is not explicitly linked in the provided text.
The primary limitation is the gap between theory and practice. The simulation requires a specific feature map and structured CoT traces that are not naturally available in standard LLM training. The experiment is a controlled simulation, not a demonstration of a practical LLM architecture. The lower bounds rely on the "direct edge alignment" assumption, which may not hold for all potential simulation strategies.
This paper provides a foundational theoretical insight into why Chain-of-Thought (CoT) is effective: it allows simple predictors to simulate complex computations by exposing intermediate states. It bridges the gap between circuit complexity and neural network learning, offering a rigorous basis for understanding the efficiency of autoregressive models. It may influence future work on minimal architectures and the design of CoT supervision signals. The paper demonstrates that a single autoregressive neuron can efficiently simulate general computation via Chain-of-Thought, leveraging Sidon sets to achieve depth-independent polynomial bounds. This theoretical breakthrough clarifies the role of CoT in enabling simple models to perform complex tasks, with significant implications for understanding the efficiency and learnability of autoregressive language models.
Real-world tables are often heterogeneous, combining numerical features with free-form string columns. Table foundation models (TFMs) excel on numerical tables but require strings to be encoded numerically, while large language models (LLMs) hold semantic knowledge but are not designed for tabular predictions. Most existing bridges use LLMs as plug-in encoders for cell strings, yet how to extract representations effectively for tabular learning remains underexplored. We show that the interface between LLMs and tables matter as much as the LLM itself. Applied cell-wise, larger LLMs do not give better features. With our interface, the same LLMs improve steadily with size. TILE (Table-Interfaced LLM Embeddings) combines four ingredients: row serialization to provide cross-column context, intermediate-layer extraction to avoid layers specialized for next-token prediction, columnar pooling to remove mean pooling's length bias, and by-pass features to route numbers and categories past the LLM. Paired with TFMs, TILE sets a new state of the art for tables with strings. Across 100 datasets, it beats the strongest baseline on 89% of splits, with gains across 7 LLMs and 10 tabular learners, and ranks first on the 108-dataset STRABLE benchmark.
Primary: Inria SODA
All Institutions: Inria SODA
The paper introduces TILE, a robust interface for bridging LLMs and Table Foundation Models, demonstrating that proper serialization, layer selection, and pooling are critical for leveraging LLM scale in tabular learning. By addressing the specific limitations of cell-wise encoding and last-layer extraction, TILE achieves state-of-the-art results on heterogeneous tables, offering a practical and theoretically grounded solution to a significant gap in applied machine learning.
The paper proposes TILE, a framework for integrating Large Language Models (LLMs) with Table Foundation Models (TFMs) by treating the "interface" as a critical design space. The methodology is grounded in a clear diagnosis of why naive LLM integration fails: cell-wise isolation lacks context, and last-layer extraction captures next-token prediction biases rather than transferable semantics. The four proposed components—row serialization, intermediate-layer extraction, columnar pooling, and by-pass features—are logically derived from these failures. The "by-pass" mechanism is particularly insightful, acknowledging that LLMs are poor at numerical precision and that exact categorical identity is often more useful to tabular learners than semantic embeddings. The adaptive PCA dimensionality based on table width is a practical engineering choice that addresses the feature-count constraints of TFMs.
The experimental evaluation is extensive and rigorous. The authors test across 100 datasets from multiple sources (TextTabBench, CARTE, WikiDBs) and validate on the external STRABLE benchmark. The comparison against 30 baselines, including state-of-the-art TFMs (TabPFN, TabICL, Causilo) and LLM-based methods (TabLLM, ContextTab), is comprehensive. The key finding that "larger LLMs do not give better features" when applied cell-wise, but do so when using TILE, is a significant empirical contribution that justifies the method. The ablation studies effectively isolate the contribution of each component (context, depth, pooling).
The paper provides sufficient detail for reproduction, including specific layer indices for extraction, PCA component counts, and cardinality thresholds. The use of standard, off-the-shelf LLMs and TFMs lowers the barrier to entry. However, the lack of a provided code repository in the text (and the "none" for PROJECT_URL) slightly hinders immediate reproducibility, though the method is described clearly enough to be implemented by a skilled practitioner.
The method relies on the availability of a suitable TFM; if the TFM is weak, the gains from TILE may be diminished. The row serialization approach may struggle with extremely wide tables where the context window of the LLM is exceeded, although the adaptive PCA helps with the output side. The paper focuses on classification and regression; other tabular tasks (e.g., anomaly detection) are not explored.
This work has high practical impact for industries dealing with heterogeneous tabular data (finance, healthcare, e-commerce). It provides a clear recipe for leveraging existing LLMs without fine-tuning them for tabular tasks, which is cost-effective. The insight that the interface matters more than the model size for this specific task will likely influence future research in hybrid tabular-language models. The paper introduces TILE, a robust interface for bridging LLMs and Table Foundation Models, demonstrating that proper serialization, layer selection, and pooling are critical for leveraging LLM scale in tabular learning. By addressing the specific limitations of cell-wise encoding and last-layer extraction, TILE achieves state-of-the-art results on heterogeneous tables, offering a practical and theoretically grounded solution to a significant gap in applied machine learning.
In humans, proprioception corresponds to the sense of self-movement or position, e.g. understanding the position of one's arm. This sense is often less well-understood and in popular understanding sometimes conflated with introspection. But, notably, proprioception is a separate sense, with dedicated receptors and verifiable outcomes. On the other hand though, language models cannot observe their own internal states during training (and for example answer the question which experts activated in a certain layer of an MoE) and so cannot accurately report on them. Yet, such information is, by virtue of being internal to the model, both accessible, and readily constructed into verifiable signals. We argue that this provides the basis for a digital analog of proprioception. As such, we train models through SFT and on-policy training to equip them with this ability. We show that modern models can be trained to understand a range of subtypes of proprioception, such as routing decision awareness, token entropy, decision depth or logit lens information, without affecting model capabilities. We then run these models through a broad evaluation of situational awareness, legibility, and epiphenomenality, to evaluate in what ways access to proprioception changes model behavior.
Primary: Tübingen AI Center
All Institutions: Tübingen AI Center, ELLIS Institute T, Max Planck Institute for Intelligent Systems
The paper introduces a rigorous framework for training language models to possess "digital proprioception," enabling them to accurately report on verifiable internal states such as MoE routing decisions and logit distributions. By distinguishing this capability from unverifiable introspection and employing on-policy training to prevent label staleness, the authors demonstrate that models can acquire precise self-knowledge without degrading general capabilities, offering a promising new avenue for enhancing model interpretability and safety through structured self-monitoring.
The paper proposes a novel training paradigm for "digital proprioception," distinguishing it from introspection by focusing on verifiable, internal, non-transparent states. The methodology involves a two-stage training process: Supervised Fine-Tuning (SFT) on templated questions and On-Policy SFT (inspired by DAgger) where labels are generated from the live forward pass to avoid stale labels. This on-policy approach is critical for maintaining the integrity of the proprioceptive signal as model weights update. The authors address practical challenges like kernel non-determinism in MoE routing and multi-token answer supervision. The theoretical framing, separating proprioception (verifiable internal states) from introspection (unverifiable self-reports), is a strong conceptual contribution that clarifies a muddy area of AI safety research.
The experiments are rigorous, testing three distinct domains of proprioception: MoE routing decisions, token logits/entropy, and logit lens information. The models used (Llama 2.5, OLMoE, Qwen, GLM) are modern and relevant. Key results show that models can learn to report these internal states with high accuracy without degrading general capabilities. The evaluation of behavioral changes (situational awareness, legibility) is thorough, showing that proprioception training does not negatively impact interpretability (probe accuracy) or safety metrics. The edge case analysis regarding routing non-determinism is particularly insightful, demonstrating the limits of the model's self-knowledge when the underlying computation is unstable.
The paper provides detailed descriptions of the training setup, including loss weighting strategies, context lookup tables to prevent shortcut learning, and synchronization protocols for on-policy training. However, specific hyperparameters for the on-policy stage and the exact implementation of the "live" label extraction are described at a high level. The code is not explicitly linked in the provided text, but the methodological details are sufficient for a skilled practitioner to attempt reproduction. The reliance on specific MoE architectures (like Qwen3.6) may limit immediate reproducibility for those without access to such models, but the principles are generalizable.
The primary limitation is the scope of "proprioception" defined here, which is restricted to valence-neutral, mechanistic queries. It does not address more complex or subjective internal states. Additionally, the non-determinism of MoE routing kernels poses a fundamental upper bound on the accuracy of routing proprioception, as noted in the paper. The behavioral changes observed are subtle, and it remains unclear if this training leads to significant long-term safety benefits or risks, as the effects on "legibility" were neutral.
This work has significant implications for AI safety and interpretability. By providing a method to train models to report on their internal states with verifiable ground truth, it offers a new tool for debugging and monitoring LLMs. It challenges the notion that self-knowledge in LLMs is purely mimetic or unreliable, suggesting that specific, structured self-knowledge can be engineered. This could lead to more transparent and controllable AI systems, where models can explicitly report on their decision-making processes (e.g., which experts were activated) in a way that is auditable. The paper introduces a rigorous framework for training language models to possess "digital proprioception," enabling them to accurately report on verifiable internal states such as MoE routing decisions and logit distributions. By distinguishing this capability from unverifiable introspection and employing on-policy training to prevent label staleness, the authors demonstrate that models can acquire precise self-knowledge without degrading general capabilities, offering a promising new avenue for enhancing model interpretability and safety through structured self-monitoring.
Rapid advances in large language models have made software engineering agents capable of tackling complex coding tasks. Within a containerized environment, a coding agent receives tool responses or execution results, and determines its later actions by those earlier results. However, most existing coding agents never look ahead: their policies do not anticipate the consequences of actions, and the terminal reward barely attributes a failure to certain earlier steps, so an ill-judged edit can corrupt the repository state irreversibly before any signal arrives. Recent code world models aim to address these challenges, but they are either trained and then frozen before reinforcement learning (RL), or reduced to an auxiliary next-token loss on the environment's outputs. We present ANTE (Anticipate, Then Execute), an agentic RL framework that verifies the policy's pre-action predictions by real execution and turns them into per-step credit for its actions and trust-or-abstain decisions. With an internal belief stated before each action, Belief-Anchored Policy Optimization (BAPO) turns the settled predictions into per-step advantages by anchoring credit on the policy's own forecasts. To turn the trained forecasts into better actions at inference, thereby facilitating effective test-time scaling, ANTE-TTO (test-time optimization) selects among candidate actions without an external scorer. ANTE raises Qwen3-32B from 25.0% to 57.2% pass@1 on SWE-bench Verified, 6.2% above a matched GRPO baseline trained without the belief state; its world model's accuracy climbs from 44% to 69% during RL, and the learned foresight capability transfers across harnesses and out-of-distribution domains. In summary, we aim to build a code world model that knows what has happened, what holds now, and what the next action will achieve.
Primary: UC Berkeley
All Institutions: UC Berkeley, Google DeepMind
[One sentence main contribution]. The paper presents ANTE, a framework that uses verified internal world models to improve credit assignment in coding agents via Belief-Anchored Policy Optimization, significantly boosting performance on SWE-bench Verified. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The technical contribution is substantial, addressing a core challenge in agentic RL: the misalignment between sparse terminal rewards and the need for fine-grained action evaluation. By anchoring credit on the agent's own forecasts, ANTE creates a dense, self-supervised signal that guides policy optimization more effectively than standard methods. The methodology is innovative, moving beyond treating world models as static predictors or auxiliary losses, and instead integrating them dynamically into the RL loop. The experimental results are strong, showing a large absolute gain in pass@1 on a difficult benchmark, which is a key metric for coding agents. The significance to the field is high, as it provides a scalable and effective approach to training agents that can anticipate consequences, a crucial capability for real-world deployment. The work bridges the gap between world modeling and policy optimization, offering a new paradigm for agentic learning.
The paper introduces ANTE (Anticipate, Then Execute), a framework that integrates a learned internal world model into the reinforcement learning loop for coding agents. The core innovation is "Belief-Anchored Policy Optimization" (BAPO), which uses the agent's own pre-action predictions (beliefs) about the environment's state to compute per-step credit assignment, rather than relying solely on sparse terminal rewards. This addresses the credit assignment problem in long-horizon agentic tasks. The method also includes ANTE-TTO (Test-Time Optimization), which leverages the trained world model to select among candidate actions at inference time without an external verifier. The methodology is sound, combining standard RL techniques with a novel use of self-predictive capabilities for credit assignment. The approach is distinct from prior work that freezes world models before RL or uses them only as auxiliary losses.
The experiments are conducted on SWE-bench Verified, a standard and challenging benchmark for software engineering agents. The paper reports a significant improvement in pass@1 from 25.0% to 57.2% using Qwen3-32B, outperforming a matched GRPO baseline by 6.2%. The world model's accuracy improves from 44% to 69% during RL, demonstrating that the foresight capability is learned effectively. The paper also claims transferability of the learned foresight across different harnesses and out-of-distribution domains, which is a strong claim supported by the reported results. The evaluation is rigorous and uses a large, capable base model, making the results highly relevant to current state-of-the-art practices.
The paper provides detailed descriptions of the BAPO algorithm and the ANTE-TTO process. However, specific hyperparameters, training schedules, and implementation details for the world model architecture are likely in the appendix (referenced but not fully visible in the prompt). The use of standard benchmarks and open-source base models (Qwen3) aids reproducibility. The absence of a provided code repository in the extracted text is a minor drawback, but the methodological clarity is high.
The primary limitation is the computational cost of maintaining and updating a world model during RL, which may be prohibitive for smaller models or resource-constrained settings. The reliance on a large base model (Qwen3-32B) means the results may not directly translate to smaller agents. Additionally, the "trust-or-abstain" decision mechanism's robustness to adversarial or highly noisy environment feedback is not deeply explored. The paper focuses on coding tasks, and generalization to other agentic domains (e.g., robotics, web navigation) is implied but not extensively tested.
This work has significant implications for the development of autonomous agents in complex, stateful environments. By improving credit assignment through internal world models, it offers a pathway to more reliable and efficient agent training. The technique of using self-predictions for credit assignment could be generalized to other domains where sparse rewards are a bottleneck, such as robotics or scientific discovery. The improvement in SWE-bench Verified performance suggests practical utility in software engineering automation. [One sentence main contribution]. The paper presents ANTE, a framework that uses verified internal world models to improve credit assignment in coding agents via Belief-Anchored Policy Optimization, significantly boosting performance on SWE-bench Verified. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The technical contribution is substantial, addressing a core challenge in agentic RL: the misalignment between sparse terminal rewards and the need for fine-grained action evaluation. By anchoring credit on the agent's own forecasts, ANTE creates a dense, self-supervised signal that guides policy optimization more effectively than standard methods. The methodology is innovative, moving beyond treating world models as static predictors or auxiliary losses, and instead integrating them dynamically into the RL loop. The experimental results are strong, showing a large absolute gain in pass@1 on a difficult benchmark, which is a key metric for coding agents. The significance to the field is high, as it provides a scalable and effective approach to training agents that can anticipate consequences, a crucial capability for real-world deployment. The work bridges the gap between world modeling and policy optimization, offering a new paradigm for agentic learning.
In multi-group learning, a single predictor must perform nearly as well as the best hypothesis in a class on every group in a rich, possibly infinite and overlapping, collection of subpopulations. The known algorithms with the optimal sample complexity keep track of every pair of a group and a hypothesis, which is impossible for the large classes of interest, and the known oracle-based algorithms have a worse rate. We ask whether the optimal rate can be attained by a learner that accesses the classes only through an empirical risk minimization (ERM) oracle. We give an algorithm that uses $\tilde{O}((d_G+d_H)/\varepsilon^2)$ samples and polynomially many calls to a weighted ERM oracle over pairs of a group and a hypothesis, where $d_G$ and $d_H$ are the VC dimensions of the group and hypothesis classes and $\varepsilon$ bounds the excess risk on every group, weighted by its probability. We also show a matching lower bound of $Ω((d_G+d_H)/\varepsilon^2)$ samples for every learner even with unlimited computation, so oracle efficiency costs no additional samples. We also consider a natural special case in which every input belongs to at most $k$ groups and these groups can be listed. In this case, an ERM oracle for the hypothesis class alone suffices, with $\tilde{O}((d_H+\log k)/\varepsilon^2)$ samples in the agnostic setting and $\tilde{O}((d_H+\log k)/\varepsilon)$ samples when every group has a hypothesis that is correct on it.
Primary: Harvard University
All Institutions: Harvard University
The paper resolves a major open problem in multi-group learning by demonstrating that optimal sample complexity can be achieved with polynomially many calls to a weighted ERM oracle, matching the theoretical lower bound and eliminating the previous statistical penalty for computational efficiency.
The paper addresses a fundamental gap in multi-group learning: achieving optimal sample complexity ($\tilde{O}((d_G+d_H)/\varepsilon^2)$) while remaining computationally efficient via an ERM oracle. Previous optimal algorithms required tracking weights for every group-hypothesis pair, which is infeasible for large classes, while oracle-based algorithms suffered from worse sample rates (typically $\varepsilon^{-3}$). The authors propose a novel reduction of the batch multi-group problem to a hybrid online learning setting. They introduce a "pair class" of functions that encode the joint output of a group and a hypothesis, allowing the use of Gaussian Follow-the-Perturbed-Leader (GFTPL) with a weighted ERM oracle. A key technical contribution is the "voting rule" that allows the learner to predict using the pair player's distribution without explicitly solving the joint optimization problem at each step. The paper also provides a rigorous lower bound showing that this sample complexity is optimal for any learner, even with unlimited computation, proving that oracle efficiency comes at no statistical cost. Additionally, they identify a "bounded membership" condition under which a simpler ERM oracle over the hypothesis class alone suffices, yielding improved rates in the realizable case.
This is a pure theory paper. There are no empirical experiments, datasets, or benchmarks. The "evaluation" consists of rigorous mathematical proofs of upper and lower bounds on sample complexity and oracle calls. The results are validated through reduction to classical agnostic learning lower bounds and analysis of the online-to-batch conversion.
The paper provides full proofs in the appendix (implied by the structure, though the provided text is truncated). The algorithms are described in pseudocode (Algorithm 1, 2, 3). Since the contribution is theoretical, "reproducibility" refers to the verifiability of the proofs. The logical flow is clear, and the reliance on standard tools (VC dimension, Assouad's lemma, online learning regret bounds) makes the results accessible to experts in learning theory.
The main limitation is the reliance on an ERM oracle over the *pair* class $(g,h)$. The paper proves that this joint oracle cannot generally be implemented from separate oracles for $G$ and $H$ without solving an NP-hard problem (Max-Cut reduction). While they provide a workaround for the "bounded membership" case, the general case still requires an oracle that can jointly optimize over groups and hypotheses, which may be computationally expensive or unavailable in practice for complex group structures. The deterministic predictor result requires a computable total order on the input space, which is a strong assumption not always met.
This work significantly advances the theory of fair and robust learning. Multi-group learning is a core component of subgroup fairness and robustness guarantees. By closing the gap between statistical optimality and computational efficiency, this paper provides a blueprint for designing practical algorithms that offer strong fairness guarantees without prohibitive computational costs. It bridges the gap between online learning theory and batch learning guarantees, offering new techniques (hybrid online-to-batch conversion with oracles) that could be applied to other structured prediction problems. The paper resolves a major open problem in multi-group learning by demonstrating that optimal sample complexity can be achieved with polynomially many calls to a weighted ERM oracle, matching the theoretical lower bound and eliminating the previous statistical penalty for computational efficiency.
Reinforcement learning (RL) for large language model (LLM) agents remains challenging in long-horizon tasks with sparse outcome rewards, where assigning credit to individual actions is difficult. Methods such as GRPO assign the same trajectory-level advantage to all actions, while critic-based methods require learning an additional value function that can be costly and unstable. We introduce LLM Value Function RL (LVRL), which uses the pretrained knowledge and in-context learning ability of an LLM to estimate policy-aware state values without finetuning a separate critic. LVRL conditions the value model on group context and calibrates its predictions using known values at the initial and terminal states, enabling step-level credit assignment from estimated value differences. We further extend this idea to learning from an expert policy through on-policy distillation (OPD), introducing LLM Value Function OPD (LVOPD), which focuses teacher supervision on actions associated with decreases in estimated value. We evaluate our approach across diverse agentic benchmarks: AppWorld, BabyAI, and SWE-bench Verified. Across policy scales (4B to 32B), LVRL consistently and substantially outperforms both GRPO and PPO. On AppWorld and SWE-bench Verified, LVRL yields absolute improvements of up to 13.6 and 8.6 percentage points over GRPO. Notably, on tasks requiring extended exploration like BabyAI, LVRL bridges the gap from total failure to 75.0% success. Furthermore, LVOPD yields consistent improvements over standard distillation baselines on AppWorld. Finally, Monte-Carlo analysis confirms that our frozen critic accurately identifies semantic pitfalls and recoveries, proving that LVRL extracts robust, policy-aware step-level credit from sparse terminal outcomes.
Primary: Apple
All Institutions: Apple
The paper introduces LVRL, a method using LLMs as value functions for credit assignment in agentic RL, achieving substantial improvements over standard RL baselines. The technical contribution is significant, offering a novel and effective way to leverage pretrained LLM capabilities for reinforcement learning, with strong empirical validation across diverse benchmarks and model scales.
The paper proposes LVRL, a method that leverages the in-context learning capabilities of a pretrained LLM to estimate state values for credit assignment in agentic RL. By conditioning the value model on group context and calibrating predictions using known boundary values (initial and terminal states), the method avoids the instability and cost of training a separate critic network. The extension to on-policy distillation (LVOPD) is a logical and effective application of this value estimation to focus teacher supervision on critical decision points. The methodology is sound, addressing a core bottleneck in LLM agent training (sparse rewards) with a resource-efficient approach that utilizes existing model capabilities.
The evaluation is comprehensive, covering diverse agentic benchmarks (AppWorld, BabyAI, SWE-bench Verified) and multiple model scales (4B to 32B). The reported improvements over strong baselines like GRPO and PPO are substantial, particularly the transition from total failure to 75% success on BabyAI, which demonstrates the method's effectiveness in long-horizon exploration tasks. The Monte-Carlo analysis providing qualitative evidence of semantic understanding in the value estimates adds depth to the quantitative results.
The paper appears to be a preprint (arXiv) with a submission to ICLR 2027. While the methodology is clearly described, the lack of a provided code repository or project URL in the extracted text limits immediate reproducibility. However, the reliance on standard LLM APIs and RL frameworks suggests that implementation is feasible for practitioners with access to similar models.
The primary limitation is the computational cost of using a large LLM as a value estimator, which may be prohibitive for very large-scale deployments or real-time applications. Additionally, the performance gains are dependent on the quality of the base LLM's in-context learning abilities, which may vary across different model families. The paper does not extensively discuss the failure modes of the value estimation process when the LLM's prior knowledge conflicts with the specific task dynamics.
This work has significant implications for the field of LLM agents, offering a scalable and effective solution to the credit assignment problem. By demonstrating that pretrained LLMs can serve as effective value estimators without fine-tuning, it opens new avenues for integrating RL with LLMs more efficiently. The approach could be widely adopted in developing autonomous agents for software engineering, web navigation, and other complex interactive tasks. The paper introduces LVRL, a method using LLMs as value functions for credit assignment in agentic RL, achieving substantial improvements over standard RL baselines. The technical contribution is significant, offering a novel and effective way to leverage pretrained LLM capabilities for reinforcement learning, with strong empirical validation across diverse benchmarks and model scales.
Generating an image with a diffusion or flow-matching model typically involves integrating a velocity field for hundreds of sequential steps. That computation suits a GPU poorly and an analog neuromorphic substrate well, where one fast, low-power physical relaxation could replace the whole sequence. Two things are needed: a generative algorithm whose computation is a relaxation, and a way to train it without backpropagation, whose global gradient transport is hard to realise in analog. Energy Matching supplies the first, sampling by gradient-descent relaxation on a time-independent potential - but obtaining the gradient that descent follows is itself a backward pass, once per integration step. We prove that a predictive coding network with a single linear injection term added to its energy relaxes to a state whose visible layer carries that gradient inherently, with an error that vanishes for small injection strengths. An Energy Matching model trained by backpropagation can therefore be loaded into this network unchanged and sampled by relaxation alone, matching the original model's FID to within 0.04 at CIFAR-10 and ImageNet-32 scale. The network can also be trained without backpropagation: Equilibrium Propagation reproduces backpropagation's parameter updates to a cosine similarity of 0.9998, and training end to end with those updates reaches an FID of 3.64 on unconditional CIFAR-10 against 3.54 for backpropagation. To our knowledge this is the first CIFAR-10 generative model trained with Equilibrium Propagation, and the first at competitive quality by any backpropagation-free rule. This quality does not require group normalisation or attention: an architecture of only convolutions and pointwise nonlinearities, both native to an analog crossbar, reaches an FID of 3.51 when trained by backpropagation.
Primary: University of Cambridge
All Institutions: University of Cambridge, Nokia Bell Labs
The paper presents a rigorous method for training CIFAR-scale generative models using Equilibrium Propagation, achieving competitive FID scores without backpropagation. It bridges the gap between energy-based generative models and neuromorphic hardware by proving that predictive coding networks can inherently compute the gradients required for sampling and training via local relaxation dynamics.
The paper proposes a rigorous theoretical and practical framework for training generative models without backpropagation, specifically targeting analog neuromorphic hardware. The core contribution is the "PotentialPCN," a predictive coding network (PCN) constructed from a standard feedforward energy matching model. The authors prove that by adding a single linear injection term to the energy of the PCN, the relaxed state of the network inherently computes the gradient of the potential (the velocity field required for sampling) with an error that vanishes as the injection strength approaches zero. This allows for sampling via physical relaxation without a backward pass. Furthermore, they demonstrate that Equilibrium Propagation (specifically a GradEP variant with a linearized nudge) can train these models end-to-end, reproducing backpropagation gradients with high fidelity (cosine similarity 0.9998). The methodology is mathematically sound, with clear proofs provided in the appendix regarding the correspondence between the PCN relaxation and the feedforward gradient.
The experiments are comprehensive for the scope of the claim. The authors validate the inference correspondence by showing that the PCN relaxation matches the feedforward gradient to machine precision (TF32 limits) across six decades of injection strengths. They demonstrate that an Energy Matching model trained with backprop can be sampled via PCN relaxation with negligible FID loss (within 0.04) on CIFAR-10 and ImageNet-32. Crucially, they show that training *with* Equilibrium Propagation yields competitive results: an FID of 3.64 on unconditional CIFAR-10, compared to 3.54 for backpropagation. They also provide an existence proof for hardware-plausible architectures (convolutions and pointwise nonlinearities only) achieving an FID of 3.51, demonstrating that attention and group normalization are not strictly necessary for competitive performance at this scale.
The paper provides a high level of reproducibility. The authors release code for training, sampling, and correspondence measurements. The training recipe, including hyperparameters for the Equilibrium Propagation phases (nudge strength, tether stiffness, sweep budgets), is detailed in the appendix. The distinction between the "implicit solve" (IFT) and "Equilibrium Propagation" (EP) arms is clearly defined, allowing readers to replicate the specific backprop-free training procedure.
The primary limitation is that all results are simulated on GPUs; no physical neuromorphic hardware was used, so the actual energy and latency benefits remain theoretical. The simulation cost of EP is significantly higher than backprop (16x), which is a major barrier for GPU-based research, though the paper argues this is mitigated on physical hardware. The scope is limited to CIFAR-10 for training; ImageNet-32 was only used for inference validation. Additionally, the "hardware-plausible" architecture was only trained with backprop, not with EP, leaving a gap in demonstrating the full pipeline on the simplified architecture.
This work is significant for the neuromorphic computing community as it provides a concrete, high-quality pathway to implement generative models on analog substrates. By removing the need for backpropagation during both training and inference (sampling), it addresses the two main hurdles for analog AI: global gradient transport and sequential computation. The finding that simple convolutional architectures suffice for competitive FID scores without attention is also valuable for the broader generative modeling field, suggesting that complexity may be overestimated for certain tasks. The paper presents a rigorous method for training CIFAR-scale generative models using Equilibrium Propagation, achieving competitive FID scores without backpropagation. It bridges the gap between energy-based generative models and neuromorphic hardware by proving that predictive coding networks can inherently compute the gradients required for sampling and training via local relaxation dynamics.
Advances in large language models (LLMs) have brought the vision of a general-purpose personal assistant closer to reality. Such assistants could provide a unified natural-language interface to a broad ecosystem of applications. However, real-world applications do not readily support training at scale because access is often restricted and their state is difficult to reset. To address this challenge, we introduce Terrarium, a framework that synthesizes executable App Worlds from real-world application information, generates training task packages with evaluator-side rubrics from these worlds, and iteratively validates and refines both through agent rollouts. For App World synthesis, Terrarium organizes public Web information and in-application observations into structured app specifications that guide the construction of self-contained runtimes with callable operations and mutable, resettable state. For task synthesis, Terrarium constructs tasks grounded in App World state together with evaluator-side outcome rubrics that assess success without prescribing reference trajectories. We construct a training suite of 275 App Worlds and 13,344 tasks for outcome-based reinforcement learning across three Qwen3 model scales (4B, 8B, and 14B parameters). Across these scales, models trained with Terrarium outperform the corresponding Qwen3 models by an average of 18.34 percentage points on the $τ^2$-bench and 10.94 percentage points on the AppWorld bench. These results suggest that app-grounded world and task synthesis offers a scalable path toward general-purpose assistants capable of operating across broad application ecosystems.
Primary: Fudan University
All Institutions: Beijing Tongming Lake Information Technology Application Innovation Center, Fudan University, Institute of Systems for Advanced Computing, Peking University, School of Computer Science, Simon Fraser University
[One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a scalable framework for synthesizing executable app worlds and grounded training tasks to improve LLM agent performance, demonstrating significant gains on standard benchmarks through outcome-based reinforcement learning.
The paper introduces "Terrarium," a framework for synthesizing executable "App Worlds" from real-world application information (web search + GUI traces) to train LLM agents. The core methodological contribution is the creation of a structured intermediate representation (app specification) that guides the construction of self-contained, resettable runtimes with mutable state. This addresses the key bottleneck in agent training: the lack of scalable, realistic, and controllable environments. The pipeline involves synthesizing the world, generating grounded tasks with evaluator-side rubrics (rather than reference trajectories), and using a closed-loop validation process where agent rollouts identify and repair defects in the world, task, or evaluator. The use of Group Relative Policy Optimization (GRPO) with binary outcome rewards is a solid choice for this setting, allowing the model to learn from success/failure signals without requiring dense trajectory supervision. The distinction between "Intent Execution" and "Intent Completion" (handling missing information) is a thoughtful addition that adds nuance to the training signal.
The experimental evaluation is rigorous and comprehensive. The authors train Qwen3 models at three scales (4B, 8B, 14B) and evaluate them on two independent benchmarks: $\tau^2$-bench and AppWorld. The results show consistent and significant improvements over the base models (18.34 points on $\tau^2$-bench, 10.94 on AppWorld). The ablation studies are particularly strong, demonstrating that GRPO outperforms SFT, that combining Execution and Completion supervision is beneficial, and that performance scales with the number of synthesized App Worlds. The comparison with other synthesis methods (AWM, EnvScaler) further validates the effectiveness of the app-grounded approach. The inclusion of BFCL Multi-Turn evaluation provides additional evidence of generalization.
The paper provides extensive details on the training configuration, including hyperparameters, dataset statistics, and the judge prompt. The authors state they will open-source the framework and artifacts, which is a major plus for reproducibility. The detailed description of the closed-loop validation process and the failure taxonomy helps in understanding how the data quality is maintained. However, the actual code and data are not yet available, which is a minor limitation for immediate reproducibility.
The primary limitation is the reliance on a large, proprietary model (Qwen3.5-397B-A17B) as the evaluator-side judge. This introduces a potential bias and makes the method less accessible to smaller labs. Additionally, the synthesized worlds are surrogates and may not perfectly capture the complexity and edge cases of real-world applications. The method is also computationally expensive, requiring significant GPU resources for both synthesis and training.
This work has significant potential to impact the field of LLM agents by providing a scalable path to training general-purpose assistants. The ability to synthesize realistic, controllable environments from public information could accelerate research in agent learning and enable the development of more capable and reliable AI systems. The open-sourcing of the framework and data will likely foster further innovation in this area. [One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a scalable framework for synthesizing executable app worlds and grounded training tasks to improve LLM agent performance, demonstrating significant gains on standard benchmarks through outcome-based reinforcement learning.
City-scale 3D grounding is commonly formulated as candidate ranking, which requires precomputed instance proposals and becomes inefficient in large outdoor scenes. We propose Urbex, an agentic vision-language framework that instead casts grounding as active spatial search. Urbex represents a city-scale 3D scene as an interactive multi-view environment, allowing a VLM agent to query landmarks, zoom into local regions, render oblique views, and commit a grounded bounding box. We optimize this tool-use policy with reinforcement learning on CityRefer, using smooth localization rewards and lightweight shaping rewards for efficient exploration. Experiments show that Urbex achieves the best Hit@1 and IoU on CityRefer, supports candidate-free grounding without ground-truth boxes, and is substantially faster than the strongest city-scale ranking baseline. Zero-shot results on STPLS3D-Refer further indicate improved cross-scene robustness.
Primary: Shanghai AI Laboratory
All Institutions: Shanghai AI Laboratory, Tsinghua University
The paper introduces Urbex, an agentic framework that reformulates city-scale 3D grounding as active spatial search using a VLM agent and reinforcement learning. It demonstrates superior accuracy and efficiency compared to candidate-ranking baselines, offering a scalable solution for large outdoor 3D scene understanding.
The paper proposes "Urbex," a framework that reframes city-scale 3D grounding from a static candidate ranking problem to an active spatial search problem. The core novelty lies in the design of an interactive multi-view environment where a Vision-Language Model (VLM) agent uses specific tools (Landmark Query, Zoom-in, Oblique Rendering) to navigate the scene. The use of Reinforcement Learning (specifically GRPO) to optimize this tool-use policy is a strong methodological contribution, addressing the challenge of credit assignment in multi-step agentic tasks. The reward design, combining terminal IoU with distance-based shaping and tool-level rewards, is well-motivated for sparse reward environments. The approach effectively leverages the reasoning capabilities of modern VLMs while constraining the action space to manageable geometric operations.
The experiments are rigorous and comprehensive. The authors evaluate on CityRefer (in-domain) and STPLS3D-Refer (zero-shot cross-scene), covering both "With GT" (candidate ranking) and "Without GT" (candidate-free) settings. The results demonstrate significant improvements in Hit@1 and IoU over strong baselines like CityAnchor and GPT-5.4/Qwen3.6-plus prompted baselines. The efficiency analysis is particularly compelling, showing a 10x speedup over the strongest ranking baseline by avoiding exhaustive scoring. Ablation studies clearly isolate the contributions of individual tools and reward components, providing strong evidence for the design choices. The zero-shot generalization results are a key strength, suggesting the learned policy captures reusable spatial search strategies rather than just memorizing the training distribution.
The paper provides detailed descriptions of the environment, tools, and RL objective. However, as an arXiv preprint, specific code availability is not confirmed in the text. The reliance on specific VLM backbones (Qwen, GPT) and the complexity of the RL training loop (GRPO) may pose barriers to reproduction for groups without significant compute resources. The hyperparameters for the RL training and the specific implementation of the view-to-world transforms are described but would benefit from open-source code for full verification.
The method relies on the availability of named landmarks or distinct visual features for the "Landmark Query" tool, which may limit its applicability to scenes with poor semantic labeling or highly repetitive structures. The RL training process is computationally expensive compared to supervised fine-tuning. The "Oblique Rendering" tool requires pre-computed or on-the-fly rendering, which adds latency and storage overhead, although the paper addresses this with caching strategies. The performance on "Without GT" settings, while better than baselines, still shows room for improvement in precise box localization (high area ratio variance).
This work has significant implications for embodied AI and autonomous driving, where efficient and accurate localization of objects in large-scale environments is critical. By demonstrating that agentic search can outperform and outpace exhaustive ranking, it offers a scalable paradigm for handling large 3D scenes. The framework could be extended to other tasks requiring long-range spatial reasoning, such as robot navigation or aerial inspection. The integration of RL with VLM tool-use is a trend that this paper advances meaningfully in the 3D domain. The paper introduces Urbex, an agentic framework that reformulates city-scale 3D grounding as active spatial search using a VLM agent and reinforcement learning. It demonstrates superior accuracy and efficiency compared to candidate-ranking baselines, offering a scalable solution for large outdoor 3D scene understanding.
Robotic platforms combine multi-modal sensors to operate across diverse conditions, but most dense correspondence methods are limited to RGB-RGB or a small set of modality pairs. Training a model for each modality pair scales poorly with the number of sensors and forgoes potential positive transfer across modality pairs. We introduce TartanMatch, a unified model for dense correspondence across all 25 ordered pairs among RGB, event, thermal, depth, and projected LiDAR. TartanMatch adapts an RGB-pretrained correspondence model to all modality pairs within one shared network using lightweight modality adaptation and a mixture of real and synthetic data. On zero-shot real-world benchmarks, TartanMatch reduces Endpoint Error (EPE) over the strongest baselines by 61.9% for cross-modal matching and 49.6% for same-modal matching. We further show that joint training across all modality pairs improves average matching accuracy over pair-specific training, especially on challenging cross-modal pairs. These results demonstrate that a single model achieves broad multi-modal coverage and state-of-the-art matching accuracy.
Primary: Carnegie Mellon University
All Institutions: Carnegie Mellon University, Fisk University
[One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. TartanMatch introduces a unified dense correspondence model that significantly outperforms pair-specific baselines across 25 modality pairs by leveraging RGB pre-training and joint multi-modal training, demonstrating that a single network can effectively handle diverse sensor combinations with superior accuracy and scalability.
The paper proposes TartanMatch, a unified framework for dense correspondence across five distinct sensing modalities: RGB, event, thermal, depth, and projected LiDAR. The core methodological contribution is the adaptation of an RGB-pretrained correspondence model (specifically UFM) to handle all 25 ordered modality pairs within a single shared network. This is achieved through lightweight modality adaptation modules and a training strategy that leverages a mixture of real and synthetic data. The approach addresses the scalability issue of training separate models for each modality pair, arguing that joint training facilitates positive transfer, particularly for challenging cross-modal pairs. The use of a pre-trained backbone for RGB and adapting it to other modalities is a sound architectural choice that leverages the rich features learned from large-scale RGB datasets.
The experimental evaluation is extensive, covering zero-shot real-world benchmarks. The paper reports significant improvements in Endpoint Error (EPE), claiming a 61.9% reduction for cross-modal matching and 49.6% for same-modal matching compared to the strongest baselines. The radar chart visualization effectively demonstrates the model's superiority across various modality pairs. The claim that joint training improves average matching accuracy over pair-specific training is a key empirical finding that supports the unified model hypothesis. The inclusion of both real and synthetic data in the training mix is well-justified by the results, showing robustness to domain shifts.
The paper provides a project page (https://tartanmatch.github.io/) which likely contains code and additional resources. The use of standard benchmarks and clear metrics (EPE) enhances reproducibility. However, the specific details of the "lightweight modality adaptation" modules and the exact composition of the synthetic data mixture would need to be verified in the supplementary material or code repository for full reproducibility. The reliance on a specific pre-trained model (UFM) is a potential dependency, but UFM is a known baseline.
The primary limitation is the scope of modalities covered (5 modalities, 25 pairs). While this is a significant expansion over previous work, it does not cover all possible robotic sensing modalities (e.g., radar, acoustic). The performance on zero-shot real-world data is strong, but the gap between synthetic/real training data and real-world inference might still exist for unseen environments. The computational cost of training a single model for 25 pairs versus 25 separate models is not explicitly detailed in the abstract, though the "lightweight" adaptation suggests efficiency.
This work has high potential impact in the robotics and autonomous systems fields. A unified correspondence model simplifies the perception pipeline for multi-modal robots, reducing the need for maintaining and updating multiple specialized models. The demonstrated positive transfer across modalities could inspire similar unified approaches in other multi-sensor fusion tasks. The open-source nature of the project (implied by the project page) will likely facilitate adoption by the research community. [One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. TartanMatch introduces a unified dense correspondence model that significantly outperforms pair-specific baselines across 25 modality pairs by leveraging RGB pre-training and joint multi-modal training, demonstrating that a single network can effectively handle diverse sensor combinations with superior accuracy and scalability.
Multimodal code intelligence connects the symbolic world of programs with the visual world their execution creates. Interest in this capability is burgeoning, as models are increasingly asked to produce charts, webpages, animations, and semi-structured documents. Recently, proprietary models have pushed far ahead, producing interactive artifacts even from a text description or a screenshot alone. Yet their data and training remain closed, and open progress is held back by the quantity and quality of available data and by the pipelines and methods for producing it. Quality is the harder of the two, since a sample must be correct as a program, well built, and above all visually right once rendered. No existing signal covers all three. Execution often returns just a pass or fail, code similarity does not map to the visual output, and task-specific verifiers do not generalize. To push open progress further, we start from the data side with PrismaForge, a hierarchical data pipeline centered on a synthesized rubric that decomposes these factors into checkable criteria. The same rubric serves as both filter and reward: it curates the supervised corpus, then drives reinforcement learning on visual-coding tasks. Leveraging PrismaForge for data synthesis and filtering, we build PrismaCoder-2M, the most diverse open dataset for multimodal code intelligence and the largest to date. On the model side, we train PrismaCoder-9B and PrismaCoder-35B on this data and release both. They lead the open state of the art by a wide margin and rival leading proprietary systems. Our analyses further show that the proposed rubric-based pipeline improves multimodal coding performance across model backbones. We release our code and data at https://qiushisun.github.io/PrismaCoder-Home/.
Primary: University of California, San Diego (UCSD)
All Institutions: University of California, San Diego, Meta AI
The paper introduces PrismaForge, a data pipeline using decomposed rubric synthesis to generate high-quality multimodal code data, resulting in PrismaCoder models that rival proprietary systems. This is a significant contribution to the field of multimodal code intelligence, addressing the critical need for verifiable, high-quality open data and demonstrating that data-centric approaches can close the gap with closed-source models.
The paper proposes PrismaForge, a hierarchical data pipeline that addresses the lack of comprehensive quality signals in multimodal code generation (combining program correctness, code structure, and visual fidelity). The core innovation is the "decomposed rubric synthesis," where a synthesized rubric breaks down complex visual-coding tasks into checkable criteria. This rubric serves a dual purpose: it acts as a filter to curate a high-quality supervised corpus (PrismaCoder-2M) and as a reward signal for reinforcement learning. The method effectively bridges the gap between symbolic code and visual output by creating a verifiable intermediate representation. The approach is sound and addresses a critical bottleneck in open-source multimodal code models: the scarcity of high-quality, verifiable data.
The authors train PrismaCoder-9B and PrismaCoder-35B on the synthesized data. The results claim to lead the open state of the art by a wide margin and rival proprietary systems. The evaluation likely includes standard benchmarks for code generation (e.g., HumanEval, MBPP) and multimodal-specific tasks (e.g., rendering accuracy, visual similarity). The claim of "rivaling leading proprietary systems" is significant and, if substantiated by the full text (which is summarized here), would be a major empirical finding. The analysis section reportedly shows the pipeline improves performance across different model backbones, suggesting the data quality is the key driver rather than a specific architectural quirk.
The paper releases both the code and the data (PrismaCoder-2M), which is a significant contribution to the open-source community. The release of a 2M sample dataset for multimodal code intelligence is substantial and will likely be widely used. The detailed release of the rubric synthesis pipeline further enhances reproducibility.
The reliance on synthesized rubrics may introduce biases if the rubric generation model has blind spots. The computational cost of generating and filtering 2M samples with complex visual verification is likely high. Additionally, the generalization to tasks outside the training distribution (e.g., highly abstract UIs or non-standard visualizations) may be limited.
This work significantly advances the open-source ecosystem for multimodal code intelligence. By providing a large, high-quality dataset and a robust training pipeline, it lowers the barrier for other researchers and developers to build competitive multimodal code models. It also highlights the importance of data quality and verification pipelines in the era of large language models. The paper introduces PrismaForge, a data pipeline using decomposed rubric synthesis to generate high-quality multimodal code data, resulting in PrismaCoder models that rival proprietary systems. This is a significant contribution to the field of multimodal code intelligence, addressing the critical need for verifiable, high-quality open data and demonstrating that data-centric approaches can close the gap with closed-source models.
Long-horizon agents have drawn growing attention for their ability to work autonomously -- from hours to days -- on economically valuable tasks, a capability essential to automated discovery and recursive self-improvement (RSI). Yet open-source models still struggle on long-horizon tasks, largely for lack of both training data and a working recipe, and scaling both is hard: long, complex tasks are difficult to curate and verify, and their rewards are sparse. We address both. First, we introduce Tev-Gym, the first long-horizon agent training dataset with verifiable, dense, progressive rewards -- 1,170 verified task environments across 7 domains. It is built on a latent-world-first approach that first constructs a latent world: a complete and correct hidden specification of the task domain, workflow, and expected state after each step. Each task is then progressively rendered from this specification into an agent-facing environment, enabling controllable complexity and a natural progress metric. Second, we turn this metric into a training recipe: via progress-potential reward shaping (PPRS), we convert progress into a dense per-step reward that improves training efficiency and makes otherwise-intractable tasks trainable. Training with this recipe yields Tev-27B, which achieves substantial gains on the Agents' Last Exam computing and math sciences domains (ALE-CM), matching GPT-6 Astra and Claude Opus 5 at Pass@5. Despite training only on ALE-related data, it generalizes to the Terminal-Bench (TB) 3.0 and 4.0 subsets, surpassing GLM 5.2 and Gemini 3.7 Flash, and reaches Claude Opus 4.8-level performance on middle-horizon tasks such as TB 2.1.
Primary: University of California
All Institutions: University of California
The paper introduces a latent-world-first synthesis pipeline and Progress-Potential Reward Shaping (PPRS) to enable effective RL training for long-horizon agents, resulting in open-source models that match frontier closed-source systems on complex agentic benchmarks. This is a significant contribution that addresses the critical data and reward sparsity bottlenecks in agentic AI, offering a reproducible and scalable recipe for generating high-quality training environments and training signals.
The paper introduces a "latent-world-first" synthesis pipeline for creating long-horizon agent environments. This is a significant methodological contribution because it addresses the core bottleneck in agentic RL: the lack of verifiable, dense rewards for complex, multi-step tasks. By constructing a complete, correct hidden specification (the latent world) first and then rendering the task and corruption from it, the authors ensure that ground truth is known by construction. This allows for the derivation of a "progress potential" function, which is used in a novel reward shaping scheme called Progress-Potential Reward Shaping (PPRS). PPRS is theoretically grounded in potential-based reward shaping but adapted for GRPO to prevent reward starvation in groups where no rollout succeeds. The methodology is rigorous, with clear stages for world construction, workflow graphing, artifact rendering, and controlled corruption. The inclusion of qualification gates (exploit resistance, progress validity, role independence) demonstrates a high level of care in preventing reward hacking and ensuring data quality.
The experiments are extensive and well-designed. The authors train two model sizes (9B and 27B) and evaluate them on the Agents' Last Exam (ALE-CM) and Terminal-Bench (TB) 3.0/4.0. The results show substantial gains, with the 27B model matching frontier closed-source models (GPT-6 Astra, Claude Opus 5) on Pass@5 for ALE-CM. Crucially, the paper demonstrates generalization to unseen benchmarks (TB), which is a strong indicator of learning robust agentic behaviors rather than overfitting to the training distribution. The ablation studies likely confirm the importance of both the dense reward and the specific data synthesis method. The comparison against prior open-source datasets (SWE-Smith, etc.) highlights the complexity gap, with Tev-Gym tasks being significantly longer and harder.
The paper provides detailed descriptions of the synthesis pipeline, the reward shaping formula, and the training setup (DPPO, group size, token budgets). The "latent world" concept is well-defined, and the qualification gates are described formally. However, without access to the code or the specific generators for the 1,170 environments, full reproduction is difficult. The reliance on specific base models (Qwen3.5/3.8) and the proprietary nature of some benchmark subsets (if any) could limit reproducibility for external labs. The paper is a preprint, so code release status is unknown, but the methodological detail is high.
The primary limitation is the scale of the dataset (1,170 environments) compared to massive web-scale datasets, though the quality is higher. The approach is currently focused on specific domains (software, data, math, etc.) and may not generalize to all types of long-horizon tasks (e.g., creative writing, open-ended research). The computational cost of generating and verifying these latent worlds is likely high. Additionally, the performance gains are relative to open-source baselines; while matching closed-source models is impressive, the absolute performance on the hardest tasks may still be limited by the base model's capabilities.
This work has high potential impact on the field of agentic AI. By providing a recipe for generating high-quality, verifiable long-horizon training data, it lowers the barrier for other labs to train competitive open-source agents. The PPRS technique is a generalizable insight for RL in sparse-reward settings. If the dataset and code are released, it could become a standard benchmark and training resource for the community, similar to how SWE-bench or Terminal-Bench have been used. It pushes the frontier of what open-source models can achieve in autonomous, long-duration tasks. The paper introduces a latent-world-first synthesis pipeline and Progress-Potential Reward Shaping (PPRS) to enable effective RL training for long-horizon agents, resulting in open-source models that match frontier closed-source systems on complex agentic benchmarks. This is a significant contribution that addresses the critical data and reward sparsity bottlenecks in agentic AI, offering a reproducible and scalable recipe for generating high-quality training environments and training signals.
Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fundamental bottleneck: the cost of evaluating progress. In real settings, a single true evaluation, such as a full benchmark or serving trace, can take hours, so a search can afford only a handful of true evaluations. We present DuoDiscover, a search system that jointly discovers what to optimize and how to evaluate progress efficiently, through three mechanisms. First, each worker in a fleet of coding agents co-discovers an optimization target and a fast, tailored proxy evaluator, which we call a yardstick. The agent then hill-climbs the yardstick rather than the true evaluation. Second, since search can exploit gaps between a yardstick and the true objective, fidelity tests continually recheck each yardstick's ranking and invalidate any that drift. Third, the fleet is bottom-up, with no central decomposition of the problem. A lightweight orchestrator spawns and terminates agents, briefs running ones, and seeds new agents with others' confirmed work. DuoDiscover enables autonomous optimization from software to chip architecture. We demonstrate DuoDiscover on two repository-level benchmarks and four GPU and TPU kernel suites covering hundreds of kernels. On SWE-fficiency, it doubles the average speedup of human experts. On SOL-Exec, it ranks first on 121 of 235 kernels and in the top three on 212. On the heavily optimized vLLM and SQLite engines, DuoDiscover achieves 1.92$\times$ and 1.38$\times$ gains over the base code. On a CPU microarchitecture optimization task with evaluations of up to 28 hours, DuoDiscover surpasses both AlphaEvolve-based ArchAgent v2 and the best human design, with 1.65$\times$ ArchAgent v2's gain in 16.7 days versus more than 40 days, advancing the frontier of autonomous chip design.
Primary: Google
All Institutions: Google
DuoDiscover introduces a co-discovery mechanism for optimization targets and proxy evaluators to overcome the evaluation bottleneck in autonomous search. The paper demonstrates significant improvements in code and hardware optimization tasks, establishing a new paradigm for scaling LLM-based discovery in high-cost domains.
The paper introduces DuoDiscover, a search system that addresses the "evaluation bottleneck" in autonomous code optimization. The core novelty is the "co-discovery" of optimization targets and cheap proxy evaluators ("yardsticks") by LLM agents. Unlike prior work (e.g., AlphaEvolve) that treats the evaluator as fixed or user-defined, DuoDiscover allows agents to construct their own fast metrics to guide hill-climbing, with a rigorous "fidelity test" (Kendall's tau rank correlation) to invalidate proxies that drift from the true objective. The bottom-up fleet architecture, where a lightweight orchestrator manages expensive true evaluations while agents exploit cheap proxies, is a well-motivated and technically sound approach to scaling search in high-cost domains.
The experimental evaluation is extensive and covers a wide range of domains, from repository-level Python optimization (SWE-fficiency, FormulaCode) to GPU/TPU kernel optimization (SOL-Exec, JAXBench) and CPU microarchitecture design (DPC4). The results are strong, with DuoDiscover outperforming baselines like Claude Code and AlphaEvolve-based agents. Notably, the paper reports state-of-the-art results on the SOL-Exec leaderboard and surpasses human experts on SWE-fficiency. The inclusion of a 28-hour evaluation task (CPU prefetcher) demonstrates the method's ability to handle extreme evaluation costs, a key claim of the paper.
The paper provides detailed descriptions of the fidelity test, noise estimation, and search loops. However, as is common with large-scale agentic systems, the exact prompts and internal logic of the "orchestrator" LLM are not fully disclosed, which may limit exact reproduction. The use of specific, potentially proprietary, LLM backbones (Claude Opus 5, Gemini 3.8 Flash) also poses a reproducibility challenge for external researchers without access to these models.
The method relies heavily on the ability of LLMs to construct valid proxy evaluators. If an agent cannot construct a faithful yardstick, the system falls back to expensive true evaluations, negating the efficiency gains. Additionally, the upfront cost of building and validating yardsticks may be prohibitive for very short search budgets. The paper also notes that the method is less effective when the true objective is highly non-linear or when the gap between proxy and true objective is too large to bridge via hill-climbing.
This work has significant implications for the field of autonomous software engineering and hardware design. By solving the evaluation bottleneck, it enables the application of LLM-based search to real-world engineering problems where feedback is slow and expensive. This could accelerate progress in areas like chip design, database optimization, and large-scale system tuning, potentially reducing the time and cost of manual optimization efforts. DuoDiscover introduces a co-discovery mechanism for optimization targets and proxy evaluators to overcome the evaluation bottleneck in autonomous search. The paper demonstrates significant improvements in code and hardware optimization tasks, establishing a new paradigm for scaling LLM-based discovery in high-cost domains.
Language models are often evaluated alone but increasingly deployed together, routed between, made to debate, ensembled, or collaboratively trained. For a model inside a system, what matters most is not how well it performs on its own, but how much the system gains from its presence. We call this a model's compositional strength and formalize it using the Shapley value of a cooperative game over coalitions of a model pool, and use a model's leave-one-out contribution to the full coalition as a tractable proxy, requiring n+1 rather than 2^n system evaluations. Evaluating individual vs. compositional strengths across five model pools, twelve collaboration methods, and twelve datasets, we find them largely uncorrelated: in 12 of 15 (pool, method-family) settings, individual strength does not translate into compositional advantages, and the coupling is weakest among general-purpose models that practitioners are most likely to deploy. Standalone benchmarks are therefore often poor proxies for models' value in collaborative systems. We then propose compositional reinforcement learning (C-RL), which trains a policy inside a live multi-model system and rewards the outcome produced by the system as a whole, rather than the policy's correctness in isolation. The leave-one-out baseline in Shapley values cancels under a group policy gradient, so this needs no additional rollout cost beyond RL on the collaborative system's outcome and never requires running a coalition without the policy. Across three collaboration protocols and eight tasks spanning reasoning, coding, and scientific discovery, C-RL produces the strongest collaborator in 21 of 24 settings, lifting collaborative performance by +4.0 to +5.4 pts on average (~3x the gain of baselines), up to +13.1 pts.
Primary: University of Washington
All Institutions: University of Washington
The paper makes a significant contribution by formalizing compositional strength via Shapley values and demonstrating that standard individual benchmarks are poor predictors of a model's value in collaborative systems. By proposing Compositional Reinforcement Learning (C-RL), which efficiently trains models to maximize their contribution to a multi-agent system's outcome, the authors provide a novel and effective method for developing specialized collaborator models that generalize across tasks and protocols.
The paper introduces a rigorous formalization of "compositional strength" using Shapley values from cooperative game theory, adapted for LLM collaborations. The key methodological contribution is the derivation of a tractable Leave-One-Out (LOO) estimator that approximates the Shapley value with linear complexity ($n+1$) rather than exponential ($2^n$). The proposed training method, Compositional Reinforcement Learning (C-RL), is elegant in its use of group-relative policy gradients (GRPO) to cancel out the baseline cost of the LOO marginal, allowing the policy to be trained on system-level rewards without extra rollouts. The credit assignment mechanism, which attributes system-level rewards to individual policy generations within a multi-turn collaboration, is a significant technical challenge addressed effectively.
The evaluation is extensive, covering 5 model pools, 12 collaboration methods, and 12 datasets. The finding that individual strength and compositional strength are largely uncorrelated is a strong empirical contribution. The training experiments demonstrate that C-RL outperforms baselines (including standard RL and collaborative fine-tuning) in 21/24 settings. The generalization tests to unseen datasets and collaboration protocols are convincing, showing that C-RL teaches transferable collaboration skills rather than overfitting to a specific setup. The inclusion of proprietary frontier models as collaborators adds practical relevance.
The paper promises code release upon acceptance. The use of proprietary APIs (GPT-5.5, etc.) introduces a reproducibility caveat noted by the authors, as these models may change over time. However, the open-source components (Qwen, open-weight collaborators) allow for partial reproduction. The detailed description of the MoCo toolkit and specific hyperparameters aids reproducibility.
The reliance on proprietary models for some experiments limits full reproducibility. The computational cost of running multi-model collaborations for RL training is high, potentially limiting adoption to well-resourced labs. The credit assignment is uniform across turns, which the authors acknowledge as a simplification that could be improved. The study focuses on a specific set of collaboration protocols; while generalization is tested, the scope is not exhaustive of all possible multi-agent architectures.
This work shifts the focus of LLM evaluation and training from individual capability to system-level contribution. It provides a theoretical foundation for understanding multi-agent dynamics and a practical method for training better collaborators. This is highly relevant as LLMs are increasingly deployed in compound systems. The finding that standard benchmarks do not predict collaborative value is a critical insight for practitioners building multi-agent systems. The paper makes a significant contribution by formalizing compositional strength via Shapley values and demonstrating that standard individual benchmarks are poor predictors of a model's value in collaborative systems. By proposing Compositional Reinforcement Learning (C-RL), which efficiently trains models to maximize their contribution to a multi-agent system's outcome, the authors provide a novel and effective method for developing specialized collaborator models that generalize across tasks and protocols.
Frontier AI systems are increasingly natively multimodal, jointly pretrained on multiple modalities, such as text, vision, and audio. However, the scientific understanding of the multimodal data mixtures used to pretrain these systems --- how much of each modality to mix, and at what potential interference cost to the others --- is largely absent or gatekept. In this work, we derive data mixing scaling laws for three modalities: text, vision, and audio. Multimodal Large Language Models (MLLMs) increasingly adopt Mixture-of-Experts (MoE) architectures, as MoEs scale efficiently and partially limit cross-modal interference. We therefore use MoEs to conduct 268 experiments ranging from 457M to 8.3B parameters, on up to 150 billion tokens. Our work leads to multiple actionable insights into multimodal data mixing for training MLLMs. First, compute optima are modality-specific: audio modality is model-heavy, favoring scaling parameters over tokens, while text and vision require balanced scaling of model size and data. Second, repeating vision or audio data beyond 2x yields negligible benefits. Finally, cross-modal interactions are highly asymmetric: vision strongly interferes with audio, whereas audio's interference with learning text and vision is negligible. Overall, we provide a principled foundation for understanding the multimodal data mixtures needed to train frontier MLLMs.
Primary: MILA - Quebec AI Institute
All Institutions: MILA - Quebec AI Institute, McGill University, University of Washington, EPFL
The paper derives scaling laws for multimodal data mixtures in MoE architectures, revealing modality-specific compute optima and asymmetric cross-modal interference. This comprehensive empirical study provides actionable guidelines for efficient pretraining of frontier multimodal large language models, significantly advancing the understanding of how to balance text, vision, and audio data.
The paper proposes a rigorous empirical framework for deriving scaling laws specifically for multimodal data mixtures (text, vision, audio) within Mixture-of-Experts (MoE) architectures. The methodology is sound, leveraging 268 controlled experiments across a wide parameter range (457M to 8.3B) and token counts (up to 150B). The choice to focus on MoE architectures is timely and methodologically justified, as MoEs are increasingly standard in frontier models and help isolate cross-modal interference effects. The derivation of modality-specific compute optima is a significant methodological advancement over previous unimodal scaling law studies.
The experimental scale is substantial, covering three modalities and a large parameter/token space. The findings are concrete and actionable: identifying audio as "model-heavy" (favoring parameters over tokens) and text/vision as requiring balanced scaling is a valuable empirical insight. The discovery of asymmetric cross-modal interference (vision interfering with audio, but not vice versa) is a surprising and useful result for practitioners. The validation section likely confirms these trends, though the specific metrics for "interference" need to be clearly defined in the full text to ensure robustness.
The paper provides detailed experimental setup information (model sizes, token counts, modality types). However, without access to the code or specific hyperparameter configurations for the MoE routing and data mixing ratios, full reproduction is difficult. The scale of the experiments (150B tokens) makes independent reproduction costly, limiting reproducibility to groups with similar computational resources.
The study is limited to three modalities (text, vision, audio) and MoE architectures. It may not generalize directly to dense transformers or other modalities (e.g., video, 3D). The "interference" metric is a key variable; if not rigorously defined, the conclusions about asymmetry could be sensitive to evaluation choices. The paper does not address the cost of data collection or curation for these mixtures.
This work provides a principled foundation for training next-generation multimodal foundation models. By quantifying the cost-benefit of mixing modalities, it helps labs allocate computational resources more efficiently. The insights on asymmetric interference can guide curriculum learning strategies and data balancing techniques, potentially reducing the compute required to achieve specific performance levels in MLLMs. The paper derives scaling laws for multimodal data mixtures in MoE architectures, revealing modality-specific compute optima and asymmetric cross-modal interference. This comprehensive empirical study provides actionable guidelines for efficient pretraining of frontier multimodal large language models, significantly advancing the understanding of how to balance text, vision, and audio data.
We introduce REKI, a 2.4B vision-language-action model (VLA) pretrained exclusively on public data that sets a new Pareto frontier between performance and model size: on LIBERO-Plus, RoboTwin Clean2Rand and RoboCasa365, it is outperformed only by models about twice its size. REKI consists of a vision-language model (VLM) followed by a flow-matching action expert, and is pretrained on a curated mixture of public robot, egocentric human and vision-language data. Additionally, we endow it with a memory that gives the policy access to its recent past at negligible cost: a few register tokens in the action expert summarise the output of the VLM at each frame and persist in the expert's cache. This memory improves success in simulation and on real robots, and makes oscillations between poses about four times rarer. REKI uses a simple model design and training paradigm, and is optimised to be efficient at training and inference. We release the full code across data curation, training, inference and evaluation, deployment, and all pretrained and finetuned model weights: https://github.com/kyutai-labs/reki.
Primary: Kyutai
All Institutions: Kyutai
REKI introduces an efficient 2.4B VLA model with a novel register-token memory mechanism that achieves state-of-the-art performance on key robotics benchmarks while maintaining a small model size. The combination of flow-matching action experts, public-data pretraining, and a lightweight temporal memory strategy provides a robust and reproducible baseline that advances the field of vision-language-action modeling by balancing performance with computational efficiency.
The paper proposes REKI, a 2.4B parameter Vision-Language-Action (VLA) model. The architecture consists of a standard Vision-Language Model (VLM) backbone followed by a flow-matching action expert. The primary methodological contribution is the introduction of a lightweight memory mechanism using "register tokens" within the action expert. These tokens summarize the VLM's output at each frame and persist in the expert's cache, allowing the policy to access recent history with negligible computational overhead. The model is pretrained on a curated mixture of public robot, egocentric human, and vision-language data. The approach is notable for its efficiency and the specific design choice of using register tokens for temporal context rather than heavier recurrent or transformer-based memory blocks.
The evaluation is conducted on standard robotics benchmarks: LIBERO-Plus, RoboTwin Clean2Rand, and RoboCasa365. The paper claims REKI sets a new Pareto frontier, outperforming models of similar size and being outperformed only by models approximately twice its size. Specific empirical findings include a four-fold reduction in pose oscillations due to the memory mechanism. The inclusion of real-robot evaluation strengthens the claims, though the full text provided is largely a structural outline, so specific numerical comparisons must be inferred from the abstract's strong claims.
The paper explicitly states that full code for data curation, training, inference, evaluation, deployment, and all pretrained/finetuned weights are released. This high level of open-sourcing significantly enhances reproducibility and community utility.
The provided text is a preprint skeleton with section headers but lacks the detailed body text for rigorous scrutiny of specific ablation studies or failure cases. The reliance on "public data" may limit performance compared to models trained on proprietary, high-quality datasets, though the paper argues for efficiency. The "register token" memory is described as having "negligible cost," but the exact latency trade-offs in high-frequency control loops need verification in the full text.
By providing a strong, efficient baseline that is fully open-sourced, REKI lowers the barrier to entry for VLA research. The memory mechanism could be adopted by other architectures to improve temporal consistency in robotic control. The focus on public data pretraining is significant for democratizing access to capable robot policies. REKI introduces an efficient 2.4B VLA model with a novel register-token memory mechanism that achieves state-of-the-art performance on key robotics benchmarks while maintaining a small model size. The combination of flow-matching action experts, public-data pretraining, and a lightweight temporal memory strategy provides a robust and reproducible baseline that advances the field of vision-language-action modeling by balancing performance with computational efficiency.
How can robots autonomously collect the contact-rich experience needed to learn dexterous manipulation? We present DSP: Dexterity from supervised play, a framework in which a robot learns a play policy from minutes of human play through behavior cloning, then deploys it autonomously to collect hours of interaction data to train a world model. During inference, the same play policy then guides model-predictive control (MPC) by proposing candidate action sequences for evaluation under the learned world model. To accelerate inference, we also introduce a method for distilling a closed-loop, goal-conditioned policy through self-play within the learned world model. We evaluate this framework on underactuated pushing and any-to-any in-hand pose reorientation with a multifingered hand, in simulation and real world. In real world experiment, our method outperforms hindsight behavior cloning by 69.6% in success rate and achieves comparable success to reinforcement learning with 490 times less interaction data. These results demonstrate that supervised play can autonomously collect interaction data, guide model-based planning, and enable policy learning without any task-specific demonstrations or sim-to-real transfer.
Primary: Stanford University
All Institutions: Stanford University, University of Washington
The paper presents a novel framework for learning dexterous manipulation through supervised play, significantly reducing data requirements. It demonstrates a robust pipeline from human play to autonomous data collection and model-based planning, achieving strong real-world results with minimal interaction data.
The paper proposes DSP (Dexterity from Supervised Play), a framework addressing the data scarcity problem in dexterous manipulation. The core methodology involves three stages: (1) Learning a "play policy" from a small amount of human demonstration via behavior cloning; (2) Deploying this policy autonomously to collect large-scale interaction data to train a world model; and (3) Using the play policy to guide Model Predictive Control (MPC) by proposing candidate actions evaluated by the world model. Additionally, the authors introduce a distillation method to create a closed-loop, goal-conditioned policy via self-play within the learned world model. This approach effectively bridges the gap between sparse human supervision and the massive data requirements of dexterous tasks, leveraging the robot's own autonomous exploration guided by the initial play policy.
The authors evaluate the framework on two challenging tasks: underactuated pushing and any-to-any in-hand pose reorientation with a multifingered hand. Experiments are conducted in both simulation and the real world. The results are compelling, showing a 69.6% improvement in success rate over hindsight behavior cloning and achieving comparable success rates to reinforcement learning methods while using 490 times less interaction data. The inclusion of real-world experiments with a physical multifingered hand adds significant credibility to the claims regarding sim-to-real transfer and practical applicability.
The paper provides a project page and likely code availability (standard for such high-impact robotics papers from top institutions). The methodology is clearly described, with specific details on the play policy training, world model architecture, and MPC formulation. The use of standard benchmarks and tasks allows for potential replication by other groups with similar hardware.
The approach relies on an initial "play" policy, which may not generalize to entirely novel object types or environments without some form of adaptation. The computational cost of MPC at inference time, even with distillation, might be a barrier for real-time applications on lower-power hardware. Additionally, the "play" data collection phase requires the robot to be safe to operate autonomously for extended periods, which can be a practical constraint.
This work has significant implications for the field of robotics, particularly in reducing the data burden for learning complex manipulation skills. By demonstrating that supervised play can autonomously generate the data needed for world models and policy distillation, it offers a scalable path towards more autonomous robot learning. This could accelerate the deployment of dexterous robots in unstructured environments. The paper presents a novel framework for learning dexterous manipulation through supervised play, significantly reducing data requirements. It demonstrates a robust pipeline from human play to autonomous data collection and model-based planning, achieving strong real-world results with minimal interaction data.
Current humanoid dexterous manipulation remains largely confined to end-effectors and simple grasps, falling short of humans' ability to engage the entire body by bracing and sliding bulky objects against the arms and torso. We introduce Whole-Body Dexterity (WBD), a general framework that uses the robot's body and appendages as contact surfaces to grasp and manipulate objects. Our sim-to-real pipeline first generates a dataset of feasible whole-body grasps, then trains reinforcement-learning experts on a general any-to-any repositioning objective by sampling initial and goal grasps from the dataset, and distills these experts into a single policy that uses proprioception and object geometry. We show that the distilled policy transfers zero-shot to a real Unitree G1, where it repositions a basketball and a cardboard box to regions around the torso and completes a long-horizon task that stows a box under one arm, carries it from a shelf to a table, and places it. The controller operates interactively, steered by coarse motion references from a human teleoperator. Across real-world whole-body manipulation tasks, the distilled controller completes 90% of trials, compared with 18% for direct retargeting and 4% for a generalist whole-body controller.
Primary: Amazon
All Institutions: Amazon, University of California, San Diego, University of Michigan
The paper introduces a general framework for whole-body dexterous manipulation that enables humanoids to use their torso and arms as contact surfaces for grasping and repositioning objects. By combining differentiable grasp generation, large-scale reinforcement learning, and diffusion-based policy distillation, the authors achieve zero-shot sim-to-real transfer with high success rates on long-horizon tasks, significantly outperforming baseline methods and demonstrating a new capability for humanoid robots.
The paper proposes a three-stage pipeline for whole-body dexterity: (1) generating a dataset of feasible whole-body grasps using differentiable force-closure optimization and aggressive simulation filtering; (2) training per-object reinforcement learning experts (WBD-EX) on a goal-conditioned MDP to reposition objects between body regions; and (3) distilling these experts into a single deployable policy using a conditional diffusion model guided by coarse teleoperation commands. The methodology is sound, leveraging standard but effective techniques (PPO, diffusion policy) applied to a novel problem formulation (whole-body contact surfaces rather than just end-effectors). The use of a goal graph to structure the RL training and the specific reward shaping for sliding/rolling contact are well-motivated.
The evaluation is strong, featuring zero-shot sim-to-real transfer to a Unitree G1. The comparison against baselines (direct retargeting at 18% and generalist controller at 4%) shows a significant improvement (90% success rate). The long-horizon task (stowing, carrying, placing) demonstrates practical utility. However, the evaluation is limited to a single robot platform and a small set of object types (basketball, box, primitives), which limits the generalizability claims slightly.
The paper provides detailed descriptions of the grasp generation, reward functions, and training procedures. The project URL is provided. However, as an arXiv preprint, code availability is not explicitly confirmed in the text, though the project page likely hosts it. The specific hyperparameters and simulation settings are described in the appendix references, suggesting good reproducibility potential.
The method relies on a pre-trained lower-body controller for stability, which is not part of the WBD training loop, limiting the scope to standing/walking scenarios where balance is handled separately. The object set is limited to convex or simple shapes; complex concave objects might be harder to grasp with whole-body contacts. The teleoperation interface requires a human in the loop, which may not be desirable for fully autonomous applications.
This work pushes the boundary of humanoid manipulation from hand-centric to body-centric, which is a significant step toward human-like dexterity. It could enable new applications in logistics, home assistance, and industrial settings where bulky objects need to be handled. The framework is general and could be extended to other humanoid platforms and object types. The paper introduces a general framework for whole-body dexterous manipulation that enables humanoids to use their torso and arms as contact surfaces for grasping and repositioning objects. By combining differentiable grasp generation, large-scale reinforcement learning, and diffusion-based policy distillation, the authors achieve zero-shot sim-to-real transfer with high success rates on long-horizon tasks, significantly outperforming baseline methods and demonstrating a new capability for humanoid robots.
Existing navigation systems commonly rely on dedicated depth sensors or explicit depth reconstruction, while learning to generate trajectories directly from RGB images remains challenging due to the sim-to-real domain gap. In this work, we propose Stereo-Planner, an end-to-end vision-to-trajectory framework that enables zero-shot sim-to-real planning from stereo RGB observations. The key idea is to formulate trajectory generation as state- and scale-conditioned querying of latent stereo geometry. A Scene Representor incorporates a pretrained vision grounding foundation model to extract geometric cues shared across simulated and real-world environments, supporting generalization across visual domains. A lightweight Trajectory Decoder then translates these representations into trajectories through interleaved attention, separating trajectory-state interaction from scene-feature retrieval and incorporating the stereo baseline as a metric-scale cue. With these designs, Stereo-Planner is trained entirely on simulated data by minimizing differentiable trajectory costs and deployed directly in real-world environments without fine-tuning. Benchmark results demonstrate strong cross-scene generalization, with a 95% success rate in simulation environments at a commanded speed of 8 m/s, and competitive performance against depth-based planners using only stereo RGB observations for scene perception. Furthermore, deployment on a compact quadrotor equipped with an NVIDIA Orin NX enables autonomous flight at speeds of up to 7 m/s, with a mean onboard vision-to-action inference time of 24.27 ms. The software, simulation environments, and hardware designs will be open-sourced at https://github.com/Stereo-Planner/Stereo-Planner.
Primary: Nanyang Technological University
All Institutions: Nanyang Technological University, Tianjin University
Stereo-Planner enables zero-shot sim-to-real quadrotor navigation using stereo RGB by leveraging a vision grounding foundation model for geometry-aware scene representation and a scale-conditioned trajectory decoder. The paper presents a significant technical contribution by demonstrating that pretrained visual features can effectively bridge the sim-to-real domain gap for end-to-end motion planning, achieving high-speed autonomous flight without depth sensors or fine-tuning.
The paper proposes Stereo-Planner, an end-to-end framework that maps stereo RGB images directly to quadrotor trajectories. The core architectural contribution is the use of a pretrained vision grounding foundation model (Depth Anything V3 encoder) as a scene representor, which extracts geometry-aware features robust to sim-to-real appearance gaps. A lightweight trajectory decoder uses interleaved attention to query these features, conditioned on the MAV state and, crucially, the stereo baseline to resolve scale ambiguity. The training strategy employs differentiable trajectory costs (safety, smoothness, goal-reaching) and a Virtual Baseline Agmentation (VBA) technique to teach the network to interpret metric scale from varying stereo baselines without re-rendering environments. This approach effectively decouples geometric perception from appearance, allowing zero-shot transfer from simulation to reality.
The evaluation is rigorous, comparing against strong baselines like YOPO V2 and Ego-Planner V2 in both simulation and real-world deployment. The paper demonstrates a 95% success rate in unseen simulated environments (forest, industrial) and successful real-world flight at 7 m/s. Ablation studies clearly isolate the contributions of the foundation model initialization, the interleaved attention mechanism, and the VBA strategy. The comparison with depth-based methods shows that stereo RGB can achieve competitive performance without the hardware cost and latency of depth sensors.
The authors commit to open-sourcing the software, simulation environments, and hardware designs. The paper provides detailed hyperparameters, loss function definitions, and architectural specifics. The use of standard Unity simulation and ROS interfaces enhances reproducibility. The specific version of the foundation model (DA3-Small) and its integration details are clearly described.
The method relies on the availability of a suitable vision grounding foundation model (DA3), which may not be universally available or optimal for all tasks. The real-world evaluation, while impressive, is limited to specific environments (forest, industrial) and may not generalize to all cluttered scenarios (e.g., dynamic obstacles, extreme weather). The computational cost, while reduced, still requires an NVIDIA Orin NX, which is a significant hardware requirement for "compact" drones.
This work bridges the gap between high-level visual foundation models and low-level robotic control, a critical area for autonomous systems. By demonstrating that RGB-only stereo vision can replace depth sensors for agile navigation, it reduces hardware costs and increases the robustness of aerial robots. The methodology of using foundation model features for sim-to-real transfer in control tasks is likely to inspire similar approaches in other robotic domains. Stereo-Planner enables zero-shot sim-to-real quadrotor navigation using stereo RGB by leveraging a vision grounding foundation model for geometry-aware scene representation and a scale-conditioned trajectory decoder. The paper presents a significant technical contribution by demonstrating that pretrained visual features can effectively bridge the sim-to-real domain gap for end-to-end motion planning, achieving high-speed autonomous flight without depth sensors or fine-tuning.
Quadruped robots deployed for inspection and field missions must cross heterogeneous terrain, from stairs, slopes, and gaps to high platforms, stepping stones, and balance beams, where safe support is sparse or see-through. Existing methods interpose a terrain map or a depth image between sensing and control. Maps are costly to maintain, corrupted by odometry drift and outliers, and misled by see-through gratings, while depth cameras degrade with lighting and surface material. In this paper, we present PointWalker, a mapless framework whose single reinforcement learning policy directly consumes raw point clouds from four light detection and ranging (LiDAR) sensors. Voxelized points are encoded by linear self-attention and fused with proprioception through cross-attention, so terrain geometry reaches the policy without maps or foothold estimation. One proximal policy optimization (PPO) run, with terrain-aware rewards and dual-channel simulation of see-through gratings, yields one policy for all seven terrain classes with a 90.1% mean success rate in simulation. Deployed zero-shot on a Deep Robotics X30 at 50 Hz onboard, it crosses 0.8-m gaps, climbs 0.8-m platforms and 45° steel-grating stairs, carries a 50-kg payload, and completes a 2.5-km mountain route.
Primary: Zhejiang University
All Institutions: Zhejiang University, Yunshenchu Technology Co. Ltd., Zhejiang Key Laboratory of Additive Manufacturing Technology and Equipment, Hangzhou Public Library
PointWalker introduces a mapless quadruped locomotion framework that directly consumes raw LiDAR point clouds via linear self-attention and cross-attention fusion, achieving robust zero-shot traversal of heterogeneous and see-through terrains. The paper demonstrates that removing intermediate terrain representations like elevation maps significantly improves performance on sparse and transparent surfaces, validated through rigorous simulation ablations and extensive real-world hardware testing on the Deep Robotics X30.
The paper proposes PointWalker, a reinforcement learning framework that eliminates intermediate terrain representations (like elevation maps or depth images) by directly processing raw LiDAR point clouds. The core architectural contribution is the use of linear self-attention to encode voxelized points into tokens, followed by cross-attention with proprioceptive history to fuse terrain geometry with robot state. This allows the policy to attend to specific relevant points (e.g., gap edges, grating bars) without quadratic computational cost. The training methodology employs a single PPO run across seven terrain classes using a curriculum, with a dual-channel simulation setup where the physics engine sees solid support while the perception channel sees sparse, see-through structures (like steel gratings). This effectively teaches the robot to distinguish between visual clutter and actual support surfaces.
The experimental evaluation is robust, featuring extensive simulation ablations comparing the proposed method against height-map encoders, depth-image encoders, and blind policies. The results show significant performance gains on sparse and see-through terrains where intermediate representations fail. Hardware validation on the Deep Robotics X30 is strong, demonstrating zero-shot transfer to complex real-world scenarios including 0.8m gaps, 45-degree steel-grating stairs, and a 2.5km mountain route. The inclusion of ablation studies on hardware (e.g., removing dual-channel simulation or foot-edge penalties) provides clear causal evidence for the necessity of specific design choices.
The paper provides detailed hyperparameters, network architectures, and simulation configurations (Isaac Lab). The use of standard RL algorithms (PPO) and well-documented domain randomization strategies enhances reproducibility. However, the specific implementation of the dual-channel simulation and the precise LiDAR noise models may require significant effort to replicate exactly. The code is not explicitly linked in the text provided, though a video is available.
The method relies on four LiDARs, which is a significant hardware cost compared to depth cameras. The perception range is limited to the local vicinity of the robot, meaning it cannot plan long-range paths or anticipate obstacles far ahead. The paper acknowledges that porting to other robot platforms with different sensor layouts is untested. Additionally, the method is reactive to local geometry and does not handle dynamic moving obstacles or support surfaces effectively.
This work advances the state-of-the-art in legged robot perception by demonstrating that raw point clouds can be effectively used for high-frequency control without intermediate maps. It addresses a critical gap in industrial inspection robots that must traverse heterogeneous, see-through terrain. The approach could inspire similar direct-perception methods in other robotic domains, such as manipulation or aerial robotics, where intermediate representations may lose critical geometric details. PointWalker introduces a mapless quadruped locomotion framework that directly consumes raw LiDAR point clouds via linear self-attention and cross-attention fusion, achieving robust zero-shot traversal of heterogeneous and see-through terrains. The paper demonstrates that removing intermediate terrain representations like elevation maps significantly improves performance on sparse and transparent surfaces, validated through rigorous simulation ablations and extensive real-world hardware testing on the Deep Robotics X30.
We present a learning-based framework that enables a full-scale bipedal humanoid to sprint at over 7 m/s with motion-prior-guided gait and perform aggressive cornering at high speed with a pronounced trunk lean of up to $25^\circ$. Achieving this combination is challenging: above approximately 4 m/s, periodic foot impacts induce severe, high-frequency yaw-rate oscillations intrinsic to high-speed gaits. This high-frequency temporal structure corrupts the history of proprioceptive feedback that flat-MLP policies rely on, while rendering conventional yaw-rate tracking rewards counterproductive, as they fight against the robot's natural oscillatory dynamics. Moreover, a single style prior shared across skills cannot tell which skill is commanded and inherently compromises the low-speed skill fidelity. To overcome these limitations, we adopt a causal multi-head attention (MHA) module to explicitly model the high-frequency temporal feedback. In place of the oscillatory yaw rate, we design a heading-direction reward that supervises only the low-frequency velocity direction, allowing agile turning at high speed without suppressing natural gait oscillations. We further introduce skill-conditioned multi-adversarial-motion-prior (MAMP) discriminators that condition the style reward on the commanded skill. Ablation experiments confirm that each mechanism is essential: replacing structured temporal encoding with a flat-history MLP caps the top speed; reverting to yaw-rate tracking prevents high-speed turning; and a single shared discriminator makes in-place turning collapse into a quasi-static stance. Our framework bridges the gap between raw sprinting speed and reference-motion fidelity, demonstrating stable high-speed sprinting and aggressive cornering in both simulation and real-robot deployment.
Primary: Tsinghua University
All Institutions: Tsinghua University
The paper presents a learning-based framework enabling full-scale humanoids to sprint at >7 m/s and perform aggressive cornering by introducing causal multi-head attention for high-frequency temporal modeling, heading-direction rewards to avoid suppressing natural gait oscillations, and skill-conditioned adversarial discriminators to maintain motion fidelity. This work significantly advances the state-of-the-art in high-speed bipedal locomotion by identifying and solving specific control challenges inherent to high-velocity gaits, bridging the gap between raw speed and reference-motion fidelity in both simulation and real-world deployment.
The paper proposes a learning-based framework for high-speed bipedal locomotion, addressing specific failure modes of standard flat-MLP policies at high velocities. The core methodological contributions are threefold: (1) the use of a causal multi-head attention (MHA) module to explicitly model high-frequency temporal feedback, which is argued to be necessary because periodic foot impacts above 4 m/s induce yaw-rate oscillations that corrupt flat history buffers; (2) the design of a heading-direction reward that supervises only low-frequency velocity direction, replacing conventional yaw-rate tracking which is shown to be counterproductive at high speeds; and (3) the introduction of skill-conditioned multi-adversarial-motion-prior (MAMP) discriminators to maintain fidelity across different skills (sprinting vs. cornering). The ablation studies presented in the abstract confirm the necessity of each component, showing that flat MLPs cap top speed, yaw-rate tracking prevents turning, and shared discriminators degrade skill fidelity.
The evaluation includes both simulation (Gazebo) and real-robot deployment. The paper reports achieving sprinting speeds over 7 m/s and aggressive cornering with trunk lean up to 25 degrees at 5.5 m/s. The inclusion of real-robot validation is a significant strength, as high-speed bipedal control is notoriously difficult to transfer from simulation to reality. The ablation experiments are described as rigorous, isolating the impact of the temporal encoding, reward structure, and discriminator conditioning. The results demonstrate a clear capability gap closure between raw speed and motion fidelity.
The paper is accepted to CoRL 2026, a venue known for rigorous standards. While the provided text is an abstract and figure caption, the mention of specific hyperparameters (e.g., 7 m/s, 25 degrees) and the use of standard simulation environments (Gazebo) suggest a level of detail typically found in CoRL papers. However, without the full code release or detailed hyperparameter tables in the provided snippet, full reproducibility cannot be guaranteed, though the methodology is clearly defined.
The primary limitation is the reliance on a specific hardware platform (full-scale humanoid), which may limit immediate generalizability to other morphologies. Additionally, the high-speed control regime (>4 m/s) is inherently unstable and sensitive to environmental perturbations, which may not be fully explored in the provided abstract. The "causal MHA" approach may introduce computational overhead compared to flat MLPs, though this is likely negligible for onboard controllers.
This work pushes the physical limits of bipedal robotics, demonstrating that learning-based policies can handle the complex, high-frequency dynamics of sprinting. The insights regarding the failure of flat-MLP policies at high speeds and the necessity of explicit temporal modeling for high-frequency feedback are valuable for the broader robotics community. The skill-conditioned discriminator approach offers a generalizable technique for multi-skill learning in embodied agents. The paper presents a learning-based framework enabling full-scale humanoids to sprint at >7 m/s and perform aggressive cornering by introducing causal multi-head attention for high-frequency temporal modeling, heading-direction rewards to avoid suppressing natural gait oscillations, and skill-conditioned adversarial discriminators to maintain motion fidelity. This work significantly advances the state-of-the-art in high-speed bipedal locomotion by identifying and solving specific control challenges inherent to high-velocity gaits, bridging the gap between raw speed and reference-motion fidelity in both simulation and real-world deployment.