Last 7 Days (October 09 – October 15, 2026)
Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fundamental bottleneck: the cost of evaluating progress. In real settings, a single true evaluation, such as a full benchmark or serving trace, can take hours, so a search can afford only a handful of true evaluations. We present DuoDiscover, a search system that jointly discovers what to optimize and how to evaluate progress efficiently, through three mechanisms. First, each worker in a fleet of coding agents co-discovers an optimization target and a fast, tailored proxy evaluator, which we call a yardstick. The agent then hill-climbs the yardstick rather than the true evaluation. Second, since search can exploit gaps between a yardstick and the true objective, fidelity tests continually recheck each yardstick's ranking and invalidate any that drift. Third, the fleet is bottom-up, with no central decomposition of the problem. A lightweight orchestrator spawns and terminates agents, briefs running ones, and seeds new agents with others' confirmed work. DuoDiscover enables autonomous optimization from software to chip architecture. We demonstrate DuoDiscover on two repository-level benchmarks and four GPU and TPU kernel suites covering hundreds of kernels. On SWE-fficiency, it doubles the average speedup of human experts. On SOL-Exec, it ranks first on 121 of 235 kernels and in the top three on 212. On the heavily optimized vLLM and SQLite engines, DuoDiscover achieves 1.92$\times$ and 1.38$\times$ gains over the base code. On a CPU microarchitecture optimization task with evaluations of up to 28 hours, DuoDiscover surpasses both AlphaEvolve-based ArchAgent v2 and the best human design, with 1.65$\times$ ArchAgent v2's gain in 16.7 days versus more than 40 days, advancing the frontier of autonomous chip design.
Primary: Google
All Institutions: Google
DuoDiscover introduces a co-discovery mechanism for optimization targets and proxy evaluators to overcome the evaluation bottleneck in autonomous search. The paper demonstrates significant improvements in code and hardware optimization tasks, establishing a new paradigm for scaling LLM-based discovery in high-cost domains.
The paper introduces DuoDiscover, a search system that addresses the "evaluation bottleneck" in autonomous code optimization. The core novelty is the "co-discovery" of optimization targets and cheap proxy evaluators ("yardsticks") by LLM agents. Unlike prior work (e.g., AlphaEvolve) that treats the evaluator as fixed or user-defined, DuoDiscover allows agents to construct their own fast metrics to guide hill-climbing, with a rigorous "fidelity test" (Kendall's tau rank correlation) to invalidate proxies that drift from the true objective. The bottom-up fleet architecture, where a lightweight orchestrator manages expensive true evaluations while agents exploit cheap proxies, is a well-motivated and technically sound approach to scaling search in high-cost domains.
The experimental evaluation is extensive and covers a wide range of domains, from repository-level Python optimization (SWE-fficiency, FormulaCode) to GPU/TPU kernel optimization (SOL-Exec, JAXBench) and CPU microarchitecture design (DPC4). The results are strong, with DuoDiscover outperforming baselines like Claude Code and AlphaEvolve-based agents. Notably, the paper reports state-of-the-art results on the SOL-Exec leaderboard and surpasses human experts on SWE-fficiency. The inclusion of a 28-hour evaluation task (CPU prefetcher) demonstrates the method's ability to handle extreme evaluation costs, a key claim of the paper.
The paper provides detailed descriptions of the fidelity test, noise estimation, and search loops. However, as is common with large-scale agentic systems, the exact prompts and internal logic of the "orchestrator" LLM are not fully disclosed, which may limit exact reproduction. The use of specific, potentially proprietary, LLM backbones (Claude Opus 5, Gemini 3.8 Flash) also poses a reproducibility challenge for external researchers without access to these models.
The method relies heavily on the ability of LLMs to construct valid proxy evaluators. If an agent cannot construct a faithful yardstick, the system falls back to expensive true evaluations, negating the efficiency gains. Additionally, the upfront cost of building and validating yardsticks may be prohibitive for very short search budgets. The paper also notes that the method is less effective when the true objective is highly non-linear or when the gap between proxy and true objective is too large to bridge via hill-climbing.
This work has significant implications for the field of autonomous software engineering and hardware design. By solving the evaluation bottleneck, it enables the application of LLM-based search to real-world engineering problems where feedback is slow and expensive. This could accelerate progress in areas like chip design, database optimization, and large-scale system tuning, potentially reducing the time and cost of manual optimization efforts. DuoDiscover introduces a co-discovery mechanism for optimization targets and proxy evaluators to overcome the evaluation bottleneck in autonomous search. The paper demonstrates significant improvements in code and hardware optimization tasks, establishing a new paradigm for scaling LLM-based discovery in high-cost domains.
Language models are often evaluated alone but increasingly deployed together, routed between, made to debate, ensembled, or collaboratively trained. For a model inside a system, what matters most is not how well it performs on its own, but how much the system gains from its presence. We call this a model's compositional strength and formalize it using the Shapley value of a cooperative game over coalitions of a model pool, and use a model's leave-one-out contribution to the full coalition as a tractable proxy, requiring n+1 rather than 2^n system evaluations. Evaluating individual vs. compositional strengths across five model pools, twelve collaboration methods, and twelve datasets, we find them largely uncorrelated: in 12 of 15 (pool, method-family) settings, individual strength does not translate into compositional advantages, and the coupling is weakest among general-purpose models that practitioners are most likely to deploy. Standalone benchmarks are therefore often poor proxies for models' value in collaborative systems. We then propose compositional reinforcement learning (C-RL), which trains a policy inside a live multi-model system and rewards the outcome produced by the system as a whole, rather than the policy's correctness in isolation. The leave-one-out baseline in Shapley values cancels under a group policy gradient, so this needs no additional rollout cost beyond RL on the collaborative system's outcome and never requires running a coalition without the policy. Across three collaboration protocols and eight tasks spanning reasoning, coding, and scientific discovery, C-RL produces the strongest collaborator in 21 of 24 settings, lifting collaborative performance by +4.0 to +5.4 pts on average (~3x the gain of baselines), up to +13.1 pts.
Primary: University of Washington
All Institutions: University of Washington
The paper makes a significant contribution by formalizing compositional strength via Shapley values and demonstrating that standard individual benchmarks are poor predictors of a model's value in collaborative systems. By proposing Compositional Reinforcement Learning (C-RL), which efficiently trains models to maximize their contribution to a multi-agent system's outcome, the authors provide a novel and effective method for developing specialized collaborator models that generalize across tasks and protocols.
The paper introduces a rigorous formalization of "compositional strength" using Shapley values from cooperative game theory, adapted for LLM collaborations. The key methodological contribution is the derivation of a tractable Leave-One-Out (LOO) estimator that approximates the Shapley value with linear complexity ($n+1$) rather than exponential ($2^n$). The proposed training method, Compositional Reinforcement Learning (C-RL), is elegant in its use of group-relative policy gradients (GRPO) to cancel out the baseline cost of the LOO marginal, allowing the policy to be trained on system-level rewards without extra rollouts. The credit assignment mechanism, which attributes system-level rewards to individual policy generations within a multi-turn collaboration, is a significant technical challenge addressed effectively.
The evaluation is extensive, covering 5 model pools, 12 collaboration methods, and 12 datasets. The finding that individual strength and compositional strength are largely uncorrelated is a strong empirical contribution. The training experiments demonstrate that C-RL outperforms baselines (including standard RL and collaborative fine-tuning) in 21/24 settings. The generalization tests to unseen datasets and collaboration protocols are convincing, showing that C-RL teaches transferable collaboration skills rather than overfitting to a specific setup. The inclusion of proprietary frontier models as collaborators adds practical relevance.
The paper promises code release upon acceptance. The use of proprietary APIs (GPT-5.5, etc.) introduces a reproducibility caveat noted by the authors, as these models may change over time. However, the open-source components (Qwen, open-weight collaborators) allow for partial reproduction. The detailed description of the MoCo toolkit and specific hyperparameters aids reproducibility.
The reliance on proprietary models for some experiments limits full reproducibility. The computational cost of running multi-model collaborations for RL training is high, potentially limiting adoption to well-resourced labs. The credit assignment is uniform across turns, which the authors acknowledge as a simplification that could be improved. The study focuses on a specific set of collaboration protocols; while generalization is tested, the scope is not exhaustive of all possible multi-agent architectures.
This work shifts the focus of LLM evaluation and training from individual capability to system-level contribution. It provides a theoretical foundation for understanding multi-agent dynamics and a practical method for training better collaborators. This is highly relevant as LLMs are increasingly deployed in compound systems. The finding that standard benchmarks do not predict collaborative value is a critical insight for practitioners building multi-agent systems. The paper makes a significant contribution by formalizing compositional strength via Shapley values and demonstrating that standard individual benchmarks are poor predictors of a model's value in collaborative systems. By proposing Compositional Reinforcement Learning (C-RL), which efficiently trains models to maximize their contribution to a multi-agent system's outcome, the authors provide a novel and effective method for developing specialized collaborator models that generalize across tasks and protocols.
Terminal agents solve tasks in software engineering and scientific research by executing commands in a terminal. Training them via reinforcement learning (RL) requires diverse and verifiable environments, but these environments are still mostly designed by human researchers, so their supply grows only as fast as researchers can work. We introduce AutoEnvScaling, which treats environment design as a terminal task and uses agents to automate the data flywheel. A proposer agent builds new RL environments from the rollouts of the solver being trained, and the solver's new rollouts guide the next round. By turning environment generation into a terminal task, AutoEnvScaling increases the valid-environment rate by up to 25 times compared to prompt-only generation, at up to 75% lower cost per valid environment. We show that training Qwen3.5-35B-A3B on AutoEnvScaling environments from GPT-5.6-sol improves its Terminal-Bench 2.1 score by 4.3 points over the matched Tmax baseline. The broader goal of AutoEnvScaling is to enable recursive self-improvement (RSI): by making both environment generation and solving terminal tasks, it lets one model co-evolve in both roles. We show that Qwen3.6-35B-A3B, acting in both roles, gains 12.2 points on Terminal-Bench 2.1 and raises its Terminal-Bench 4.0 score from 1.6 to 3.2. The pipeline also generalizes across agentic domains, generating environments for software engineering, computer use, professional work, and GPU kernel programming. Overall, these results show a path toward agents building their own training environments, reducing human reliance and taking a core step toward RSI.
Primary: Northeastern University
All Institutions: Northeastern University, Microsoft Research
AutoEnvScaling automates the generation of RL training environments by treating environment design as a terminal agent task, enabling a recursive self-improvement loop where agents co-evolve as both task designers and solvers. The paper demonstrates that this approach significantly increases the rate of valid environment generation and improves downstream agent performance on terminal benchmarks, offering a scalable path toward reducing human reliance in agent training pipelines.
The paper proposes AutoEnvScaling, a framework that reframes the generation of RL training environments as a terminal agent task. Instead of using static pipelines or simple prompt-based generation, the method employs a "proposer" agent operating within a sandboxed terminal environment. This agent reads solver rollouts, identifies failure modes, and iteratively builds, tests, and calibrates new Harbor tasks (instructions, environments, reference solutions, tests). The core innovation is the "data flywheel" where the solver's performance directly informs the proposer's next environment generation, and the proposer itself can be trained (via DPPO) to improve at environment design. This creates a recursive self-improvement (RSI) loop where a single model can co-evolve as both the task designer and the solver. The methodology is technically sound, leveraging existing RL techniques (DPPO) but applying them to a novel multi-agent, self-play context with distinct reward structures for design vs. solving.
The experiments are extensive and well-controlled. The authors compare their method against prompt-only and feedback-driven baselines across eight frontier models, showing a 25x increase in valid environment rates. They demonstrate downstream utility by training Qwen3.5 models on AutoEnvScaling-generated environments, showing significant gains (4.3-7.4 points) on Terminal-Bench 2.1 over static Tmax baselines. The RSI experiment is particularly compelling, showing a 12.2-point gain when the model acts as both proposer and solver. The inclusion of a qualitative study on "learning behavior" (asking for help) further validates the quality of the generated curricula. The use of matched baselines (regenerating Tmax pools with the same proposer) ensures fair comparison.
The paper provides detailed descriptions of the Harbor framework, the sandbox setup, and the validation/calibration criteria. However, specific hyperparameters for the DPPO training and the exact implementation details of the "harness optimization" are relegated to appendices or referenced generally. The reliance on proprietary frontier models (GPT-5.6, Claude Opus 5) for the proposer role limits immediate reproducibility for those without access, though the open-weight Qwen experiments provide a path for replication. The code and environment definitions are not explicitly linked in the provided text, which is a minor gap.
The primary limitation is the heavy reliance on strong frontier models to bootstrap the proposer role, which may not be accessible to all researchers. The RSI results, while impressive, are demonstrated on a specific model family (Qwen) and benchmark (Terminal-Bench), leaving open questions about generalization to other domains or model architectures. The paper also acknowledges that the validation checks cannot catch all types of errors (e.g., shared mistakes between reference solution and tests), which could lead to subtle reward hacking if not carefully monitored.
This work has significant implications for the scalability of RL training for agents. By automating the most labor-intensive part of the RL pipeline (environment design), it could drastically reduce the cost and time required to train capable agents. The concept of RSI through self-generated curricula is a major step toward autonomous AI improvement, with potential applications in scientific discovery, software engineering, and other complex problem-solving domains. It shifts the paradigm from human-designed curricula to agent-designed curricula, potentially enabling agents to discover novel skills and strategies that humans might not anticipate. AutoEnvScaling automates the generation of RL training environments by treating environment design as a terminal agent task, enabling a recursive self-improvement loop where agents co-evolve as both task designers and solvers. The paper demonstrates that this approach significantly increases the rate of valid environment generation and improves downstream agent performance on terminal benchmarks, offering a scalable path toward reducing human reliance in agent training pipelines.
Configuring distributed LLM training across hundreds of GPUs requires jointly optimizing tensor, pipeline, expert, context, and data parallelism alongside micro-batch size, recomputation, sharding, and overlap, a combinatorial space where memory legality can hinge on a single byte of activation accounting. Today, teams navigate it through expensive multi-node trial and error, burning GPU-hours on crashes and suboptimal recipes. We present Primus Projection, which closes this loop with two coupled components. First, a hybrid projection engine that measures what it can and simulates what it cannot: GPU microbenchmarks on as few as a single device capture compute kernels and any in-scope communication; a sub-node downscale-measure-upscale workflow analytically restores out-of-scope collectives, FSDP overlap, and the full pipeline schedule at the multi-node target. A pure simulation mode runs entirely on CPU, enabling pre-silicon planning without any GPU. Second, the Primus Tuning Agent treats the projection engine as a fast scoring oracle (seconds per candidate instead of tens of minutes) and uses it to drive an intelligent search over the configuration space. A deterministic seed planner sweeps single axes; an LLM investigation loop proposes cross-axis combinations the planner cannot reach, with an analytic memory pre-filter that rejects infeasible candidates before any GPU time is spent. Validated on Llama 3.1 (dense) and Mixtral 8x22B (MoE) across two GPU generations, all projections stay within ~10% of measured multi-node throughput, and fidelity holds at production scale: 4.7% error on DeepSeek V3 671B at 1,024 GPUs and 1.9% on Qwen3-235B-A22B at 256 GPUs. In a Mixtral 8x22B case study, the Primus Tuning Agent discovers a configuration delivering +27% throughput over the published 4-node BF16 reference at the same scale, in under 30 minutes of single-node exploration.
Primary: Advanced Micro Devices (AMD)
All Institutions: Advanced Micro Devices (AMD)
The paper presents a highly effective hybrid system for optimizing distributed LLM training configurations by combining single-node micro-benchmarks with analytical simulation and LLM-driven search. Its rigorous validation on production-scale models and significant throughput improvements demonstrate substantial practical value for large-scale AI infrastructure, though its impact is somewhat constrained by its specific validation on AMD hardware.
The paper introduces "Primus Projection," a hybrid performance modeling framework that combines empirical micro-benchmarks on limited hardware (down to a single GPU) with analytical simulation for out-of-scope communication and pipeline scheduling. The methodology is rigorous, addressing the "combinatorial explosion" of distributed training configurations (TP, PP, EP, CP, DP, etc.) by creating a fast scoring oracle. The "downscale-measure-upscale" workflow is a clever engineering solution to the problem of requiring full-cluster access for profiling. The integration of an LLM-based tuning agent (using DSPy and ReAct-style reasoning) to navigate this configuration space is a modern and effective approach, leveraging the projection engine as a cheap surrogate to avoid expensive trial-and-error on real clusters. The memory pre-filtering mechanism is particularly strong, preventing OOM errors before any GPU time is spent.
The evaluation is extensive and convincing. The authors validate their projection engine against measured multi-node throughput on AMD MI325X and MI355X GPUs, achieving within ~10% error for standard models and as low as 1.9-4.7% error for production-scale models (DeepSeek V3, Qwen3-235B). The case study on Mixtral 8x22B demonstrates a +27% throughput improvement over published references using only single-node exploration. The comparison against analytical baselines (Calculon, Galvatron) clearly shows the superiority of the hybrid benchmark-anchored approach, particularly for MoE models where analytical models fail. The robustness analysis regarding MoE load balancing is a valuable addition, showing how the model handles imbalance.
The paper provides detailed algorithmic descriptions (Algorithm 1), hardware profile formats (YAML), and clear definitions of the search space. However, as an industry paper from AMD, the specific code implementation and hardware profiles are likely proprietary or not fully open-sourced, which may limit immediate reproducibility by external researchers without access to similar AMD hardware. The methodology is described well enough for replication in principle, but the "black box" nature of the LLM agent's prompts and the specific hardware calibration data are potential barriers.
The primary limitation is the vendor-specific validation; all experiments are on AMD Instinct GPUs and the ROCm stack. While the authors argue the method is vendor-agnostic, the lack of validation on NVIDIA or TPU hardware is a significant gap for a general ML audience. Additionally, the LLM agent's performance depends on the quality of the projection oracle; if the oracle has systematic biases (as noted with the optimistic/pessimistic offsets), the agent might converge to suboptimal local minima if the ranking is distorted, although the paper argues rank preservation is robust. Network contention is not modeled, which could be an issue in highly congested clusters.
This work has high practical impact for organizations training large-scale LLMs, as it reduces the cost and time of cluster configuration. It bridges the gap between theoretical performance modeling and practical engineering constraints. The use of LLMs for systems optimization is a growing trend, and this paper provides a concrete, high-fidelity example of its application. It could influence how future distributed training frameworks are designed, potentially integrating such projection oracles natively. The paper presents a highly effective hybrid system for optimizing distributed LLM training configurations by combining single-node micro-benchmarks with analytical simulation and LLM-driven search. Its rigorous validation on production-scale models and significant throughput improvements demonstrate substantial practical value for large-scale AI infrastructure, though its impact is somewhat constrained by its specific validation on AMD hardware.
Language captures only a subset of the perceptions, intuitions and intents that form meaning between speakers. What, then, are the limits of language? We take steps towards measuring this in LLMs as follows: for a set of concepts, we measure how well each concept can be transmitted between two LLMs as their capabilities scale. This requires novel measurement tools: (1) optimal messages: we introduce a prompt optimization algorithm that beats existing algorithms by up to 14% on benchmarks and accurately estimates prompt upper bounds. (2) A transmission measure: the sender is asked to label text; the receiver then tries to recover the sender's prompt, and recovery is measured; we find this correlates strongly with human label error. With these measures, we use observational scaling laws to study how transmissibility of a concept, conveyed with optimal messages, increases as transmitter and receiver capacity grows: does it asymptote below or above a threshold? Measuring 1.2k concepts in eight domains, we find concepts that (a) can be fully articulated; these concepts usually depend on local patterns, e.g. "joke ends with surprise"; (b) might be articulated by future, stronger models; these rely on whole-text structural patterns, e.g. "story's worldbuilding coherence"; (c) concepts that face articulability limits; these concepts' meaning, we find, is tied to identity and social-group awareness, e.g. "comic voice". Interesting, these same concepts are not all challenging for humans: humans have moderate-to-high levels of agreement in 76% of concepts that LLMs struggle with; articulation limits, we find, are executor relative. Finally, we characterize the nature of improvement in each case: while recoverable concepts can be articulated through definitions and explanations, irrecoverable concepts cannot be transmitted, even with examples, and might require novel learning mechanisms.
Primary: Stanford University
All Institutions: Stanford University
The paper introduces a rigorous framework for measuring the limits of language articulation in LLMs, combining novel prompt optimization with a transmission metric to categorize concepts by their articulability. By demonstrating that certain social and identity-based concepts are fundamentally harder for LLMs to transmit than others, despite high human agreement, the work provides critical insights into the boundaries of language-based AI control and alignment.
The paper introduces a novel framework for measuring the "articulability" of concepts in LLMs, decomposing the problem into message optimization and transmission measurement. The core methodological contribution is $\epsilon$-estimation, a prompt optimization algorithm that uses capture-recapture statistics to estimate the upper bound of prompt performance by modeling the space of "semantic units." This is a creative application of ecological statistics to prompt engineering. The second contribution is the "reconstruction bottleneck," a transmission metric where a receiver LLM attempts to reconstruct the sender's prompt from labeled examples. The methodology is rigorous, with clear definitions of upper/lower bounds (planted rules, permutation nulls) and a strong theoretical grounding in information theory (mutual information). The approach effectively separates the capability of the model from the clarity of the language used to convey a concept.
The experiments are extensive, covering 1.2k concepts across eight diverse domains (humor, math, code, etc.). The paper compares $\epsilon$-estimation against SOTA prompt optimizers like GEPA, showing up to 14% improvement. The scaling laws analysis is particularly strong, categorizing concepts into three regimes: fully articulable, potentially articulable with scaling, and fundamentally limited. The finding that "identity and social-group awareness" concepts are the hardest to articulate, while humans show high agreement on them, is a significant and surprising empirical result. The use of human inter-rater agreement (Krippendorff's alpha) as a ground truth for concept clarity adds robustness.
The paper provides detailed descriptions of the unit mining process, the capture-recapture estimator, and the reconstruction protocol. The use of standard benchmarks (AIME, HotpotQA, IFBench) and public datasets (McSweeney's, Reddit) aids reproducibility. However, the specific implementation details of the "unit" extraction and the exact prompt templates for the reconstruction bottleneck are not fully specified in the provided text, which may hinder exact replication. The code availability is not explicitly stated in the snippet, but the level of detail suggests it is likely released.
The primary limitation is the reliance on LLMs as both transmitters and receivers, which may not fully capture human-like articulation limits. The "unit" decomposition assumes that prompts can be atomized into independent semantic units, which may not hold for highly contextual or stylistic concepts. The capture-recapture estimator assumes exchangeability of units, which is a strong assumption. Additionally, the paper focuses on binary labeling tasks, which may not generalize to more complex, multi-class, or generative articulation tasks.
This paper has significant implications for AI alignment and interpretability. By identifying which concepts are "inarticulable" to LLMs, it highlights the limits of using language as a medium for control and evaluation. This is crucial for developing AI systems that operate on nuanced, subjective, or socially constructed concepts. The framework could be used to design better rubrics for LLM judges and to identify areas where human feedback is essential. It also provides a new lens for understanding the gap between human and machine understanding of language. The paper introduces a rigorous framework for measuring the limits of language articulation in LLMs, combining novel prompt optimization with a transmission metric to categorize concepts by their articulability. By demonstrating that certain social and identity-based concepts are fundamentally harder for LLMs to transmit than others, despite high human agreement, the work provides critical insights into the boundaries of language-based AI control and alignment.
Recent work has sought to transfer first-order nonconvex optimization guarantees to zeroth-order methods through gradient estimators, typically incurring a factor linear in $d$. \citet{kornowski2024} obtained an $\mathcal{O}(dδ^{-1}ε^{-3})$ oracle complexity for nonsmooth nonconvex optimization and asked whether its dependence on all three parameters is jointly optimal. Although the dimension dependence and the first-order dependence on $(δ,ε)$ are separately known to be unavoidable, a matching zeroth-order lower bound has remained open. We answer this question in the unrestricted full-space setting. For smooth stochastic nonconvex optimization, we establish a scalar-oracle lower bound of $Ω\!\left(dL_0^2\min\{ΔH,L_0^2\}/ε^4\right)$ assuming a smooth, globally lower-bounded population objective and a second-moment bound on sample Lipschitz constants. A smooth reduction then yields a lower bound for Goldstein stationarity, proving the joint minimax optimality of the dependence on $(d,δ,ε)$ under explicit high-dimensional and small-accuracy conditions. Crucially, our results allow algorithms to query arbitrary points in $\mathbb{R}^d$, adaptively select sample-function labels, reuse the same sampled function at any number of locations, and output points without spatial restrictions. Thus, unlike lower bounds based on bounded query regions or fresh evaluation noise, our guarantees apply to a fully adaptive, reusable-sample scalar oracle. We also establish a separate noiseless exact-value lower bound against randomized algorithms, and hence also against deterministic methods. Our proof combines a globally defined sequential nonconvex construction with an exact Gaussian scalar channel, multistage random-field estimates, and an antithetic finite-sample realization that preserves information hardness under adaptive reuse.
Primary: Nanjing University
All Institutions: Nanjing University, State Key Laboratory for Novel Software Technology
The paper establishes a joint minimax lower bound for stochastic zeroth-order nonconvex optimization, proving that the dependence on dimension, smoothness, and accuracy is jointly optimal under a fully adaptive, reusable-sample scalar oracle model. By constructing a global zero-chain with hidden workspace directions and employing information-theoretic tools to control adaptive sample reuse, the authors demonstrate that the oracle complexity scales as $\Omega(d L_0^2 \min\{\Delta H, L_0^2\} / \epsilon^4)$, resolving a key open question in the field and providing a rigorous theoretical limit for gradient-free optimization methods.
The paper addresses a significant gap in zeroth-order optimization theory: establishing a joint lower bound for the dependence on dimension ($d$), smoothness ($H$), and accuracy ($\epsilon$) in the stochastic nonconvex setting. The authors propose a novel construction that combines a global zero-chain geometry with a "moving well" mechanism to force sequential information acquisition. The core technical contribution is the proof that even with a fully adaptive, reusable-sample scalar oracle (where algorithms can query any point and reuse sample functions), the complexity scales as $\Omega(d L_0^2 \min\{\Delta H, L_0^2\} / \epsilon^4)$. The methodology is rigorous, utilizing Gaussian process representations, antithetic sampling to create a finite static oracle, and information-theoretic arguments (specifically spherical concentration and log-Sobolev inequalities) to bound the information gain per query. The construction is complex, involving careful management of global Lipschitz moments and spatial cutoffs to ensure the hardness persists globally rather than just locally.
As a pure theory paper, there are no empirical experiments. The "evaluation" is the mathematical proof of the lower bound. The rigor of the proof is the primary metric of quality here. The paper provides detailed appendices (referenced but not fully visible in the snippet, though the main text outlines the structure) to support the claims. The lack of empirical validation is standard for this subcategory of optimization theory.
The paper is highly reproducible in the sense that the mathematical results are deterministic and verifiable by other theorists. There is no code to reproduce. The constants and conditions are explicitly stated, allowing for independent verification of the bounds.
The lower bound holds under specific high-dimensional and small-accuracy conditions (explicitly stated as $d \ge C(B/\epsilon^2)(2+BL_0^2/\epsilon^4)$). The bound is for the smooth stochastic case, and the extension to nonsmooth Goldstein stationarity is via a reduction, which may not be tight in all regimes. The construction is quite intricate, making it difficult to generalize to other oracle models without significant modification. The constants involved are conservative (e.g., $2^{128}$), which limits immediate practical utility for tight algorithm design but is typical for lower bound constructions.
This paper settles an open question regarding the joint optimality of zeroth-order optimization complexities. It provides a definitive answer to whether the linear dependence on dimension $d$ is unavoidable in the stochastic nonconvex setting when using scalar oracles. This has implications for the design of gradient-free optimization algorithms, confirming that one cannot hope for sub-linear dimension dependence in the worst case under these assumptions. It strengthens the theoretical foundation for understanding the limitations of zeroth-order methods compared to first-order methods. The paper establishes a joint minimax lower bound for stochastic zeroth-order nonconvex optimization, proving that the dependence on dimension, smoothness, and accuracy is jointly optimal under a fully adaptive, reusable-sample scalar oracle model. By constructing a global zero-chain with hidden workspace directions and employing information-theoretic tools to control adaptive sample reuse, the authors demonstrate that the oracle complexity scales as $\Omega(d L_0^2 \min\{\Delta H, L_0^2\} / \epsilon^4)$, resolving a key open question in the field and providing a rigorous theoretical limit for gradient-free optimization methods.
We introduce an approach for self-supervised representation learning designed to be used as a tokenizer for generative modeling. Our objective supervises a representation loss by factorizing the pointwise mutual information of the data, which we estimate by taking information differences between conditional predictions from a learned generative model. We justify our algorithm theoretically and empirically, demonstrating that our generative approach to representation learning avoids dimensional collapse and feature suppression in cases where commonly used methods like DINO and LeJEPA fail. Moreover, we show our representation learning technique is scalable; expending more compute leads to predictably better latent tokenization. For the first time, we perform a careful analysis of the combined flop efficiency of tokenizer and generative model and demonstrate that our approach is a compute-optimal improvement over direct diffusion for pixel-space generation, obtaining an FID-50K of 2.18 with JiT-L. Applied to ImageNet and the text-to-image dataset GPIC, we show tokenizer encoder scaling laws for diffusability, where extrapolating our approach achieves representations in 7x fewer FLOPs than DINOv2 on ImageNet and 8x fewer FLOPs than WebSSL on GPIC.
Primary: Massachusetts Institute of Technology
All Institutions: Massachusetts Institute of Technology
[One sentence main contribution]. The paper introduces RePR, a generative representation learning method that factorizes Pointwise Mutual Information to create scalable, feature-suppression-robust tokenizers, demonstrating that this approach yields compute-optimal improvements over direct pixel diffusion for image generation.
The paper proposes RePR (Relative Prediction Representations), a self-supervised representation learning method that explicitly factorizes the Pointwise Mutual Information (PMI) of the data using a generative model to estimate PMI. The core innovation is replacing the discriminative InfoNCE loss (which relies on negative sampling and is prone to feature suppression) with a generative objective that estimates PMI via the ELBO of a diffusion model. The method introduces "Timestep Weighting" to bias the PMI estimation toward semantically relevant noise scales and "Bootstrapped PMI Estimation" to make the likelihood estimation tractable for high-dimensional inputs by decomposing the PMI into incremental token predictions. The architecture uses a Masked Autoencoder (MAE) variant with a diffusion head for the prediction model and a separate representation encoder. The theoretical grounding is strong, linking the method directly to the information-theoretic properties of contrastive learning while avoiding its practical pitfalls.
The experiments are rigorous and address a critical gap in the field: the compute efficiency of tokenizers. The authors demonstrate that RePR encoders exhibit predictable scaling laws for "diffusability" (how well a generative model can learn from the latent space). They show that RePR achieves DINOv2-level performance with 7x fewer FLOPs on ImageNet and 8x fewer than WebSSL on GPIC. A key result is the demonstration that using RePR with Latent Forcing yields a better FID-50K (2.18) than direct pixel diffusion (JiT-L) at equal total training FLOPs, proving that representation learning is a compute-optimal component of the generative pipeline. The ablation studies on masking strategies, loss weights, and iterative feature prediction are thorough. The feature suppression experiment using watermarked images effectively highlights the robustness of the generative PMI objective compared to contrastive baselines like DINOv3 and LeJEPA.
The paper provides a GitHub link and detailed implementation notes in the appendix, including architectural specifics (RoPE, register tokens, layer splits) and training hyperparameters. The use of standard components (ViT, MAE, Diffusion) aids reproducibility. However, the specific "Latent Forcing" method is cited as a separate work (Baade et al.), which may introduce dependency on external codebases not fully detailed here. The FLOP calculations are explicitly defined, which is crucial for the claims made.
The method relies on a diffusion model for PMI estimation, which adds computational overhead during the representation learning phase, although the paper argues this is offset by downstream efficiency. The scaling laws are extrapolated, and the "7x/8x" claims depend on the accuracy of these power-law fits. The method is primarily evaluated on image generation; applicability to other modalities (audio, video) is mentioned but not extensively tested. The reliance on a specific masking strategy and timestep weighting scheme requires careful tuning.
This paper has high potential impact by providing a theoretically grounded, scalable alternative to contrastive representation learning for generative tasks. It challenges the prevailing assumption that discriminative objectives are sufficient for tokenizers and offers a path to more compute-efficient generative models. The demonstration of compute-optimal generation via learned representations could influence the design of future foundation models, encouraging a shift towards generative self-supervision for representation learning. [One sentence main contribution]. The paper introduces RePR, a generative representation learning method that factorizes Pointwise Mutual Information to create scalable, feature-suppression-robust tokenizers, demonstrating that this approach yields compute-optimal improvements over direct pixel diffusion for image generation.
Textual reasoning before acting has improved vision-language-action (VLA) models, but at the cost of latency: every action conditioned on the reasoning text must wait for autoregressive decoding. We show that actions do not need to wait for or condition on the generated reasoning to benefit from it, because the reasoning supervision already improves the backbone representation. We introduce TacitVLA, a reasoning VLA whose action expert never attends to the reasoning it generates for the current call. On long-horizon tasks where the reasoning text also serves as episodic memory, it can generate each entry during action execution rather than before acting. We evaluate it in a controlled study on four benchmarks spanning manipulation and autonomous driving, memory and non-memory tasks: RoboMME, RMBench, LIBERO, and PhysicalAI-AV. On all four, TacitVLA matches the performance of its explicit-reasoning counterpart at up to 3.4x lower latency, and matches the latency of the same architecture trained without reasoning at up to +17.1% success. In absolute terms, it surpasses the best previously published results on the RoboMME Counting suite (82.7% vs. 66.8%) and RMBench (77.5% vs. 56.5%), and reaches 72.9% on LIBERO-Plus and a 52.5% scene score on the public split of the PhysicalAI-AV closed-loop driving challenge. Project page: https://tacitvla.github.io.
Primary: NVIDIA
All Institutions: NVIDIA
TacitVLA demonstrates that reasoning supervision improves VLA performance through representation learning rather than explicit inference-time conditioning, enabling real-time deployment with up to 3.4x lower latency. This paper provides a rigorous, controlled study across manipulation and driving benchmarks, showing that masking the action expert's attention to reasoning tokens retains the performance benefits of reasoning while eliminating the autoregressive decoding bottleneck, a significant contribution to the efficiency of embodied AI systems.
The paper proposes TacitVLA, a method that decouples the *generation* of reasoning text from the *conditioning* of actions on that text. The core insight is that the benefit of reasoning in Vision-Language-Action (VLA) models primarily stems from the representation learning induced by the reasoning supervision signal (cross-entropy loss on text tokens) during training, rather than the explicit attention to the generated text tokens during inference. By masking the action expert's attention to the reasoning tokens during both training and inference, the model retains the improved backbone representations without incurring the autoregressive decoding latency. For memory tasks, the paper introduces an asynchronous generation strategy where the reasoning text (ledger entry) is generated during the execution of the current action chunk, ensuring it is available for the next call without blocking the current action. This is a clever architectural optimization that leverages the existing Mixture-of-Transformers structure.
The evaluation is rigorous and comprehensive, covering four distinct benchmarks: RoboMME (memory/counting), RMBench (long-horizon manipulation), LIBERO/LIBERO-Plus (standard manipulation/robustness), and PhysicalAI-AV (autonomous driving). The controlled study comparing Explicit Reasoning, TacitVLA, and No Reasoning is well-designed, isolating the effect of attention masking. The results show that TacitVLA matches the performance of Explicit Reasoning while achieving the latency of the No Reasoning baseline (or slightly higher due to deferred generation on memory tasks). The latency improvements are significant (up to 3.4x), which is critical for real-time deployment. The performance gains over the No Reasoning baseline (up to +17.1% success) confirm the value of the reasoning supervision.
The paper provides detailed implementation details in the appendix, including architecture specifics, training hyperparameters, data sources, and evaluation protocols. The use of standard benchmarks (LIBERO, RoboMME, RMBench) and public datasets (PhysicalAI-AV public split) enhances reproducibility. The code and project page are available. However, the driving dataset is proprietary (3.8M events), which limits full reproducibility for that specific benchmark, though the methodology is clear.
The primary limitation is the reliance on high-quality reasoning annotations for training. If the reasoning data is noisy or incorrect, the representation learning benefit may diminish. Additionally, the asynchronous generation on memory tasks assumes that the generation time fits within the action execution window; if the reasoning text becomes very long or the hardware is slower, this assumption may break, leading to latency spikes. The paper also notes that on non-memory tasks, the reasoning text is discarded, which might miss opportunities for interpretability or safety checks that could be performed post-hoc.
This work has significant implications for the deployment of VLA models in real-world robotics and autonomous driving, where latency is a critical constraint. By demonstrating that reasoning can be "internalized" into the model's representations without the inference cost, it paves the way for more efficient and responsive embodied AI systems. The approach is generalizable to other architectures that use reasoning as a training target, potentially influencing future designs of foundation models for robotics. TacitVLA demonstrates that reasoning supervision improves VLA performance through representation learning rather than explicit inference-time conditioning, enabling real-time deployment with up to 3.4x lower latency. This paper provides a rigorous, controlled study across manipulation and driving benchmarks, showing that masking the action expert's attention to reasoning tokens retains the performance benefits of reasoning while eliminating the autoregressive decoding bottleneck, a significant contribution to the efficiency of embodied AI systems.
Generating information-rich visual artifacts such as slides, posters, and dashboards as code is appealing because the outputs are structured, editable, and verifiable. Yet code models cannot see how their choices of layout and color come together on the canvas, often limiting spatial planning and aesthetic quality. Image generators excel at visual design but can misrender requested content, making their outputs unreliable targets for direct reproduction. We therefore introduce AesCode, which generates code grounded in the content prompt while using an image generated from the same prompt as an aesthetic guidance. To do so, AesCode represents the target as a design graph that describes the whole canvas and makes each property attributable. From this graph, we derive complementary cross-modal rewards: deterministic verifiers score properties parsable from code and rendering, while a VLM judge assesses the non-parsable ones with a sample-specific Visual Graph Rubric (VGR) tied to graph elements and relations. We optimize these decoupled rewards through reinforcement learning with per-channel normalization to jointly improve spatial planning, aesthetic quality, and verifiable correctness. To evaluate this setting, we build a benchmark that scores verifiable correctness and visual quality separately, with visual scores highly consistent with human judgments. We find that reference images substantially improve the visual quality of code generated by existing models, and ablations show that AesCode benefits from references both as model input and as a source of visual rewards. Relative to same-scale Qwen3-VL baselines given the same references, AesCode-8B and AesCode-32B improve visual quality by 31.1 and 22.4 percentage points, reaching scores comparable to those of closed-source frontier models such as GPT-5.5 and Claude Opus 4.8.
Primary: Microsoft
All Institutions: Microsoft, Yale University, Nanyang Technological University, University of Pittsburgh, Shanghai Jiao Tong University, Fudan University
AesCode introduces a decoupled cross-modal reward system using a Design Graph representation to optimize aesthetic code generation via reinforcement learning. The paper presents a rigorous methodology that effectively combines deterministic verification with VLM-based aesthetic judgment, achieving state-of-the-art results that rival frontier closed-source models, thereby validating the potential of code-based approaches for high-fidelity visual design.
The paper proposes AesCode, a framework for generating visual artifacts (slides, posters) as code. The core innovation is the "Design Graph" representation, which decomposes the visual canvas into attributable properties. This allows for a decoupled reward system: deterministic verifiers check parsable properties (e.g., text presence, layout constraints), while a Vision-Language Model (VLM) judge assesses non-parsable aesthetic qualities using a sample-specific Visual Graph Rubric (VGR). The training utilizes Reinforcement Learning (RL) with per-channel normalization to optimize these distinct reward signals simultaneously. This approach effectively bridges the gap between the structural reliability of code generation and the aesthetic fidelity of image generation.
The authors introduce a new benchmark that separates verifiable correctness from visual quality, noting high consistency with human judgments. Experiments show that AesCode-8B and AesCode-32B outperform same-scale Qwen3-VL baselines by 31.1 and 22.4 percentage points in visual quality, respectively, when given reference images. The results claim parity with closed-source frontier models like GPT-5.5 and Claude Opus 4.8. The ablation studies confirm the utility of reference images both as input context and as a source for visual rewards.
The paper provides a project page and likely code release (implied by the GitHub URL structure). The methodology is detailed enough to understand the RL setup and reward decomposition. However, the reliance on a VLM judge for aesthetic scoring introduces a dependency on the specific VLM used, which may vary in robustness.
The primary limitation is the dependency on high-quality reference images for optimal performance; without them, the aesthetic guidance is lost. Additionally, the "Design Graph" construction may be complex for highly unstructured or free-form artistic layouts, potentially limiting generalizability to strict UI/dashboard design. The evaluation relies heavily on VLM-based scoring, which, while consistent with humans, is not a ground-truth metric.
This work has significant implications for the "code-as-design" paradigm, making it viable for professional-grade visual content creation. By decoupling aesthetic and functional rewards, it offers a template for other multimodal generation tasks where structural integrity and perceptual quality are distinct objectives. It could drive adoption of code-based design tools in enterprise settings where editability is crucial. AesCode introduces a decoupled cross-modal reward system using a Design Graph representation to optimize aesthetic code generation via reinforcement learning. The paper presents a rigorous methodology that effectively combines deterministic verification with VLM-based aesthetic judgment, achieving state-of-the-art results that rival frontier closed-source models, thereby validating the potential of code-based approaches for high-fidelity visual design.
Terminal agents have shown strong abilities in complex coding tasks. However, they often struggle to recover from errors during long-running execution (e.g., command errors, faulty implementations), eventually leading to task failures. To this end, we present ReTerm, a self-improving terminal agent trained by learning how to recover from its own failed terminal tasks. To support its training, we construct a customized data pipeline and produce ReTerm-10K, a high-quality dataset that provides effective recovery demonstrations for SFT cold start. Furthermore, we propose ReTerm-GRPO, an online RL algorithm that forks from the agent's own failure states to generate multiple independent self-recovery attempts. By learning from whether these attempts successfully complete the task, the agent self-improves through its own recovery experience. Built on Qwen3.5-9B, our ReTerm-9B achieves 34.8% Pass@1 on Terminal-Bench 2.0, improving over the base model by 14.6 points. Further analysis shows that our method achieves consistently higher recovery rates across diverse error categories compared with standard GRPO training. All code, data and model are released at https://github.com/twothsnd/ReTerm.
Primary: Shanghai Jiao Tong University
All Institutions: Fudan University, MMLab, Shanghai Jiao Tong University, The Chinese University of Hong Kong
ReTerm introduces a recovery-oriented training framework for terminal agents, combining a curated dataset of recovery trajectories with an online RL algorithm that learns from self-generated failure states. The paper demonstrates that explicitly training agents to recover from errors leads to substantial performance gains on terminal and coding benchmarks, offering a new perspective on agent self-improvement beyond simple task completion.
The paper proposes ReTerm, a framework for training terminal agents to recover from their own execution errors. The methodology consists of two main components: (1) ReTerm-10K, a dataset constructed by collecting failed trajectories from a base model (Qwen3.5-9B), restoring the environment state, and using a stronger teacher model (DeepSeek-V4-Pro) to generate recovery continuations, which are then filtered for quality; and (2) ReTerm-GRPO, an online reinforcement learning algorithm that forks from the agent's own failure states during training to generate multiple independent self-recovery attempts. The use of a "recovery task queue" to manage the distribution of failure states across training steps is a practical engineering contribution that addresses the sparsity of failure events in RL. The approach is logically sound, addressing the specific gap of "recovery" rather than just "task completion," and the distinction between SFT cold-start and online RL self-improvement is well-motivated.
The experiments are comprehensive, evaluating the model on Terminal-Bench 2.0, OpenThoughts-TBLite, SWE-bench Verified, and out-of-domain tasks like AIME and SpreadsheetBench. The results show significant improvements over the base model (Qwen3.5-9B) and standard GRPO baselines. The ablation study clearly isolates the contribution of the recovery-specific RL component. The error analysis section, which categorizes errors into planning, tool use, implementation, verification, and resource management, provides valuable insight into where the model improves. The case studies are detailed and illustrative of the recovery process. The performance gains are substantial (e.g., +14.6 points on Terminal-Bench 2.0), suggesting a meaningful impact on the state of the art for 9B-scale models.
The authors release code, data, and models, which is a strong positive for reproducibility. The paper provides detailed descriptions of the data pipeline, filtering criteria, and training hyperparameters. The use of specific model versions (Qwen3.5-9B, DeepSeek-V4-Pro) and benchmarks (Terminal-Bench 2.0) allows for clear replication. The prompt templates for data curation and error analysis are included in the appendix, further enhancing transparency.
The method relies on a stronger teacher model (DeepSeek-V4-Pro) for the SFT cold-start phase, which may limit scalability if such a teacher is not available. The recovery task queue mechanism adds complexity to the training loop. The evaluation is primarily on terminal/coding tasks; while out-of-domain generalization is tested, the core contribution is specific to interactive terminal environments. The paper does not extensively discuss the computational cost of maintaining and sampling from the recovery task queue compared to standard RL.
This work has significant implications for the development of autonomous coding agents. By focusing on recovery, it addresses a critical failure mode in long-horizon tasks. The techniques could be extended to other interactive domains where agents must correct their own mistakes, such as robotics or GUI agents. The release of the ReTerm-10K dataset and the ReTerm-GRPO algorithm provides a valuable resource for the community. ReTerm introduces a recovery-oriented training framework for terminal agents, combining a curated dataset of recovery trajectories with an online RL algorithm that learns from self-generated failure states. The paper demonstrates that explicitly training agents to recover from errors leads to substantial performance gains on terminal and coding benchmarks, offering a new perspective on agent self-improvement beyond simple task completion.
The success of autoregressive language models trained with Chain-of-Thought~(CoT) supervision raises a fundamental question: How simple can a next-token predictor be while still enabling the efficient simulation of general computation through its autoregressively generated CoT? Surprisingly, we show that a single neuron suffices. For a broad family of activations, including threshold and ReLU, a single autoregressive neuron can simulate any feedforward circuit with the same activation. Specifically, a size-$s$ circuit on $n$ inputs can be simulated by a neuron of dimension $O(ns+s^2)$ using $O(s^2)$ generation steps, with both bounds independent of circuit depth and tight under natural assumptions. For threshold activations, this improves upon the simulation of Joshi et al. (COLT 2025), whose bounds scale exponentially with circuit depth. Consequently, autoregressive threshold neurons can simulate polynomial-time Turing machines, while remaining efficiently learnable from CoT supervision. Our main expressivity result relies on a novel connection to Sidon sets from additive combinatorics. As an experimental illustration, we efficiently train a single autoregressive ReLU neuron to mimic a neural network teacher with nontrivial test accuracy on CIFAR-10.
Primary: Weizmann Institute of Science
All Institutions: Weizmann Institute of Science, Apple
The paper demonstrates that a single autoregressive neuron can efficiently simulate general computation via Chain-of-Thought, leveraging Sidon sets to achieve depth-independent polynomial bounds. This theoretical breakthrough clarifies the role of CoT in enabling simple models to perform complex tasks, with significant implications for understanding the efficiency and learnability of autoregressive language models.
The paper introduces a rigorous theoretical framework for the expressivity of autoregressive single-neuron models. The core innovation is the use of Sidon sets (additive combinatorics) to construct collision-free encodings of circuit weights within a single shared parameter vector. This allows a single neuron to simulate feedforward circuits of arbitrary depth with polynomial bounds on dimension and generation steps, independent of circuit depth. This is a significant improvement over prior work (Joshi et al., 2025) which suffered from exponential scaling with depth. The methodology is mathematically sound, providing tight upper and lower bounds under natural assumptions.
The experimental section is minimal but illustrative. It demonstrates that a single autoregressive ReLU neuron can be trained to mimic a CNN teacher on CIFAR-10 using activation traces. While the results (99% agreement with teacher) are strong, the experiment is a toy setup designed to validate the theory rather than a competitive benchmark. It does not compare against standard baselines or evaluate on unseen tasks, limiting its practical impact.
The theoretical proofs are detailed and self-contained. The experimental setup is described with sufficient detail (Adam optimizer, sparse implementation, 4000 trajectories) to be reproducible, though the code is not explicitly linked in the provided text.
The primary limitation is the gap between theory and practice. The simulation requires a specific feature map and structured CoT traces that are not naturally available in standard LLM training. The experiment is a controlled simulation, not a demonstration of a practical LLM architecture. The lower bounds rely on the "direct edge alignment" assumption, which may not hold for all potential simulation strategies.
This paper provides a foundational theoretical insight into why Chain-of-Thought (CoT) is effective: it allows simple predictors to simulate complex computations by exposing intermediate states. It bridges the gap between circuit complexity and neural network learning, offering a rigorous basis for understanding the efficiency of autoregressive models. It may influence future work on minimal architectures and the design of CoT supervision signals. The paper demonstrates that a single autoregressive neuron can efficiently simulate general computation via Chain-of-Thought, leveraging Sidon sets to achieve depth-independent polynomial bounds. This theoretical breakthrough clarifies the role of CoT in enabling simple models to perform complex tasks, with significant implications for understanding the efficiency and learnability of autoregressive language models.
Real-world tables are often heterogeneous, combining numerical features with free-form string columns. Table foundation models (TFMs) excel on numerical tables but require strings to be encoded numerically, while large language models (LLMs) hold semantic knowledge but are not designed for tabular predictions. Most existing bridges use LLMs as plug-in encoders for cell strings, yet how to extract representations effectively for tabular learning remains underexplored. We show that the interface between LLMs and tables matter as much as the LLM itself. Applied cell-wise, larger LLMs do not give better features. With our interface, the same LLMs improve steadily with size. TILE (Table-Interfaced LLM Embeddings) combines four ingredients: row serialization to provide cross-column context, intermediate-layer extraction to avoid layers specialized for next-token prediction, columnar pooling to remove mean pooling's length bias, and by-pass features to route numbers and categories past the LLM. Paired with TFMs, TILE sets a new state of the art for tables with strings. Across 100 datasets, it beats the strongest baseline on 89% of splits, with gains across 7 LLMs and 10 tabular learners, and ranks first on the 108-dataset STRABLE benchmark.
Primary: Inria SODA
All Institutions: Inria SODA
The paper introduces TILE, a robust interface for bridging LLMs and Table Foundation Models, demonstrating that proper serialization, layer selection, and pooling are critical for leveraging LLM scale in tabular learning. By addressing the specific limitations of cell-wise encoding and last-layer extraction, TILE achieves state-of-the-art results on heterogeneous tables, offering a practical and theoretically grounded solution to a significant gap in applied machine learning.
The paper proposes TILE, a framework for integrating Large Language Models (LLMs) with Table Foundation Models (TFMs) by treating the "interface" as a critical design space. The methodology is grounded in a clear diagnosis of why naive LLM integration fails: cell-wise isolation lacks context, and last-layer extraction captures next-token prediction biases rather than transferable semantics. The four proposed components—row serialization, intermediate-layer extraction, columnar pooling, and by-pass features—are logically derived from these failures. The "by-pass" mechanism is particularly insightful, acknowledging that LLMs are poor at numerical precision and that exact categorical identity is often more useful to tabular learners than semantic embeddings. The adaptive PCA dimensionality based on table width is a practical engineering choice that addresses the feature-count constraints of TFMs.
The experimental evaluation is extensive and rigorous. The authors test across 100 datasets from multiple sources (TextTabBench, CARTE, WikiDBs) and validate on the external STRABLE benchmark. The comparison against 30 baselines, including state-of-the-art TFMs (TabPFN, TabICL, Causilo) and LLM-based methods (TabLLM, ContextTab), is comprehensive. The key finding that "larger LLMs do not give better features" when applied cell-wise, but do so when using TILE, is a significant empirical contribution that justifies the method. The ablation studies effectively isolate the contribution of each component (context, depth, pooling).
The paper provides sufficient detail for reproduction, including specific layer indices for extraction, PCA component counts, and cardinality thresholds. The use of standard, off-the-shelf LLMs and TFMs lowers the barrier to entry. However, the lack of a provided code repository in the text (and the "none" for PROJECT_URL) slightly hinders immediate reproducibility, though the method is described clearly enough to be implemented by a skilled practitioner.
The method relies on the availability of a suitable TFM; if the TFM is weak, the gains from TILE may be diminished. The row serialization approach may struggle with extremely wide tables where the context window of the LLM is exceeded, although the adaptive PCA helps with the output side. The paper focuses on classification and regression; other tabular tasks (e.g., anomaly detection) are not explored.
This work has high practical impact for industries dealing with heterogeneous tabular data (finance, healthcare, e-commerce). It provides a clear recipe for leveraging existing LLMs without fine-tuning them for tabular tasks, which is cost-effective. The insight that the interface matters more than the model size for this specific task will likely influence future research in hybrid tabular-language models. The paper introduces TILE, a robust interface for bridging LLMs and Table Foundation Models, demonstrating that proper serialization, layer selection, and pooling are critical for leveraging LLM scale in tabular learning. By addressing the specific limitations of cell-wise encoding and last-layer extraction, TILE achieves state-of-the-art results on heterogeneous tables, offering a practical and theoretically grounded solution to a significant gap in applied machine learning.
In humans, proprioception corresponds to the sense of self-movement or position, e.g. understanding the position of one's arm. This sense is often less well-understood and in popular understanding sometimes conflated with introspection. But, notably, proprioception is a separate sense, with dedicated receptors and verifiable outcomes. On the other hand though, language models cannot observe their own internal states during training (and for example answer the question which experts activated in a certain layer of an MoE) and so cannot accurately report on them. Yet, such information is, by virtue of being internal to the model, both accessible, and readily constructed into verifiable signals. We argue that this provides the basis for a digital analog of proprioception. As such, we train models through SFT and on-policy training to equip them with this ability. We show that modern models can be trained to understand a range of subtypes of proprioception, such as routing decision awareness, token entropy, decision depth or logit lens information, without affecting model capabilities. We then run these models through a broad evaluation of situational awareness, legibility, and epiphenomenality, to evaluate in what ways access to proprioception changes model behavior.
Primary: Tübingen AI Center
All Institutions: Tübingen AI Center, ELLIS Institute T, Max Planck Institute for Intelligent Systems
The paper introduces a rigorous framework for training language models to possess "digital proprioception," enabling them to accurately report on verifiable internal states such as MoE routing decisions and logit distributions. By distinguishing this capability from unverifiable introspection and employing on-policy training to prevent label staleness, the authors demonstrate that models can acquire precise self-knowledge without degrading general capabilities, offering a promising new avenue for enhancing model interpretability and safety through structured self-monitoring.
The paper proposes a novel training paradigm for "digital proprioception," distinguishing it from introspection by focusing on verifiable, internal, non-transparent states. The methodology involves a two-stage training process: Supervised Fine-Tuning (SFT) on templated questions and On-Policy SFT (inspired by DAgger) where labels are generated from the live forward pass to avoid stale labels. This on-policy approach is critical for maintaining the integrity of the proprioceptive signal as model weights update. The authors address practical challenges like kernel non-determinism in MoE routing and multi-token answer supervision. The theoretical framing, separating proprioception (verifiable internal states) from introspection (unverifiable self-reports), is a strong conceptual contribution that clarifies a muddy area of AI safety research.
The experiments are rigorous, testing three distinct domains of proprioception: MoE routing decisions, token logits/entropy, and logit lens information. The models used (Llama 2.5, OLMoE, Qwen, GLM) are modern and relevant. Key results show that models can learn to report these internal states with high accuracy without degrading general capabilities. The evaluation of behavioral changes (situational awareness, legibility) is thorough, showing that proprioception training does not negatively impact interpretability (probe accuracy) or safety metrics. The edge case analysis regarding routing non-determinism is particularly insightful, demonstrating the limits of the model's self-knowledge when the underlying computation is unstable.
The paper provides detailed descriptions of the training setup, including loss weighting strategies, context lookup tables to prevent shortcut learning, and synchronization protocols for on-policy training. However, specific hyperparameters for the on-policy stage and the exact implementation of the "live" label extraction are described at a high level. The code is not explicitly linked in the provided text, but the methodological details are sufficient for a skilled practitioner to attempt reproduction. The reliance on specific MoE architectures (like Qwen3.6) may limit immediate reproducibility for those without access to such models, but the principles are generalizable.
The primary limitation is the scope of "proprioception" defined here, which is restricted to valence-neutral, mechanistic queries. It does not address more complex or subjective internal states. Additionally, the non-determinism of MoE routing kernels poses a fundamental upper bound on the accuracy of routing proprioception, as noted in the paper. The behavioral changes observed are subtle, and it remains unclear if this training leads to significant long-term safety benefits or risks, as the effects on "legibility" were neutral.
This work has significant implications for AI safety and interpretability. By providing a method to train models to report on their internal states with verifiable ground truth, it offers a new tool for debugging and monitoring LLMs. It challenges the notion that self-knowledge in LLMs is purely mimetic or unreliable, suggesting that specific, structured self-knowledge can be engineered. This could lead to more transparent and controllable AI systems, where models can explicitly report on their decision-making processes (e.g., which experts were activated) in a way that is auditable. The paper introduces a rigorous framework for training language models to possess "digital proprioception," enabling them to accurately report on verifiable internal states such as MoE routing decisions and logit distributions. By distinguishing this capability from unverifiable introspection and employing on-policy training to prevent label staleness, the authors demonstrate that models can acquire precise self-knowledge without degrading general capabilities, offering a promising new avenue for enhancing model interpretability and safety through structured self-monitoring.
Rapid advances in large language models have made software engineering agents capable of tackling complex coding tasks. Within a containerized environment, a coding agent receives tool responses or execution results, and determines its later actions by those earlier results. However, most existing coding agents never look ahead: their policies do not anticipate the consequences of actions, and the terminal reward barely attributes a failure to certain earlier steps, so an ill-judged edit can corrupt the repository state irreversibly before any signal arrives. Recent code world models aim to address these challenges, but they are either trained and then frozen before reinforcement learning (RL), or reduced to an auxiliary next-token loss on the environment's outputs. We present ANTE (Anticipate, Then Execute), an agentic RL framework that verifies the policy's pre-action predictions by real execution and turns them into per-step credit for its actions and trust-or-abstain decisions. With an internal belief stated before each action, Belief-Anchored Policy Optimization (BAPO) turns the settled predictions into per-step advantages by anchoring credit on the policy's own forecasts. To turn the trained forecasts into better actions at inference, thereby facilitating effective test-time scaling, ANTE-TTO (test-time optimization) selects among candidate actions without an external scorer. ANTE raises Qwen3-32B from 25.0% to 57.2% pass@1 on SWE-bench Verified, 6.2% above a matched GRPO baseline trained without the belief state; its world model's accuracy climbs from 44% to 69% during RL, and the learned foresight capability transfers across harnesses and out-of-distribution domains. In summary, we aim to build a code world model that knows what has happened, what holds now, and what the next action will achieve.
Primary: UC Berkeley
All Institutions: UC Berkeley, Google DeepMind
[One sentence main contribution]. The paper presents ANTE, a framework that uses verified internal world models to improve credit assignment in coding agents via Belief-Anchored Policy Optimization, significantly boosting performance on SWE-bench Verified. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The technical contribution is substantial, addressing a core challenge in agentic RL: the misalignment between sparse terminal rewards and the need for fine-grained action evaluation. By anchoring credit on the agent's own forecasts, ANTE creates a dense, self-supervised signal that guides policy optimization more effectively than standard methods. The methodology is innovative, moving beyond treating world models as static predictors or auxiliary losses, and instead integrating them dynamically into the RL loop. The experimental results are strong, showing a large absolute gain in pass@1 on a difficult benchmark, which is a key metric for coding agents. The significance to the field is high, as it provides a scalable and effective approach to training agents that can anticipate consequences, a crucial capability for real-world deployment. The work bridges the gap between world modeling and policy optimization, offering a new paradigm for agentic learning.
The paper introduces ANTE (Anticipate, Then Execute), a framework that integrates a learned internal world model into the reinforcement learning loop for coding agents. The core innovation is "Belief-Anchored Policy Optimization" (BAPO), which uses the agent's own pre-action predictions (beliefs) about the environment's state to compute per-step credit assignment, rather than relying solely on sparse terminal rewards. This addresses the credit assignment problem in long-horizon agentic tasks. The method also includes ANTE-TTO (Test-Time Optimization), which leverages the trained world model to select among candidate actions at inference time without an external verifier. The methodology is sound, combining standard RL techniques with a novel use of self-predictive capabilities for credit assignment. The approach is distinct from prior work that freezes world models before RL or uses them only as auxiliary losses.
The experiments are conducted on SWE-bench Verified, a standard and challenging benchmark for software engineering agents. The paper reports a significant improvement in pass@1 from 25.0% to 57.2% using Qwen3-32B, outperforming a matched GRPO baseline by 6.2%. The world model's accuracy improves from 44% to 69% during RL, demonstrating that the foresight capability is learned effectively. The paper also claims transferability of the learned foresight across different harnesses and out-of-distribution domains, which is a strong claim supported by the reported results. The evaluation is rigorous and uses a large, capable base model, making the results highly relevant to current state-of-the-art practices.
The paper provides detailed descriptions of the BAPO algorithm and the ANTE-TTO process. However, specific hyperparameters, training schedules, and implementation details for the world model architecture are likely in the appendix (referenced but not fully visible in the prompt). The use of standard benchmarks and open-source base models (Qwen3) aids reproducibility. The absence of a provided code repository in the extracted text is a minor drawback, but the methodological clarity is high.
The primary limitation is the computational cost of maintaining and updating a world model during RL, which may be prohibitive for smaller models or resource-constrained settings. The reliance on a large base model (Qwen3-32B) means the results may not directly translate to smaller agents. Additionally, the "trust-or-abstain" decision mechanism's robustness to adversarial or highly noisy environment feedback is not deeply explored. The paper focuses on coding tasks, and generalization to other agentic domains (e.g., robotics, web navigation) is implied but not extensively tested.
This work has significant implications for the development of autonomous agents in complex, stateful environments. By improving credit assignment through internal world models, it offers a pathway to more reliable and efficient agent training. The technique of using self-predictions for credit assignment could be generalized to other domains where sparse rewards are a bottleneck, such as robotics or scientific discovery. The improvement in SWE-bench Verified performance suggests practical utility in software engineering automation. [One sentence main contribution]. The paper presents ANTE, a framework that uses verified internal world models to improve credit assignment in coding agents via Belief-Anchored Policy Optimization, significantly boosting performance on SWE-bench Verified. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The technical contribution is substantial, addressing a core challenge in agentic RL: the misalignment between sparse terminal rewards and the need for fine-grained action evaluation. By anchoring credit on the agent's own forecasts, ANTE creates a dense, self-supervised signal that guides policy optimization more effectively than standard methods. The methodology is innovative, moving beyond treating world models as static predictors or auxiliary losses, and instead integrating them dynamically into the RL loop. The experimental results are strong, showing a large absolute gain in pass@1 on a difficult benchmark, which is a key metric for coding agents. The significance to the field is high, as it provides a scalable and effective approach to training agents that can anticipate consequences, a crucial capability for real-world deployment. The work bridges the gap between world modeling and policy optimization, offering a new paradigm for agentic learning.
In multi-group learning, a single predictor must perform nearly as well as the best hypothesis in a class on every group in a rich, possibly infinite and overlapping, collection of subpopulations. The known algorithms with the optimal sample complexity keep track of every pair of a group and a hypothesis, which is impossible for the large classes of interest, and the known oracle-based algorithms have a worse rate. We ask whether the optimal rate can be attained by a learner that accesses the classes only through an empirical risk minimization (ERM) oracle. We give an algorithm that uses $\tilde{O}((d_G+d_H)/\varepsilon^2)$ samples and polynomially many calls to a weighted ERM oracle over pairs of a group and a hypothesis, where $d_G$ and $d_H$ are the VC dimensions of the group and hypothesis classes and $\varepsilon$ bounds the excess risk on every group, weighted by its probability. We also show a matching lower bound of $Ω((d_G+d_H)/\varepsilon^2)$ samples for every learner even with unlimited computation, so oracle efficiency costs no additional samples. We also consider a natural special case in which every input belongs to at most $k$ groups and these groups can be listed. In this case, an ERM oracle for the hypothesis class alone suffices, with $\tilde{O}((d_H+\log k)/\varepsilon^2)$ samples in the agnostic setting and $\tilde{O}((d_H+\log k)/\varepsilon)$ samples when every group has a hypothesis that is correct on it.
Primary: Harvard University
All Institutions: Harvard University
The paper resolves a major open problem in multi-group learning by demonstrating that optimal sample complexity can be achieved with polynomially many calls to a weighted ERM oracle, matching the theoretical lower bound and eliminating the previous statistical penalty for computational efficiency.
The paper addresses a fundamental gap in multi-group learning: achieving optimal sample complexity ($\tilde{O}((d_G+d_H)/\varepsilon^2)$) while remaining computationally efficient via an ERM oracle. Previous optimal algorithms required tracking weights for every group-hypothesis pair, which is infeasible for large classes, while oracle-based algorithms suffered from worse sample rates (typically $\varepsilon^{-3}$). The authors propose a novel reduction of the batch multi-group problem to a hybrid online learning setting. They introduce a "pair class" of functions that encode the joint output of a group and a hypothesis, allowing the use of Gaussian Follow-the-Perturbed-Leader (GFTPL) with a weighted ERM oracle. A key technical contribution is the "voting rule" that allows the learner to predict using the pair player's distribution without explicitly solving the joint optimization problem at each step. The paper also provides a rigorous lower bound showing that this sample complexity is optimal for any learner, even with unlimited computation, proving that oracle efficiency comes at no statistical cost. Additionally, they identify a "bounded membership" condition under which a simpler ERM oracle over the hypothesis class alone suffices, yielding improved rates in the realizable case.
This is a pure theory paper. There are no empirical experiments, datasets, or benchmarks. The "evaluation" consists of rigorous mathematical proofs of upper and lower bounds on sample complexity and oracle calls. The results are validated through reduction to classical agnostic learning lower bounds and analysis of the online-to-batch conversion.
The paper provides full proofs in the appendix (implied by the structure, though the provided text is truncated). The algorithms are described in pseudocode (Algorithm 1, 2, 3). Since the contribution is theoretical, "reproducibility" refers to the verifiability of the proofs. The logical flow is clear, and the reliance on standard tools (VC dimension, Assouad's lemma, online learning regret bounds) makes the results accessible to experts in learning theory.
The main limitation is the reliance on an ERM oracle over the *pair* class $(g,h)$. The paper proves that this joint oracle cannot generally be implemented from separate oracles for $G$ and $H$ without solving an NP-hard problem (Max-Cut reduction). While they provide a workaround for the "bounded membership" case, the general case still requires an oracle that can jointly optimize over groups and hypotheses, which may be computationally expensive or unavailable in practice for complex group structures. The deterministic predictor result requires a computable total order on the input space, which is a strong assumption not always met.
This work significantly advances the theory of fair and robust learning. Multi-group learning is a core component of subgroup fairness and robustness guarantees. By closing the gap between statistical optimality and computational efficiency, this paper provides a blueprint for designing practical algorithms that offer strong fairness guarantees without prohibitive computational costs. It bridges the gap between online learning theory and batch learning guarantees, offering new techniques (hybrid online-to-batch conversion with oracles) that could be applied to other structured prediction problems. The paper resolves a major open problem in multi-group learning by demonstrating that optimal sample complexity can be achieved with polynomially many calls to a weighted ERM oracle, matching the theoretical lower bound and eliminating the previous statistical penalty for computational efficiency.
Reinforcement learning (RL) for large language model (LLM) agents remains challenging in long-horizon tasks with sparse outcome rewards, where assigning credit to individual actions is difficult. Methods such as GRPO assign the same trajectory-level advantage to all actions, while critic-based methods require learning an additional value function that can be costly and unstable. We introduce LLM Value Function RL (LVRL), which uses the pretrained knowledge and in-context learning ability of an LLM to estimate policy-aware state values without finetuning a separate critic. LVRL conditions the value model on group context and calibrates its predictions using known values at the initial and terminal states, enabling step-level credit assignment from estimated value differences. We further extend this idea to learning from an expert policy through on-policy distillation (OPD), introducing LLM Value Function OPD (LVOPD), which focuses teacher supervision on actions associated with decreases in estimated value. We evaluate our approach across diverse agentic benchmarks: AppWorld, BabyAI, and SWE-bench Verified. Across policy scales (4B to 32B), LVRL consistently and substantially outperforms both GRPO and PPO. On AppWorld and SWE-bench Verified, LVRL yields absolute improvements of up to 13.6 and 8.6 percentage points over GRPO. Notably, on tasks requiring extended exploration like BabyAI, LVRL bridges the gap from total failure to 75.0% success. Furthermore, LVOPD yields consistent improvements over standard distillation baselines on AppWorld. Finally, Monte-Carlo analysis confirms that our frozen critic accurately identifies semantic pitfalls and recoveries, proving that LVRL extracts robust, policy-aware step-level credit from sparse terminal outcomes.
Primary: Apple
All Institutions: Apple
The paper introduces LVRL, a method using LLMs as value functions for credit assignment in agentic RL, achieving substantial improvements over standard RL baselines. The technical contribution is significant, offering a novel and effective way to leverage pretrained LLM capabilities for reinforcement learning, with strong empirical validation across diverse benchmarks and model scales.
The paper proposes LVRL, a method that leverages the in-context learning capabilities of a pretrained LLM to estimate state values for credit assignment in agentic RL. By conditioning the value model on group context and calibrating predictions using known boundary values (initial and terminal states), the method avoids the instability and cost of training a separate critic network. The extension to on-policy distillation (LVOPD) is a logical and effective application of this value estimation to focus teacher supervision on critical decision points. The methodology is sound, addressing a core bottleneck in LLM agent training (sparse rewards) with a resource-efficient approach that utilizes existing model capabilities.
The evaluation is comprehensive, covering diverse agentic benchmarks (AppWorld, BabyAI, SWE-bench Verified) and multiple model scales (4B to 32B). The reported improvements over strong baselines like GRPO and PPO are substantial, particularly the transition from total failure to 75% success on BabyAI, which demonstrates the method's effectiveness in long-horizon exploration tasks. The Monte-Carlo analysis providing qualitative evidence of semantic understanding in the value estimates adds depth to the quantitative results.
The paper appears to be a preprint (arXiv) with a submission to ICLR 2027. While the methodology is clearly described, the lack of a provided code repository or project URL in the extracted text limits immediate reproducibility. However, the reliance on standard LLM APIs and RL frameworks suggests that implementation is feasible for practitioners with access to similar models.
The primary limitation is the computational cost of using a large LLM as a value estimator, which may be prohibitive for very large-scale deployments or real-time applications. Additionally, the performance gains are dependent on the quality of the base LLM's in-context learning abilities, which may vary across different model families. The paper does not extensively discuss the failure modes of the value estimation process when the LLM's prior knowledge conflicts with the specific task dynamics.
This work has significant implications for the field of LLM agents, offering a scalable and effective solution to the credit assignment problem. By demonstrating that pretrained LLMs can serve as effective value estimators without fine-tuning, it opens new avenues for integrating RL with LLMs more efficiently. The approach could be widely adopted in developing autonomous agents for software engineering, web navigation, and other complex interactive tasks. The paper introduces LVRL, a method using LLMs as value functions for credit assignment in agentic RL, achieving substantial improvements over standard RL baselines. The technical contribution is significant, offering a novel and effective way to leverage pretrained LLM capabilities for reinforcement learning, with strong empirical validation across diverse benchmarks and model scales.
Generating an image with a diffusion or flow-matching model typically involves integrating a velocity field for hundreds of sequential steps. That computation suits a GPU poorly and an analog neuromorphic substrate well, where one fast, low-power physical relaxation could replace the whole sequence. Two things are needed: a generative algorithm whose computation is a relaxation, and a way to train it without backpropagation, whose global gradient transport is hard to realise in analog. Energy Matching supplies the first, sampling by gradient-descent relaxation on a time-independent potential - but obtaining the gradient that descent follows is itself a backward pass, once per integration step. We prove that a predictive coding network with a single linear injection term added to its energy relaxes to a state whose visible layer carries that gradient inherently, with an error that vanishes for small injection strengths. An Energy Matching model trained by backpropagation can therefore be loaded into this network unchanged and sampled by relaxation alone, matching the original model's FID to within 0.04 at CIFAR-10 and ImageNet-32 scale. The network can also be trained without backpropagation: Equilibrium Propagation reproduces backpropagation's parameter updates to a cosine similarity of 0.9998, and training end to end with those updates reaches an FID of 3.64 on unconditional CIFAR-10 against 3.54 for backpropagation. To our knowledge this is the first CIFAR-10 generative model trained with Equilibrium Propagation, and the first at competitive quality by any backpropagation-free rule. This quality does not require group normalisation or attention: an architecture of only convolutions and pointwise nonlinearities, both native to an analog crossbar, reaches an FID of 3.51 when trained by backpropagation.
Primary: University of Cambridge
All Institutions: University of Cambridge, Nokia Bell Labs
The paper presents a rigorous method for training CIFAR-scale generative models using Equilibrium Propagation, achieving competitive FID scores without backpropagation. It bridges the gap between energy-based generative models and neuromorphic hardware by proving that predictive coding networks can inherently compute the gradients required for sampling and training via local relaxation dynamics.
The paper proposes a rigorous theoretical and practical framework for training generative models without backpropagation, specifically targeting analog neuromorphic hardware. The core contribution is the "PotentialPCN," a predictive coding network (PCN) constructed from a standard feedforward energy matching model. The authors prove that by adding a single linear injection term to the energy of the PCN, the relaxed state of the network inherently computes the gradient of the potential (the velocity field required for sampling) with an error that vanishes as the injection strength approaches zero. This allows for sampling via physical relaxation without a backward pass. Furthermore, they demonstrate that Equilibrium Propagation (specifically a GradEP variant with a linearized nudge) can train these models end-to-end, reproducing backpropagation gradients with high fidelity (cosine similarity 0.9998). The methodology is mathematically sound, with clear proofs provided in the appendix regarding the correspondence between the PCN relaxation and the feedforward gradient.
The experiments are comprehensive for the scope of the claim. The authors validate the inference correspondence by showing that the PCN relaxation matches the feedforward gradient to machine precision (TF32 limits) across six decades of injection strengths. They demonstrate that an Energy Matching model trained with backprop can be sampled via PCN relaxation with negligible FID loss (within 0.04) on CIFAR-10 and ImageNet-32. Crucially, they show that training *with* Equilibrium Propagation yields competitive results: an FID of 3.64 on unconditional CIFAR-10, compared to 3.54 for backpropagation. They also provide an existence proof for hardware-plausible architectures (convolutions and pointwise nonlinearities only) achieving an FID of 3.51, demonstrating that attention and group normalization are not strictly necessary for competitive performance at this scale.
The paper provides a high level of reproducibility. The authors release code for training, sampling, and correspondence measurements. The training recipe, including hyperparameters for the Equilibrium Propagation phases (nudge strength, tether stiffness, sweep budgets), is detailed in the appendix. The distinction between the "implicit solve" (IFT) and "Equilibrium Propagation" (EP) arms is clearly defined, allowing readers to replicate the specific backprop-free training procedure.
The primary limitation is that all results are simulated on GPUs; no physical neuromorphic hardware was used, so the actual energy and latency benefits remain theoretical. The simulation cost of EP is significantly higher than backprop (16x), which is a major barrier for GPU-based research, though the paper argues this is mitigated on physical hardware. The scope is limited to CIFAR-10 for training; ImageNet-32 was only used for inference validation. Additionally, the "hardware-plausible" architecture was only trained with backprop, not with EP, leaving a gap in demonstrating the full pipeline on the simplified architecture.
This work is significant for the neuromorphic computing community as it provides a concrete, high-quality pathway to implement generative models on analog substrates. By removing the need for backpropagation during both training and inference (sampling), it addresses the two main hurdles for analog AI: global gradient transport and sequential computation. The finding that simple convolutional architectures suffice for competitive FID scores without attention is also valuable for the broader generative modeling field, suggesting that complexity may be overestimated for certain tasks. The paper presents a rigorous method for training CIFAR-scale generative models using Equilibrium Propagation, achieving competitive FID scores without backpropagation. It bridges the gap between energy-based generative models and neuromorphic hardware by proving that predictive coding networks can inherently compute the gradients required for sampling and training via local relaxation dynamics.
Advances in large language models (LLMs) have brought the vision of a general-purpose personal assistant closer to reality. Such assistants could provide a unified natural-language interface to a broad ecosystem of applications. However, real-world applications do not readily support training at scale because access is often restricted and their state is difficult to reset. To address this challenge, we introduce Terrarium, a framework that synthesizes executable App Worlds from real-world application information, generates training task packages with evaluator-side rubrics from these worlds, and iteratively validates and refines both through agent rollouts. For App World synthesis, Terrarium organizes public Web information and in-application observations into structured app specifications that guide the construction of self-contained runtimes with callable operations and mutable, resettable state. For task synthesis, Terrarium constructs tasks grounded in App World state together with evaluator-side outcome rubrics that assess success without prescribing reference trajectories. We construct a training suite of 275 App Worlds and 13,344 tasks for outcome-based reinforcement learning across three Qwen3 model scales (4B, 8B, and 14B parameters). Across these scales, models trained with Terrarium outperform the corresponding Qwen3 models by an average of 18.34 percentage points on the $τ^2$-bench and 10.94 percentage points on the AppWorld bench. These results suggest that app-grounded world and task synthesis offers a scalable path toward general-purpose assistants capable of operating across broad application ecosystems.
Primary: Fudan University
All Institutions: Beijing Tongming Lake Information Technology Application Innovation Center, Fudan University, Institute of Systems for Advanced Computing, Peking University, School of Computer Science, Simon Fraser University
[One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a scalable framework for synthesizing executable app worlds and grounded training tasks to improve LLM agent performance, demonstrating significant gains on standard benchmarks through outcome-based reinforcement learning.
The paper introduces "Terrarium," a framework for synthesizing executable "App Worlds" from real-world application information (web search + GUI traces) to train LLM agents. The core methodological contribution is the creation of a structured intermediate representation (app specification) that guides the construction of self-contained, resettable runtimes with mutable state. This addresses the key bottleneck in agent training: the lack of scalable, realistic, and controllable environments. The pipeline involves synthesizing the world, generating grounded tasks with evaluator-side rubrics (rather than reference trajectories), and using a closed-loop validation process where agent rollouts identify and repair defects in the world, task, or evaluator. The use of Group Relative Policy Optimization (GRPO) with binary outcome rewards is a solid choice for this setting, allowing the model to learn from success/failure signals without requiring dense trajectory supervision. The distinction between "Intent Execution" and "Intent Completion" (handling missing information) is a thoughtful addition that adds nuance to the training signal.
The experimental evaluation is rigorous and comprehensive. The authors train Qwen3 models at three scales (4B, 8B, 14B) and evaluate them on two independent benchmarks: $\tau^2$-bench and AppWorld. The results show consistent and significant improvements over the base models (18.34 points on $\tau^2$-bench, 10.94 on AppWorld). The ablation studies are particularly strong, demonstrating that GRPO outperforms SFT, that combining Execution and Completion supervision is beneficial, and that performance scales with the number of synthesized App Worlds. The comparison with other synthesis methods (AWM, EnvScaler) further validates the effectiveness of the app-grounded approach. The inclusion of BFCL Multi-Turn evaluation provides additional evidence of generalization.
The paper provides extensive details on the training configuration, including hyperparameters, dataset statistics, and the judge prompt. The authors state they will open-source the framework and artifacts, which is a major plus for reproducibility. The detailed description of the closed-loop validation process and the failure taxonomy helps in understanding how the data quality is maintained. However, the actual code and data are not yet available, which is a minor limitation for immediate reproducibility.
The primary limitation is the reliance on a large, proprietary model (Qwen3.5-397B-A17B) as the evaluator-side judge. This introduces a potential bias and makes the method less accessible to smaller labs. Additionally, the synthesized worlds are surrogates and may not perfectly capture the complexity and edge cases of real-world applications. The method is also computationally expensive, requiring significant GPU resources for both synthesis and training.
This work has significant potential to impact the field of LLM agents by providing a scalable path to training general-purpose assistants. The ability to synthesize realistic, controllable environments from public information could accelerate research in agent learning and enable the development of more capable and reliable AI systems. The open-sourcing of the framework and data will likely foster further innovation in this area. [One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a scalable framework for synthesizing executable app worlds and grounded training tasks to improve LLM agent performance, demonstrating significant gains on standard benchmarks through outcome-based reinforcement learning.
Terminal agents solve tasks in software engineering and scientific research by executing commands in a terminal. Training them via reinforcement learning (RL) requires diverse and verifiable environments, but these environments are still mostly designed by human researchers, so their supply grows only as fast as researchers can work. We introduce AutoEnvScaling, which treats environment design as a terminal task and uses agents to automate the data flywheel. A proposer agent builds new RL environments from the rollouts of the solver being trained, and the solver's new rollouts guide the next round. By turning environment generation into a terminal task, AutoEnvScaling increases the valid-environment rate by up to 25 times compared to prompt-only generation, at up to 75% lower cost per valid environment. We show that training Qwen3.5-35B-A3B on AutoEnvScaling environments from GPT-5.6-sol improves its Terminal-Bench 2.1 score by 4.3 points over the matched Tmax baseline. The broader goal of AutoEnvScaling is to enable recursive self-improvement (RSI): by making both environment generation and solving terminal tasks, it lets one model co-evolve in both roles. We show that Qwen3.6-35B-A3B, acting in both roles, gains 12.2 points on Terminal-Bench 2.1 and raises its Terminal-Bench 4.0 score from 1.6 to 3.2. The pipeline also generalizes across agentic domains, generating environments for software engineering, computer use, professional work, and GPU kernel programming. Overall, these results show a path toward agents building their own training environments, reducing human reliance and taking a core step toward RSI.
Primary: Northeastern University
All Institutions: Northeastern University, Microsoft Research
AutoEnvScaling automates the generation of RL training environments by treating environment design as a terminal agent task, enabling a recursive self-improvement loop where agents co-evolve as both task designers and solvers. The paper demonstrates that this approach significantly increases the rate of valid environment generation and improves downstream agent performance on terminal benchmarks, offering a scalable path toward reducing human reliance in agent training pipelines.
The paper proposes AutoEnvScaling, a framework that reframes the generation of RL training environments as a terminal agent task. Instead of using static pipelines or simple prompt-based generation, the method employs a "proposer" agent operating within a sandboxed terminal environment. This agent reads solver rollouts, identifies failure modes, and iteratively builds, tests, and calibrates new Harbor tasks (instructions, environments, reference solutions, tests). The core innovation is the "data flywheel" where the solver's performance directly informs the proposer's next environment generation, and the proposer itself can be trained (via DPPO) to improve at environment design. This creates a recursive self-improvement (RSI) loop where a single model can co-evolve as both the task designer and the solver. The methodology is technically sound, leveraging existing RL techniques (DPPO) but applying them to a novel multi-agent, self-play context with distinct reward structures for design vs. solving.
The experiments are extensive and well-controlled. The authors compare their method against prompt-only and feedback-driven baselines across eight frontier models, showing a 25x increase in valid environment rates. They demonstrate downstream utility by training Qwen3.5 models on AutoEnvScaling-generated environments, showing significant gains (4.3-7.4 points) on Terminal-Bench 2.1 over static Tmax baselines. The RSI experiment is particularly compelling, showing a 12.2-point gain when the model acts as both proposer and solver. The inclusion of a qualitative study on "learning behavior" (asking for help) further validates the quality of the generated curricula. The use of matched baselines (regenerating Tmax pools with the same proposer) ensures fair comparison.
The paper provides detailed descriptions of the Harbor framework, the sandbox setup, and the validation/calibration criteria. However, specific hyperparameters for the DPPO training and the exact implementation details of the "harness optimization" are relegated to appendices or referenced generally. The reliance on proprietary frontier models (GPT-5.6, Claude Opus 5) for the proposer role limits immediate reproducibility for those without access, though the open-weight Qwen experiments provide a path for replication. The code and environment definitions are not explicitly linked in the provided text, which is a minor gap.
The primary limitation is the heavy reliance on strong frontier models to bootstrap the proposer role, which may not be accessible to all researchers. The RSI results, while impressive, are demonstrated on a specific model family (Qwen) and benchmark (Terminal-Bench), leaving open questions about generalization to other domains or model architectures. The paper also acknowledges that the validation checks cannot catch all types of errors (e.g., shared mistakes between reference solution and tests), which could lead to subtle reward hacking if not carefully monitored.
This work has significant implications for the scalability of RL training for agents. By automating the most labor-intensive part of the RL pipeline (environment design), it could drastically reduce the cost and time required to train capable agents. The concept of RSI through self-generated curricula is a major step toward autonomous AI improvement, with potential applications in scientific discovery, software engineering, and other complex problem-solving domains. It shifts the paradigm from human-designed curricula to agent-designed curricula, potentially enabling agents to discover novel skills and strategies that humans might not anticipate. AutoEnvScaling automates the generation of RL training environments by treating environment design as a terminal agent task, enabling a recursive self-improvement loop where agents co-evolve as both task designers and solvers. The paper demonstrates that this approach significantly increases the rate of valid environment generation and improves downstream agent performance on terminal benchmarks, offering a scalable path toward reducing human reliance in agent training pipelines.
A predictor is multicalibrated with respect to a class of subpopulations (formally, of functions on the inputs, called auditors) if, for every subpopulation in the class and every value the predictor outputs, the average outcome among the members of that subpopulation who receive that value is close to the value itself. For finite classes, about $\varepsilon^{-3}$ samples are necessary and sufficient to learn a predictor with multicalibration error $\varepsilon$, but the algorithms that attain this rate enumerate the class, which is impossible for the large classes of interest. We ask how many samples suffice when the learner accesses the class only through an empirical risk minimization (ERM) oracle. We give an algorithm that uses $\tilde{O}(d/\varepsilon^{3})$ samples and polynomially many ERM calls, where $d$ is the VC dimension of the class. Its dependence on $\varepsilon$ is optimal for all learners. When $d$ is at least about $1/\varepsilon$, every learner that can only access the class through polynomially many ERM calls needs $\tildeΩ(d/\varepsilon^{3})$ samples. Learners with unlimited computation need only $\tilde{O}(\varepsilon^{-3}+d\varepsilon^{-2})$ samples, so the extra factor $1/\varepsilon$ on the dimension term is the price of oracle efficiency. For swap multicalibration, which allows a different function from the class for each predicted value, $\tildeΘ(d/\varepsilon^{3})$ samples are necessary and sufficient. For estimating the error of a given predictor, the ordinary notion is the harder one: the swap error can be estimated with an ERM oracle, whereas the ordinary error is NP-hard to approximate even for classes with linear-time ERM. Consequently, there are classes for which it is computationally easier to learn a multicalibrated predictor than to compute the multicalibration error of a given predictor.
Primary: Harvard University
All Institutions: Harvard University
The paper resolves key open questions regarding the sample complexity of multicalibration under oracle efficiency constraints, providing tight upper and lower bounds that define the theoretical limits of fair learning algorithms. It introduces novel techniques combining Hadamard tests with hybrid online learning to achieve optimal rates, and demonstrates the computational hardness of auditing multicalibration error, offering profound insights into the trade-offs between statistical efficiency and computational feasibility in fairness-aware machine learning.
The paper presents a rigorous theoretical framework for understanding the sample complexity of multicalibration when the learner is restricted to using an Empirical Risk Minimization (ERM) oracle. The core methodological contribution is the development of an algorithm that achieves the optimal $\tilde{O}(d/\varepsilon^3)$ sample complexity for ordinary multicalibration, where $d$ is the VC dimension. This is achieved by replacing the exponential number of sign patterns required in previous enumeration-based approaches with a polynomial number of "signed tests" derived from Hadamard matrices. The authors cleverly combine this with a high-probability regret bound for Gaussian Follow-the-Perturbed-Leader (FTPL) in a hybrid online learning setting (random inputs, adversarial feedback). The paper also provides matching lower bounds, proving that this sample complexity is optimal for any learner making polynomially many ERM calls, thereby establishing a clear separation between oracle-efficient learning and enumeration-based learning. The extension to swap multicalibration and the analysis of error estimation (showing NP-hardness for ordinary error estimation) are also technically sound and well-supported.
As a pure theory paper, there are no empirical experiments. The "evaluation" consists of rigorous mathematical proofs of upper and lower bounds on sample complexity. The results are definitive within the theoretical model defined (VC dimension, ERM oracle access, distribution-free setting).
The paper is fully reproducible in the sense that the algorithms are described in sufficient detail for implementation, and the theoretical guarantees are proven. The algorithms rely on standard oracles (ERM) which are well-defined. The deterministic predictor construction requires a computable total order on the input space, which is a standard assumption in such contexts.
The results are theoretical and do not provide empirical validation on real-world datasets. The lower bounds rely on the assumption that the learner accesses the class only through polynomially many ERM calls; if a learner had unlimited computational power to enumerate the class (if finite), better rates are possible, but this is often infeasible for large classes. The NP-hardness result for error estimation applies to the worst-case scenario for specific classes, which may not reflect typical practical instances.
This paper significantly advances the understanding of the computational and statistical limits of fair machine learning. By establishing the optimal sample complexity for oracle-efficient multicalibration, it provides a benchmark for future algorithmic developments. The separation between learning and error estimation has important implications for the auditability of ML models, suggesting that verifying a model's fairness may be computationally harder than training a fair model. This insight is crucial for regulatory frameworks and practical deployment of fair AI systems. The paper resolves key open questions regarding the sample complexity of multicalibration under oracle efficiency constraints, providing tight upper and lower bounds that define the theoretical limits of fair learning algorithms. It introduces novel techniques combining Hadamard tests with hybrid online learning to achieve optimal rates, and demonstrates the computational hardness of auditing multicalibration error, offering profound insights into the trade-offs between statistical efficiency and computational feasibility in fairness-aware machine learning.
Generative Recommenders increasingly use Reinforcement Learning (RL) to optimize outcomes that may appear hundreds or thousands of interactions after a decision. Standard RL propagates reward according to temporal proximity rather than explicit relevance, attenuating the signal assigned to distant but important actions and creating a source of sample inefficiency. We introduce Explanatory World Models (EWM), an attribution framework that uses explanations to identify the earlier events that plausibly contributed to a realized outcome. We instantiate EWM with a frontier language model and use its explanations as learning signals using a conservative hierarchical operator that recursively redistributes reward while preserving the undiscounted episode return. To investigate what constitutes a useful explanation, we operationalize three complementary variants inspired by Halpern's actual causation, Lombrozo's explanatory simplicity, and Deutsch's hard-to-vary explanations. We evaluate EWM on two controlled long-horizon RL benchmarks, a public sequential-recommendation benchmark, and an industry-scale generative recommender. On Crafter and WordCraft, EWM substantially outperforms PPO and LaRe, with its largest gains when rewards are most delayed. On KuaiRand, EWM outperforms SFT and LaRe offline and A2C online. Finally, in an offline evaluation of an industry generative recommender, EWM considerably improves Recall and increases behavioral proxies. These results suggest that explanations can provide structured and reusable credit for learning from delayed outcomes.
Primary: University of Cambridge
All Institutions: University of Cambridge
The paper proposes Explanatory World Models (EWM) to improve credit assignment in RL by using LLM-generated causal explanations to redistribute delayed rewards. This approach significantly outperforms standard RL baselines in long-horizon tasks and industrial recommendation settings, demonstrating the potential of integrating causal reasoning with reinforcement learning to overcome sample inefficiency.
The paper introduces Explanatory World Models (EWM), a framework that leverages Large Language Models (LLMs) to generate causal explanations for delayed rewards in Reinforcement Learning (RL). The core innovation is the use of these natural language explanations to identify non-temporal dependencies between actions and outcomes, thereby improving credit assignment. The method employs a "conservative hierarchical operator" to redistribute reward based on the LLM's identified causal links while preserving the undiscounted episode return. The theoretical grounding draws from Halpern's actual causation, Lombrozo's explanatory simplicity, and Deutsch's hard-to-vary explanations, providing a robust conceptual foundation for what constitutes a "useful" explanation in this context. The integration of LLMs as a credit assignment mechanism rather than just a policy or value estimator is a significant methodological shift.
The evaluation is comprehensive, spanning controlled long-horizon benchmarks (Crafter, WordCraft), a public sequential recommendation dataset (KuaiRand), and an industry-scale generative recommender. The results show substantial outperformance of PPO and LaRe, particularly in scenarios with high reward delay, which is the primary pain point the paper addresses. The inclusion of an industry-scale offline evaluation adds significant practical relevance, demonstrating that the method scales beyond toy problems. The gains in Recall and behavioral proxies on the industrial dataset are strong indicators of real-world utility.
The paper provides detailed descriptions of the LLM variants and the reward redistribution operator. However, the specific prompts used to elicit explanations from the "frontier language model" and the exact hyperparameters for the conservative operator are critical for reproducibility. While the appendix is referenced, the main text suggests a level of detail that should allow for replication if the code is released (which is not provided in the text snippet). The use of a "frontier" model implies potential dependency on proprietary APIs, which could hinder open reproduction.
The reliance on LLMs introduces inference costs and latency, which may be prohibitive for real-time online RL in some settings. The method's effectiveness depends heavily on the quality of the LLM's causal reasoning, which can be inconsistent. Furthermore, the "conservative" nature of the reward redistribution might limit the magnitude of improvement compared to more aggressive credit assignment strategies. The paper does not extensively discuss the failure modes of the LLM explanations or how the system handles incorrect causal attributions.
This work has significant implications for the field of RL, particularly for applications involving sparse or delayed rewards such as robotics, long-horizon planning, and recommendation systems. By bridging the gap between symbolic causal reasoning (via LLMs) and statistical learning (RL), it offers a pathway to more sample-efficient agents. The findings could inspire further research into hybrid neuro-symbolic approaches for credit assignment and the use of generative models as world models for causal inference. The paper proposes Explanatory World Models (EWM) to improve credit assignment in RL by using LLM-generated causal explanations to redistribute delayed rewards. This approach significantly outperforms standard RL baselines in long-horizon tasks and industrial recommendation settings, demonstrating the potential of integrating causal reasoning with reinforcement learning to overcome sample inefficiency.
Reinforcement learning (RL) has emerged as a key paradigm for improving large language model (LLM) agents, enabling stronger reasoning, planning, and tool-use in interactive environments. However, existing agent RL methods remain fundamentally constrained by an on-policy reasoning bottleneck: when tasks exceed the model's current capability boundary, effective trajectories become extremely sparse and training often collapses. We introduce Rollout-Time Intervention (RTI), a technique designed to elicit successful trajectories beyond the policy's intrinsic exploration capacity. RTI operates by leveraging reward-side evidence (i.e., evaluation rubrics) during rollout to convert erroneous trajectories into successful ones, providing effective learning signals for tasks that the policy rarely solves on its own. Although intervention trajectories are inherently off-policy, we find that they can effectively guide policy optimization when trajectories with excessive distributional shifts are filtered out. Extensive experiments across five benchmarks demonstrate that RTI consistently outperforms RL baselines, enabling the post-trained policy to approach the performance of substantially stronger contemporary LLMs and, more importantly, transcend its original capability boundary. The code is available at https://github.com/Alibaba-NLP/qqr.
Primary: Alibaba Group
All Institutions: Alibaba Group, Alibaba Group Amap, Alibaba Group HKUST (Guangzhou)
The paper introduces Rollout-Time Intervention (RTI), a technique that leverages reward-side evidence to correct failed agent trajectories during rollout, enabling LLMs to learn beyond their intrinsic exploration limits. By combining a multi-agent correction pipeline with distribution gating and token reweighting, RTI effectively mitigates the on-policy bottleneck in reinforcement learning, demonstrating substantial improvements in complex agentic tasks and providing a robust framework for training agents on tasks where successful trajectories are initially sparse.
The paper proposes Rollout-Time Intervention (RTI), a method to address the "on-policy bottleneck" in RL training for LLM agents. The core idea is to use a multi-agent system (Validator-Corrector-Verifier) to repair failed trajectories during the rollout phase, effectively creating successful off-policy examples. The methodology is technically sound, addressing key challenges of off-policy learning: (1) preventing reward hacking/leakage via the Verifier, (2) managing distribution shift via perplexity-based gating, and (3) improving credit assignment via reasoning-shift-aware token reweighting. The use of self-intervention (using the policy model itself for correction) is a clever and resource-efficient design choice, though it relies on the model having sufficient latent capability to correct its own errors when guided by rubrics.
The evaluation is extensive, covering five benchmarks including a novel Navigation benchmark and the challenging Claw-Eval-Hard20. The results show consistent improvements over strong baselines like GRPO and teacher-guided methods (LUFFY, OPD). The ablation studies are rigorous, clearly isolating the contribution of each component (Post-Check, PPL Gating, Reweighting). The analysis of training dynamics (showing that the policy eventually succeeds without intervention) is particularly convincing evidence that the method teaches the model rather than just hacking the reward.
The authors provide a GitHub link and state that comprehensive details, including prompts and hyperparameters, are in the appendix. The use of standard benchmarks (tau2-bench) and open-source backbones (Qwen3) enhances reproducibility. However, the specific implementation of the MAS prompts and the "Claw-Eval" benchmark details might require careful reading of the appendix to fully replicate.
The method relies heavily on the availability of "reward-side evidence" or evaluation rubrics, which may not be available for all agentic tasks. The perplexity gating threshold requires tuning. Additionally, the method is primarily tested on reasoning and tool-use tasks; its applicability to other types of agent interactions (e.g., physical robotics) is not explored. The reliance on self-intervention means that if the model lacks the fundamental knowledge to correct the error, the intervention may fail, though the paper argues this is less of an issue than with external teachers.
This work has significant implications for the training of LLM agents, offering a practical way to overcome the sparsity of rewards in complex environments. By enabling models to learn from corrected failures, it could accelerate the development of more capable autonomous agents. The technique is modular and could potentially be integrated into other RL frameworks beyond GRPO. The paper introduces Rollout-Time Intervention (RTI), a technique that leverages reward-side evidence to correct failed agent trajectories during rollout, enabling LLMs to learn beyond their intrinsic exploration limits. By combining a multi-agent correction pipeline with distribution gating and token reweighting, RTI effectively mitigates the on-policy bottleneck in reinforcement learning, demonstrating substantial improvements in complex agentic tasks and providing a robust framework for training agents on tasks where successful trajectories are initially sparse.
In-context-learning Tabular Foundation Models (TFMs) are uniquely suited to scenarios with limited resources or privacy constraints, as they solve new tasks without additional training. However, their architecture consists of multiple transformer blocks, resulting in high computational and memory footprints. Meanwhile, traditional alternatives like Gradient-Boosted Decision Trees (GBDTs) remain highly competitive and run efficiently on CPUs, suggesting that tabular tasks may not inherently require deep architectures. This contrast raises a fundamental question: can TFMs be simplified by removing architectural redundancies without sacrificing performance? To answer this, we introduce a closed-form, label-free affine translator that replaces a contiguous window of up to 96% of the blocks, leaving the rest unchanged. Evaluating eight frontier TFMs on 171 classification datasets, we reveal that tabular transformers are vastly overparameterized: replacing half of a model's blocks costs at most $0.002$ median AUROC across all architectures, with the half-depth TabFM still outperforming every other full-depth TFMs and tuned GBDTs. Furthermore, different model families are overparameterized in different ways: TabPFN-3 relies on its encoders, maintaining performance regardless of which single block is retained. Other single-axis models spread computation thinly, needing only their first block, while dual-axis models concentrate it in one critical block. Ultimately, removing depth saves up to 1.5B parameters, speeds up GPU inference by up to $6\times$ at a median cost of $0.03$ AUROC, while a single-block TabPFN-3 serves cached queries $2.4\times$ faster using $6\times$ less memory at negligible cost.
Primary: University of Oxford
All Institutions: University of Oxford
The paper demonstrates that Tabular Foundation Models are effectively shallow by showing that up to 96% of their transformer blocks can be replaced with a simple, label-free affine map with negligible performance loss, revealing significant architectural redundancy and enabling substantial efficiency gains.
The paper introduces a "closed-form, label-free affine translator" to replace contiguous windows of transformer blocks in Tabular Foundation Models (TFMs). The method fits a linear map between the hidden states of the start and end of the bypassed window using ridge regression on an unlabeled calibration set. The window selection is guided by minimizing the forward KL divergence between the full model's output distribution and the compressed model's output distribution. The methodology is clever in that it requires no gradient updates, no labeled data, and preserves the original classification head, allowing for a direct comparison of output probabilities. The per-feature parameterization for dual-axis models is a necessary and well-motivated extension.
The evaluation is extensive, covering 8 frontier TFMs from 6 families across 171 classification datasets (including clinical and standard benchmarks). The results show that replacing 50% of the blocks costs negligible performance (median AUROC drop <= 0.002), and even reducing to a single block retains competitive performance for many models. The ablation study comparing the affine translator to naive layer dropping (identity map) is crucial, showing that the blocks are not inactive but their computation is linearly absorbable. The comparison with GBDTs and the efficiency analysis (speedup/memory reduction) further solidify the practical relevance.
The paper provides a GitHub link to the code. The experimental setup is detailed, including dataset splits, hyperparameter tuning for GBDTs, and specific model configurations. The use of standard benchmarks and clear metrics (AUROC, R^2, latency, memory) ensures reproducibility.
The study is limited to classification and regression tasks; generative tasks are not covered. The pre-training study is limited to a small "nanoTabPFN" model, so it is unclear if the findings scale to much larger architectures. The per-feature translator for dual-axis models can have a high parameter count on wide datasets, potentially negating some efficiency gains.
This paper has significant implications for the deployment of foundation models in resource-constrained environments (e.g., edge devices, clinical servers). It challenges the assumption that deep architectures are necessary for tabular data, suggesting that current TFMs are overparameterized. This could lead to more efficient model design and reduced computational costs for inference. The paper demonstrates that Tabular Foundation Models are effectively shallow by showing that up to 96% of their transformer blocks can be replaced with a simple, label-free affine map with negligible performance loss, revealing significant architectural redundancy and enabling substantial efficiency gains.
City-scale 3D grounding is commonly formulated as candidate ranking, which requires precomputed instance proposals and becomes inefficient in large outdoor scenes. We propose Urbex, an agentic vision-language framework that instead casts grounding as active spatial search. Urbex represents a city-scale 3D scene as an interactive multi-view environment, allowing a VLM agent to query landmarks, zoom into local regions, render oblique views, and commit a grounded bounding box. We optimize this tool-use policy with reinforcement learning on CityRefer, using smooth localization rewards and lightweight shaping rewards for efficient exploration. Experiments show that Urbex achieves the best Hit@1 and IoU on CityRefer, supports candidate-free grounding without ground-truth boxes, and is substantially faster than the strongest city-scale ranking baseline. Zero-shot results on STPLS3D-Refer further indicate improved cross-scene robustness.
Primary: Shanghai AI Laboratory
All Institutions: Shanghai AI Laboratory, Tsinghua University
The paper introduces Urbex, an agentic framework that reformulates city-scale 3D grounding as active spatial search using a VLM agent and reinforcement learning. It demonstrates superior accuracy and efficiency compared to candidate-ranking baselines, offering a scalable solution for large outdoor 3D scene understanding.
The paper proposes "Urbex," a framework that reframes city-scale 3D grounding from a static candidate ranking problem to an active spatial search problem. The core novelty lies in the design of an interactive multi-view environment where a Vision-Language Model (VLM) agent uses specific tools (Landmark Query, Zoom-in, Oblique Rendering) to navigate the scene. The use of Reinforcement Learning (specifically GRPO) to optimize this tool-use policy is a strong methodological contribution, addressing the challenge of credit assignment in multi-step agentic tasks. The reward design, combining terminal IoU with distance-based shaping and tool-level rewards, is well-motivated for sparse reward environments. The approach effectively leverages the reasoning capabilities of modern VLMs while constraining the action space to manageable geometric operations.
The experiments are rigorous and comprehensive. The authors evaluate on CityRefer (in-domain) and STPLS3D-Refer (zero-shot cross-scene), covering both "With GT" (candidate ranking) and "Without GT" (candidate-free) settings. The results demonstrate significant improvements in Hit@1 and IoU over strong baselines like CityAnchor and GPT-5.4/Qwen3.6-plus prompted baselines. The efficiency analysis is particularly compelling, showing a 10x speedup over the strongest ranking baseline by avoiding exhaustive scoring. Ablation studies clearly isolate the contributions of individual tools and reward components, providing strong evidence for the design choices. The zero-shot generalization results are a key strength, suggesting the learned policy captures reusable spatial search strategies rather than just memorizing the training distribution.
The paper provides detailed descriptions of the environment, tools, and RL objective. However, as an arXiv preprint, specific code availability is not confirmed in the text. The reliance on specific VLM backbones (Qwen, GPT) and the complexity of the RL training loop (GRPO) may pose barriers to reproduction for groups without significant compute resources. The hyperparameters for the RL training and the specific implementation of the view-to-world transforms are described but would benefit from open-source code for full verification.
The method relies on the availability of named landmarks or distinct visual features for the "Landmark Query" tool, which may limit its applicability to scenes with poor semantic labeling or highly repetitive structures. The RL training process is computationally expensive compared to supervised fine-tuning. The "Oblique Rendering" tool requires pre-computed or on-the-fly rendering, which adds latency and storage overhead, although the paper addresses this with caching strategies. The performance on "Without GT" settings, while better than baselines, still shows room for improvement in precise box localization (high area ratio variance).
This work has significant implications for embodied AI and autonomous driving, where efficient and accurate localization of objects in large-scale environments is critical. By demonstrating that agentic search can outperform and outpace exhaustive ranking, it offers a scalable paradigm for handling large 3D scenes. The framework could be extended to other tasks requiring long-range spatial reasoning, such as robot navigation or aerial inspection. The integration of RL with VLM tool-use is a trend that this paper advances meaningfully in the 3D domain. The paper introduces Urbex, an agentic framework that reformulates city-scale 3D grounding as active spatial search using a VLM agent and reinforcement learning. It demonstrates superior accuracy and efficiency compared to candidate-ranking baselines, offering a scalable solution for large outdoor 3D scene understanding.
Robotic platforms combine multi-modal sensors to operate across diverse conditions, but most dense correspondence methods are limited to RGB-RGB or a small set of modality pairs. Training a model for each modality pair scales poorly with the number of sensors and forgoes potential positive transfer across modality pairs. We introduce TartanMatch, a unified model for dense correspondence across all 25 ordered pairs among RGB, event, thermal, depth, and projected LiDAR. TartanMatch adapts an RGB-pretrained correspondence model to all modality pairs within one shared network using lightweight modality adaptation and a mixture of real and synthetic data. On zero-shot real-world benchmarks, TartanMatch reduces Endpoint Error (EPE) over the strongest baselines by 61.9% for cross-modal matching and 49.6% for same-modal matching. We further show that joint training across all modality pairs improves average matching accuracy over pair-specific training, especially on challenging cross-modal pairs. These results demonstrate that a single model achieves broad multi-modal coverage and state-of-the-art matching accuracy.
Primary: Carnegie Mellon University
All Institutions: Carnegie Mellon University, Fisk University
[One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. TartanMatch introduces a unified dense correspondence model that significantly outperforms pair-specific baselines across 25 modality pairs by leveraging RGB pre-training and joint multi-modal training, demonstrating that a single network can effectively handle diverse sensor combinations with superior accuracy and scalability.
The paper proposes TartanMatch, a unified framework for dense correspondence across five distinct sensing modalities: RGB, event, thermal, depth, and projected LiDAR. The core methodological contribution is the adaptation of an RGB-pretrained correspondence model (specifically UFM) to handle all 25 ordered modality pairs within a single shared network. This is achieved through lightweight modality adaptation modules and a training strategy that leverages a mixture of real and synthetic data. The approach addresses the scalability issue of training separate models for each modality pair, arguing that joint training facilitates positive transfer, particularly for challenging cross-modal pairs. The use of a pre-trained backbone for RGB and adapting it to other modalities is a sound architectural choice that leverages the rich features learned from large-scale RGB datasets.
The experimental evaluation is extensive, covering zero-shot real-world benchmarks. The paper reports significant improvements in Endpoint Error (EPE), claiming a 61.9% reduction for cross-modal matching and 49.6% for same-modal matching compared to the strongest baselines. The radar chart visualization effectively demonstrates the model's superiority across various modality pairs. The claim that joint training improves average matching accuracy over pair-specific training is a key empirical finding that supports the unified model hypothesis. The inclusion of both real and synthetic data in the training mix is well-justified by the results, showing robustness to domain shifts.
The paper provides a project page (https://tartanmatch.github.io/) which likely contains code and additional resources. The use of standard benchmarks and clear metrics (EPE) enhances reproducibility. However, the specific details of the "lightweight modality adaptation" modules and the exact composition of the synthetic data mixture would need to be verified in the supplementary material or code repository for full reproducibility. The reliance on a specific pre-trained model (UFM) is a potential dependency, but UFM is a known baseline.
The primary limitation is the scope of modalities covered (5 modalities, 25 pairs). While this is a significant expansion over previous work, it does not cover all possible robotic sensing modalities (e.g., radar, acoustic). The performance on zero-shot real-world data is strong, but the gap between synthetic/real training data and real-world inference might still exist for unseen environments. The computational cost of training a single model for 25 pairs versus 25 separate models is not explicitly detailed in the abstract, though the "lightweight" adaptation suggests efficiency.
This work has high potential impact in the robotics and autonomous systems fields. A unified correspondence model simplifies the perception pipeline for multi-modal robots, reducing the need for maintaining and updating multiple specialized models. The demonstrated positive transfer across modalities could inspire similar unified approaches in other multi-sensor fusion tasks. The open-source nature of the project (implied by the project page) will likely facilitate adoption by the research community. [One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. TartanMatch introduces a unified dense correspondence model that significantly outperforms pair-specific baselines across 25 modality pairs by leveraging RGB pre-training and joint multi-modal training, demonstrating that a single network can effectively handle diverse sensor combinations with superior accuracy and scalability.
Multimodal code intelligence connects the symbolic world of programs with the visual world their execution creates. Interest in this capability is burgeoning, as models are increasingly asked to produce charts, webpages, animations, and semi-structured documents. Recently, proprietary models have pushed far ahead, producing interactive artifacts even from a text description or a screenshot alone. Yet their data and training remain closed, and open progress is held back by the quantity and quality of available data and by the pipelines and methods for producing it. Quality is the harder of the two, since a sample must be correct as a program, well built, and above all visually right once rendered. No existing signal covers all three. Execution often returns just a pass or fail, code similarity does not map to the visual output, and task-specific verifiers do not generalize. To push open progress further, we start from the data side with PrismaForge, a hierarchical data pipeline centered on a synthesized rubric that decomposes these factors into checkable criteria. The same rubric serves as both filter and reward: it curates the supervised corpus, then drives reinforcement learning on visual-coding tasks. Leveraging PrismaForge for data synthesis and filtering, we build PrismaCoder-2M, the most diverse open dataset for multimodal code intelligence and the largest to date. On the model side, we train PrismaCoder-9B and PrismaCoder-35B on this data and release both. They lead the open state of the art by a wide margin and rival leading proprietary systems. Our analyses further show that the proposed rubric-based pipeline improves multimodal coding performance across model backbones. We release our code and data at https://qiushisun.github.io/PrismaCoder-Home/.
Primary: University of California, San Diego (UCSD)
All Institutions: University of California, San Diego, Meta AI
The paper introduces PrismaForge, a data pipeline using decomposed rubric synthesis to generate high-quality multimodal code data, resulting in PrismaCoder models that rival proprietary systems. This is a significant contribution to the field of multimodal code intelligence, addressing the critical need for verifiable, high-quality open data and demonstrating that data-centric approaches can close the gap with closed-source models.
The paper proposes PrismaForge, a hierarchical data pipeline that addresses the lack of comprehensive quality signals in multimodal code generation (combining program correctness, code structure, and visual fidelity). The core innovation is the "decomposed rubric synthesis," where a synthesized rubric breaks down complex visual-coding tasks into checkable criteria. This rubric serves a dual purpose: it acts as a filter to curate a high-quality supervised corpus (PrismaCoder-2M) and as a reward signal for reinforcement learning. The method effectively bridges the gap between symbolic code and visual output by creating a verifiable intermediate representation. The approach is sound and addresses a critical bottleneck in open-source multimodal code models: the scarcity of high-quality, verifiable data.
The authors train PrismaCoder-9B and PrismaCoder-35B on the synthesized data. The results claim to lead the open state of the art by a wide margin and rival proprietary systems. The evaluation likely includes standard benchmarks for code generation (e.g., HumanEval, MBPP) and multimodal-specific tasks (e.g., rendering accuracy, visual similarity). The claim of "rivaling leading proprietary systems" is significant and, if substantiated by the full text (which is summarized here), would be a major empirical finding. The analysis section reportedly shows the pipeline improves performance across different model backbones, suggesting the data quality is the key driver rather than a specific architectural quirk.
The paper releases both the code and the data (PrismaCoder-2M), which is a significant contribution to the open-source community. The release of a 2M sample dataset for multimodal code intelligence is substantial and will likely be widely used. The detailed release of the rubric synthesis pipeline further enhances reproducibility.
The reliance on synthesized rubrics may introduce biases if the rubric generation model has blind spots. The computational cost of generating and filtering 2M samples with complex visual verification is likely high. Additionally, the generalization to tasks outside the training distribution (e.g., highly abstract UIs or non-standard visualizations) may be limited.
This work significantly advances the open-source ecosystem for multimodal code intelligence. By providing a large, high-quality dataset and a robust training pipeline, it lowers the barrier for other researchers and developers to build competitive multimodal code models. It also highlights the importance of data quality and verification pipelines in the era of large language models. The paper introduces PrismaForge, a data pipeline using decomposed rubric synthesis to generate high-quality multimodal code data, resulting in PrismaCoder models that rival proprietary systems. This is a significant contribution to the field of multimodal code intelligence, addressing the critical need for verifiable, high-quality open data and demonstrating that data-centric approaches can close the gap with closed-source models.
Long-horizon agents have drawn growing attention for their ability to work autonomously -- from hours to days -- on economically valuable tasks, a capability essential to automated discovery and recursive self-improvement (RSI). Yet open-source models still struggle on long-horizon tasks, largely for lack of both training data and a working recipe, and scaling both is hard: long, complex tasks are difficult to curate and verify, and their rewards are sparse. We address both. First, we introduce Tev-Gym, the first long-horizon agent training dataset with verifiable, dense, progressive rewards -- 1,170 verified task environments across 7 domains. It is built on a latent-world-first approach that first constructs a latent world: a complete and correct hidden specification of the task domain, workflow, and expected state after each step. Each task is then progressively rendered from this specification into an agent-facing environment, enabling controllable complexity and a natural progress metric. Second, we turn this metric into a training recipe: via progress-potential reward shaping (PPRS), we convert progress into a dense per-step reward that improves training efficiency and makes otherwise-intractable tasks trainable. Training with this recipe yields Tev-27B, which achieves substantial gains on the Agents' Last Exam computing and math sciences domains (ALE-CM), matching GPT-6 Astra and Claude Opus 5 at Pass@5. Despite training only on ALE-related data, it generalizes to the Terminal-Bench (TB) 3.0 and 4.0 subsets, surpassing GLM 5.2 and Gemini 3.7 Flash, and reaches Claude Opus 4.8-level performance on middle-horizon tasks such as TB 2.1.
Primary: University of California
All Institutions: University of California
The paper introduces a latent-world-first synthesis pipeline and Progress-Potential Reward Shaping (PPRS) to enable effective RL training for long-horizon agents, resulting in open-source models that match frontier closed-source systems on complex agentic benchmarks. This is a significant contribution that addresses the critical data and reward sparsity bottlenecks in agentic AI, offering a reproducible and scalable recipe for generating high-quality training environments and training signals.
The paper introduces a "latent-world-first" synthesis pipeline for creating long-horizon agent environments. This is a significant methodological contribution because it addresses the core bottleneck in agentic RL: the lack of verifiable, dense rewards for complex, multi-step tasks. By constructing a complete, correct hidden specification (the latent world) first and then rendering the task and corruption from it, the authors ensure that ground truth is known by construction. This allows for the derivation of a "progress potential" function, which is used in a novel reward shaping scheme called Progress-Potential Reward Shaping (PPRS). PPRS is theoretically grounded in potential-based reward shaping but adapted for GRPO to prevent reward starvation in groups where no rollout succeeds. The methodology is rigorous, with clear stages for world construction, workflow graphing, artifact rendering, and controlled corruption. The inclusion of qualification gates (exploit resistance, progress validity, role independence) demonstrates a high level of care in preventing reward hacking and ensuring data quality.
The experiments are extensive and well-designed. The authors train two model sizes (9B and 27B) and evaluate them on the Agents' Last Exam (ALE-CM) and Terminal-Bench (TB) 3.0/4.0. The results show substantial gains, with the 27B model matching frontier closed-source models (GPT-6 Astra, Claude Opus 5) on Pass@5 for ALE-CM. Crucially, the paper demonstrates generalization to unseen benchmarks (TB), which is a strong indicator of learning robust agentic behaviors rather than overfitting to the training distribution. The ablation studies likely confirm the importance of both the dense reward and the specific data synthesis method. The comparison against prior open-source datasets (SWE-Smith, etc.) highlights the complexity gap, with Tev-Gym tasks being significantly longer and harder.
The paper provides detailed descriptions of the synthesis pipeline, the reward shaping formula, and the training setup (DPPO, group size, token budgets). The "latent world" concept is well-defined, and the qualification gates are described formally. However, without access to the code or the specific generators for the 1,170 environments, full reproduction is difficult. The reliance on specific base models (Qwen3.5/3.8) and the proprietary nature of some benchmark subsets (if any) could limit reproducibility for external labs. The paper is a preprint, so code release status is unknown, but the methodological detail is high.
The primary limitation is the scale of the dataset (1,170 environments) compared to massive web-scale datasets, though the quality is higher. The approach is currently focused on specific domains (software, data, math, etc.) and may not generalize to all types of long-horizon tasks (e.g., creative writing, open-ended research). The computational cost of generating and verifying these latent worlds is likely high. Additionally, the performance gains are relative to open-source baselines; while matching closed-source models is impressive, the absolute performance on the hardest tasks may still be limited by the base model's capabilities.
This work has high potential impact on the field of agentic AI. By providing a recipe for generating high-quality, verifiable long-horizon training data, it lowers the barrier for other labs to train competitive open-source agents. The PPRS technique is a generalizable insight for RL in sparse-reward settings. If the dataset and code are released, it could become a standard benchmark and training resource for the community, similar to how SWE-bench or Terminal-Bench have been used. It pushes the frontier of what open-source models can achieve in autonomous, long-duration tasks. The paper introduces a latent-world-first synthesis pipeline and Progress-Potential Reward Shaping (PPRS) to enable effective RL training for long-horizon agents, resulting in open-source models that match frontier closed-source systems on complex agentic benchmarks. This is a significant contribution that addresses the critical data and reward sparsity bottlenecks in agentic AI, offering a reproducible and scalable recipe for generating high-quality training environments and training signals.
Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fundamental bottleneck: the cost of evaluating progress. In real settings, a single true evaluation, such as a full benchmark or serving trace, can take hours, so a search can afford only a handful of true evaluations. We present DuoDiscover, a search system that jointly discovers what to optimize and how to evaluate progress efficiently, through three mechanisms. First, each worker in a fleet of coding agents co-discovers an optimization target and a fast, tailored proxy evaluator, which we call a yardstick. The agent then hill-climbs the yardstick rather than the true evaluation. Second, since search can exploit gaps between a yardstick and the true objective, fidelity tests continually recheck each yardstick's ranking and invalidate any that drift. Third, the fleet is bottom-up, with no central decomposition of the problem. A lightweight orchestrator spawns and terminates agents, briefs running ones, and seeds new agents with others' confirmed work. DuoDiscover enables autonomous optimization from software to chip architecture. We demonstrate DuoDiscover on two repository-level benchmarks and four GPU and TPU kernel suites covering hundreds of kernels. On SWE-fficiency, it doubles the average speedup of human experts. On SOL-Exec, it ranks first on 121 of 235 kernels and in the top three on 212. On the heavily optimized vLLM and SQLite engines, DuoDiscover achieves 1.92$\times$ and 1.38$\times$ gains over the base code. On a CPU microarchitecture optimization task with evaluations of up to 28 hours, DuoDiscover surpasses both AlphaEvolve-based ArchAgent v2 and the best human design, with 1.65$\times$ ArchAgent v2's gain in 16.7 days versus more than 40 days, advancing the frontier of autonomous chip design.
Primary: Google
All Institutions: Google
DuoDiscover introduces a co-discovery mechanism for optimization targets and proxy evaluators to overcome the evaluation bottleneck in autonomous search. The paper demonstrates significant improvements in code and hardware optimization tasks, establishing a new paradigm for scaling LLM-based discovery in high-cost domains.
The paper introduces DuoDiscover, a search system that addresses the "evaluation bottleneck" in autonomous code optimization. The core novelty is the "co-discovery" of optimization targets and cheap proxy evaluators ("yardsticks") by LLM agents. Unlike prior work (e.g., AlphaEvolve) that treats the evaluator as fixed or user-defined, DuoDiscover allows agents to construct their own fast metrics to guide hill-climbing, with a rigorous "fidelity test" (Kendall's tau rank correlation) to invalidate proxies that drift from the true objective. The bottom-up fleet architecture, where a lightweight orchestrator manages expensive true evaluations while agents exploit cheap proxies, is a well-motivated and technically sound approach to scaling search in high-cost domains.
The experimental evaluation is extensive and covers a wide range of domains, from repository-level Python optimization (SWE-fficiency, FormulaCode) to GPU/TPU kernel optimization (SOL-Exec, JAXBench) and CPU microarchitecture design (DPC4). The results are strong, with DuoDiscover outperforming baselines like Claude Code and AlphaEvolve-based agents. Notably, the paper reports state-of-the-art results on the SOL-Exec leaderboard and surpasses human experts on SWE-fficiency. The inclusion of a 28-hour evaluation task (CPU prefetcher) demonstrates the method's ability to handle extreme evaluation costs, a key claim of the paper.
The paper provides detailed descriptions of the fidelity test, noise estimation, and search loops. However, as is common with large-scale agentic systems, the exact prompts and internal logic of the "orchestrator" LLM are not fully disclosed, which may limit exact reproduction. The use of specific, potentially proprietary, LLM backbones (Claude Opus 5, Gemini 3.8 Flash) also poses a reproducibility challenge for external researchers without access to these models.
The method relies heavily on the ability of LLMs to construct valid proxy evaluators. If an agent cannot construct a faithful yardstick, the system falls back to expensive true evaluations, negating the efficiency gains. Additionally, the upfront cost of building and validating yardsticks may be prohibitive for very short search budgets. The paper also notes that the method is less effective when the true objective is highly non-linear or when the gap between proxy and true objective is too large to bridge via hill-climbing.
This work has significant implications for the field of autonomous software engineering and hardware design. By solving the evaluation bottleneck, it enables the application of LLM-based search to real-world engineering problems where feedback is slow and expensive. This could accelerate progress in areas like chip design, database optimization, and large-scale system tuning, potentially reducing the time and cost of manual optimization efforts. DuoDiscover introduces a co-discovery mechanism for optimization targets and proxy evaluators to overcome the evaluation bottleneck in autonomous search. The paper demonstrates significant improvements in code and hardware optimization tasks, establishing a new paradigm for scaling LLM-based discovery in high-cost domains.
Language models are often evaluated alone but increasingly deployed together, routed between, made to debate, ensembled, or collaboratively trained. For a model inside a system, what matters most is not how well it performs on its own, but how much the system gains from its presence. We call this a model's compositional strength and formalize it using the Shapley value of a cooperative game over coalitions of a model pool, and use a model's leave-one-out contribution to the full coalition as a tractable proxy, requiring n+1 rather than 2^n system evaluations. Evaluating individual vs. compositional strengths across five model pools, twelve collaboration methods, and twelve datasets, we find them largely uncorrelated: in 12 of 15 (pool, method-family) settings, individual strength does not translate into compositional advantages, and the coupling is weakest among general-purpose models that practitioners are most likely to deploy. Standalone benchmarks are therefore often poor proxies for models' value in collaborative systems. We then propose compositional reinforcement learning (C-RL), which trains a policy inside a live multi-model system and rewards the outcome produced by the system as a whole, rather than the policy's correctness in isolation. The leave-one-out baseline in Shapley values cancels under a group policy gradient, so this needs no additional rollout cost beyond RL on the collaborative system's outcome and never requires running a coalition without the policy. Across three collaboration protocols and eight tasks spanning reasoning, coding, and scientific discovery, C-RL produces the strongest collaborator in 21 of 24 settings, lifting collaborative performance by +4.0 to +5.4 pts on average (~3x the gain of baselines), up to +13.1 pts.
Primary: University of Washington
All Institutions: University of Washington
The paper makes a significant contribution by formalizing compositional strength via Shapley values and demonstrating that standard individual benchmarks are poor predictors of a model's value in collaborative systems. By proposing Compositional Reinforcement Learning (C-RL), which efficiently trains models to maximize their contribution to a multi-agent system's outcome, the authors provide a novel and effective method for developing specialized collaborator models that generalize across tasks and protocols.
The paper introduces a rigorous formalization of "compositional strength" using Shapley values from cooperative game theory, adapted for LLM collaborations. The key methodological contribution is the derivation of a tractable Leave-One-Out (LOO) estimator that approximates the Shapley value with linear complexity ($n+1$) rather than exponential ($2^n$). The proposed training method, Compositional Reinforcement Learning (C-RL), is elegant in its use of group-relative policy gradients (GRPO) to cancel out the baseline cost of the LOO marginal, allowing the policy to be trained on system-level rewards without extra rollouts. The credit assignment mechanism, which attributes system-level rewards to individual policy generations within a multi-turn collaboration, is a significant technical challenge addressed effectively.
The evaluation is extensive, covering 5 model pools, 12 collaboration methods, and 12 datasets. The finding that individual strength and compositional strength are largely uncorrelated is a strong empirical contribution. The training experiments demonstrate that C-RL outperforms baselines (including standard RL and collaborative fine-tuning) in 21/24 settings. The generalization tests to unseen datasets and collaboration protocols are convincing, showing that C-RL teaches transferable collaboration skills rather than overfitting to a specific setup. The inclusion of proprietary frontier models as collaborators adds practical relevance.
The paper promises code release upon acceptance. The use of proprietary APIs (GPT-5.5, etc.) introduces a reproducibility caveat noted by the authors, as these models may change over time. However, the open-source components (Qwen, open-weight collaborators) allow for partial reproduction. The detailed description of the MoCo toolkit and specific hyperparameters aids reproducibility.
The reliance on proprietary models for some experiments limits full reproducibility. The computational cost of running multi-model collaborations for RL training is high, potentially limiting adoption to well-resourced labs. The credit assignment is uniform across turns, which the authors acknowledge as a simplification that could be improved. The study focuses on a specific set of collaboration protocols; while generalization is tested, the scope is not exhaustive of all possible multi-agent architectures.
This work shifts the focus of LLM evaluation and training from individual capability to system-level contribution. It provides a theoretical foundation for understanding multi-agent dynamics and a practical method for training better collaborators. This is highly relevant as LLMs are increasingly deployed in compound systems. The finding that standard benchmarks do not predict collaborative value is a critical insight for practitioners building multi-agent systems. The paper makes a significant contribution by formalizing compositional strength via Shapley values and demonstrating that standard individual benchmarks are poor predictors of a model's value in collaborative systems. By proposing Compositional Reinforcement Learning (C-RL), which efficiently trains models to maximize their contribution to a multi-agent system's outcome, the authors provide a novel and effective method for developing specialized collaborator models that generalize across tasks and protocols.
Frontier AI systems are increasingly natively multimodal, jointly pretrained on multiple modalities, such as text, vision, and audio. However, the scientific understanding of the multimodal data mixtures used to pretrain these systems --- how much of each modality to mix, and at what potential interference cost to the others --- is largely absent or gatekept. In this work, we derive data mixing scaling laws for three modalities: text, vision, and audio. Multimodal Large Language Models (MLLMs) increasingly adopt Mixture-of-Experts (MoE) architectures, as MoEs scale efficiently and partially limit cross-modal interference. We therefore use MoEs to conduct 268 experiments ranging from 457M to 8.3B parameters, on up to 150 billion tokens. Our work leads to multiple actionable insights into multimodal data mixing for training MLLMs. First, compute optima are modality-specific: audio modality is model-heavy, favoring scaling parameters over tokens, while text and vision require balanced scaling of model size and data. Second, repeating vision or audio data beyond 2x yields negligible benefits. Finally, cross-modal interactions are highly asymmetric: vision strongly interferes with audio, whereas audio's interference with learning text and vision is negligible. Overall, we provide a principled foundation for understanding the multimodal data mixtures needed to train frontier MLLMs.
Primary: MILA - Quebec AI Institute
All Institutions: MILA - Quebec AI Institute, McGill University, University of Washington, EPFL
The paper derives scaling laws for multimodal data mixtures in MoE architectures, revealing modality-specific compute optima and asymmetric cross-modal interference. This comprehensive empirical study provides actionable guidelines for efficient pretraining of frontier multimodal large language models, significantly advancing the understanding of how to balance text, vision, and audio data.
The paper proposes a rigorous empirical framework for deriving scaling laws specifically for multimodal data mixtures (text, vision, audio) within Mixture-of-Experts (MoE) architectures. The methodology is sound, leveraging 268 controlled experiments across a wide parameter range (457M to 8.3B) and token counts (up to 150B). The choice to focus on MoE architectures is timely and methodologically justified, as MoEs are increasingly standard in frontier models and help isolate cross-modal interference effects. The derivation of modality-specific compute optima is a significant methodological advancement over previous unimodal scaling law studies.
The experimental scale is substantial, covering three modalities and a large parameter/token space. The findings are concrete and actionable: identifying audio as "model-heavy" (favoring parameters over tokens) and text/vision as requiring balanced scaling is a valuable empirical insight. The discovery of asymmetric cross-modal interference (vision interfering with audio, but not vice versa) is a surprising and useful result for practitioners. The validation section likely confirms these trends, though the specific metrics for "interference" need to be clearly defined in the full text to ensure robustness.
The paper provides detailed experimental setup information (model sizes, token counts, modality types). However, without access to the code or specific hyperparameter configurations for the MoE routing and data mixing ratios, full reproduction is difficult. The scale of the experiments (150B tokens) makes independent reproduction costly, limiting reproducibility to groups with similar computational resources.
The study is limited to three modalities (text, vision, audio) and MoE architectures. It may not generalize directly to dense transformers or other modalities (e.g., video, 3D). The "interference" metric is a key variable; if not rigorously defined, the conclusions about asymmetry could be sensitive to evaluation choices. The paper does not address the cost of data collection or curation for these mixtures.
This work provides a principled foundation for training next-generation multimodal foundation models. By quantifying the cost-benefit of mixing modalities, it helps labs allocate computational resources more efficiently. The insights on asymmetric interference can guide curriculum learning strategies and data balancing techniques, potentially reducing the compute required to achieve specific performance levels in MLLMs. The paper derives scaling laws for multimodal data mixtures in MoE architectures, revealing modality-specific compute optima and asymmetric cross-modal interference. This comprehensive empirical study provides actionable guidelines for efficient pretraining of frontier multimodal large language models, significantly advancing the understanding of how to balance text, vision, and audio data.
Existing navigation systems commonly rely on dedicated depth sensors or explicit depth reconstruction, while learning to generate trajectories directly from RGB images remains challenging due to the sim-to-real domain gap. In this work, we propose Stereo-Planner, an end-to-end vision-to-trajectory framework that enables zero-shot sim-to-real planning from stereo RGB observations. The key idea is to formulate trajectory generation as state- and scale-conditioned querying of latent stereo geometry. A Scene Representor incorporates a pretrained vision grounding foundation model to extract geometric cues shared across simulated and real-world environments, supporting generalization across visual domains. A lightweight Trajectory Decoder then translates these representations into trajectories through interleaved attention, separating trajectory-state interaction from scene-feature retrieval and incorporating the stereo baseline as a metric-scale cue. With these designs, Stereo-Planner is trained entirely on simulated data by minimizing differentiable trajectory costs and deployed directly in real-world environments without fine-tuning. Benchmark results demonstrate strong cross-scene generalization, with a 95% success rate in simulation environments at a commanded speed of 8 m/s, and competitive performance against depth-based planners using only stereo RGB observations for scene perception. Furthermore, deployment on a compact quadrotor equipped with an NVIDIA Orin NX enables autonomous flight at speeds of up to 7 m/s, with a mean onboard vision-to-action inference time of 24.27 ms. The software, simulation environments, and hardware designs will be open-sourced at https://github.com/Stereo-Planner/Stereo-Planner.
Primary: Nanyang Technological University
All Institutions: Nanyang Technological University, Tianjin University
Stereo-Planner enables zero-shot sim-to-real quadrotor navigation using stereo RGB by leveraging a vision grounding foundation model for geometry-aware scene representation and a scale-conditioned trajectory decoder. The paper presents a significant technical contribution by demonstrating that pretrained visual features can effectively bridge the sim-to-real domain gap for end-to-end motion planning, achieving high-speed autonomous flight without depth sensors or fine-tuning.
The paper proposes Stereo-Planner, an end-to-end framework that maps stereo RGB images directly to quadrotor trajectories. The core architectural contribution is the use of a pretrained vision grounding foundation model (Depth Anything V3 encoder) as a scene representor, which extracts geometry-aware features robust to sim-to-real appearance gaps. A lightweight trajectory decoder uses interleaved attention to query these features, conditioned on the MAV state and, crucially, the stereo baseline to resolve scale ambiguity. The training strategy employs differentiable trajectory costs (safety, smoothness, goal-reaching) and a Virtual Baseline Agmentation (VBA) technique to teach the network to interpret metric scale from varying stereo baselines without re-rendering environments. This approach effectively decouples geometric perception from appearance, allowing zero-shot transfer from simulation to reality.
The evaluation is rigorous, comparing against strong baselines like YOPO V2 and Ego-Planner V2 in both simulation and real-world deployment. The paper demonstrates a 95% success rate in unseen simulated environments (forest, industrial) and successful real-world flight at 7 m/s. Ablation studies clearly isolate the contributions of the foundation model initialization, the interleaved attention mechanism, and the VBA strategy. The comparison with depth-based methods shows that stereo RGB can achieve competitive performance without the hardware cost and latency of depth sensors.
The authors commit to open-sourcing the software, simulation environments, and hardware designs. The paper provides detailed hyperparameters, loss function definitions, and architectural specifics. The use of standard Unity simulation and ROS interfaces enhances reproducibility. The specific version of the foundation model (DA3-Small) and its integration details are clearly described.
The method relies on the availability of a suitable vision grounding foundation model (DA3), which may not be universally available or optimal for all tasks. The real-world evaluation, while impressive, is limited to specific environments (forest, industrial) and may not generalize to all cluttered scenarios (e.g., dynamic obstacles, extreme weather). The computational cost, while reduced, still requires an NVIDIA Orin NX, which is a significant hardware requirement for "compact" drones.
This work bridges the gap between high-level visual foundation models and low-level robotic control, a critical area for autonomous systems. By demonstrating that RGB-only stereo vision can replace depth sensors for agile navigation, it reduces hardware costs and increases the robustness of aerial robots. The methodology of using foundation model features for sim-to-real transfer in control tasks is likely to inspire similar approaches in other robotic domains. Stereo-Planner enables zero-shot sim-to-real quadrotor navigation using stereo RGB by leveraging a vision grounding foundation model for geometry-aware scene representation and a scale-conditioned trajectory decoder. The paper presents a significant technical contribution by demonstrating that pretrained visual features can effectively bridge the sim-to-real domain gap for end-to-end motion planning, achieving high-speed autonomous flight without depth sensors or fine-tuning.
Quadruped robots deployed for inspection and field missions must cross heterogeneous terrain, from stairs, slopes, and gaps to high platforms, stepping stones, and balance beams, where safe support is sparse or see-through. Existing methods interpose a terrain map or a depth image between sensing and control. Maps are costly to maintain, corrupted by odometry drift and outliers, and misled by see-through gratings, while depth cameras degrade with lighting and surface material. In this paper, we present PointWalker, a mapless framework whose single reinforcement learning policy directly consumes raw point clouds from four light detection and ranging (LiDAR) sensors. Voxelized points are encoded by linear self-attention and fused with proprioception through cross-attention, so terrain geometry reaches the policy without maps or foothold estimation. One proximal policy optimization (PPO) run, with terrain-aware rewards and dual-channel simulation of see-through gratings, yields one policy for all seven terrain classes with a 90.1% mean success rate in simulation. Deployed zero-shot on a Deep Robotics X30 at 50 Hz onboard, it crosses 0.8-m gaps, climbs 0.8-m platforms and 45° steel-grating stairs, carries a 50-kg payload, and completes a 2.5-km mountain route.
Primary: Zhejiang University
All Institutions: Zhejiang University, Yunshenchu Technology Co. Ltd., Zhejiang Key Laboratory of Additive Manufacturing Technology and Equipment, Hangzhou Public Library
PointWalker introduces a mapless quadruped locomotion framework that directly consumes raw LiDAR point clouds via linear self-attention and cross-attention fusion, achieving robust zero-shot traversal of heterogeneous and see-through terrains. The paper demonstrates that removing intermediate terrain representations like elevation maps significantly improves performance on sparse and transparent surfaces, validated through rigorous simulation ablations and extensive real-world hardware testing on the Deep Robotics X30.
The paper proposes PointWalker, a reinforcement learning framework that eliminates intermediate terrain representations (like elevation maps or depth images) by directly processing raw LiDAR point clouds. The core architectural contribution is the use of linear self-attention to encode voxelized points into tokens, followed by cross-attention with proprioceptive history to fuse terrain geometry with robot state. This allows the policy to attend to specific relevant points (e.g., gap edges, grating bars) without quadratic computational cost. The training methodology employs a single PPO run across seven terrain classes using a curriculum, with a dual-channel simulation setup where the physics engine sees solid support while the perception channel sees sparse, see-through structures (like steel gratings). This effectively teaches the robot to distinguish between visual clutter and actual support surfaces.
The experimental evaluation is robust, featuring extensive simulation ablations comparing the proposed method against height-map encoders, depth-image encoders, and blind policies. The results show significant performance gains on sparse and see-through terrains where intermediate representations fail. Hardware validation on the Deep Robotics X30 is strong, demonstrating zero-shot transfer to complex real-world scenarios including 0.8m gaps, 45-degree steel-grating stairs, and a 2.5km mountain route. The inclusion of ablation studies on hardware (e.g., removing dual-channel simulation or foot-edge penalties) provides clear causal evidence for the necessity of specific design choices.
The paper provides detailed hyperparameters, network architectures, and simulation configurations (Isaac Lab). The use of standard RL algorithms (PPO) and well-documented domain randomization strategies enhances reproducibility. However, the specific implementation of the dual-channel simulation and the precise LiDAR noise models may require significant effort to replicate exactly. The code is not explicitly linked in the text provided, though a video is available.
The method relies on four LiDARs, which is a significant hardware cost compared to depth cameras. The perception range is limited to the local vicinity of the robot, meaning it cannot plan long-range paths or anticipate obstacles far ahead. The paper acknowledges that porting to other robot platforms with different sensor layouts is untested. Additionally, the method is reactive to local geometry and does not handle dynamic moving obstacles or support surfaces effectively.
This work advances the state-of-the-art in legged robot perception by demonstrating that raw point clouds can be effectively used for high-frequency control without intermediate maps. It addresses a critical gap in industrial inspection robots that must traverse heterogeneous, see-through terrain. The approach could inspire similar direct-perception methods in other robotic domains, such as manipulation or aerial robotics, where intermediate representations may lose critical geometric details. PointWalker introduces a mapless quadruped locomotion framework that directly consumes raw LiDAR point clouds via linear self-attention and cross-attention fusion, achieving robust zero-shot traversal of heterogeneous and see-through terrains. The paper demonstrates that removing intermediate terrain representations like elevation maps significantly improves performance on sparse and transparent surfaces, validated through rigorous simulation ablations and extensive real-world hardware testing on the Deep Robotics X30.
We present a learning-based framework that enables a full-scale bipedal humanoid to sprint at over 7 m/s with motion-prior-guided gait and perform aggressive cornering at high speed with a pronounced trunk lean of up to $25^\circ$. Achieving this combination is challenging: above approximately 4 m/s, periodic foot impacts induce severe, high-frequency yaw-rate oscillations intrinsic to high-speed gaits. This high-frequency temporal structure corrupts the history of proprioceptive feedback that flat-MLP policies rely on, while rendering conventional yaw-rate tracking rewards counterproductive, as they fight against the robot's natural oscillatory dynamics. Moreover, a single style prior shared across skills cannot tell which skill is commanded and inherently compromises the low-speed skill fidelity. To overcome these limitations, we adopt a causal multi-head attention (MHA) module to explicitly model the high-frequency temporal feedback. In place of the oscillatory yaw rate, we design a heading-direction reward that supervises only the low-frequency velocity direction, allowing agile turning at high speed without suppressing natural gait oscillations. We further introduce skill-conditioned multi-adversarial-motion-prior (MAMP) discriminators that condition the style reward on the commanded skill. Ablation experiments confirm that each mechanism is essential: replacing structured temporal encoding with a flat-history MLP caps the top speed; reverting to yaw-rate tracking prevents high-speed turning; and a single shared discriminator makes in-place turning collapse into a quasi-static stance. Our framework bridges the gap between raw sprinting speed and reference-motion fidelity, demonstrating stable high-speed sprinting and aggressive cornering in both simulation and real-robot deployment.
Primary: Tsinghua University
All Institutions: Tsinghua University
The paper presents a learning-based framework enabling full-scale humanoids to sprint at >7 m/s and perform aggressive cornering by introducing causal multi-head attention for high-frequency temporal modeling, heading-direction rewards to avoid suppressing natural gait oscillations, and skill-conditioned adversarial discriminators to maintain motion fidelity. This work significantly advances the state-of-the-art in high-speed bipedal locomotion by identifying and solving specific control challenges inherent to high-velocity gaits, bridging the gap between raw speed and reference-motion fidelity in both simulation and real-world deployment.
The paper proposes a learning-based framework for high-speed bipedal locomotion, addressing specific failure modes of standard flat-MLP policies at high velocities. The core methodological contributions are threefold: (1) the use of a causal multi-head attention (MHA) module to explicitly model high-frequency temporal feedback, which is argued to be necessary because periodic foot impacts above 4 m/s induce yaw-rate oscillations that corrupt flat history buffers; (2) the design of a heading-direction reward that supervises only low-frequency velocity direction, replacing conventional yaw-rate tracking which is shown to be counterproductive at high speeds; and (3) the introduction of skill-conditioned multi-adversarial-motion-prior (MAMP) discriminators to maintain fidelity across different skills (sprinting vs. cornering). The ablation studies presented in the abstract confirm the necessity of each component, showing that flat MLPs cap top speed, yaw-rate tracking prevents turning, and shared discriminators degrade skill fidelity.
The evaluation includes both simulation (Gazebo) and real-robot deployment. The paper reports achieving sprinting speeds over 7 m/s and aggressive cornering with trunk lean up to 25 degrees at 5.5 m/s. The inclusion of real-robot validation is a significant strength, as high-speed bipedal control is notoriously difficult to transfer from simulation to reality. The ablation experiments are described as rigorous, isolating the impact of the temporal encoding, reward structure, and discriminator conditioning. The results demonstrate a clear capability gap closure between raw speed and motion fidelity.
The paper is accepted to CoRL 2026, a venue known for rigorous standards. While the provided text is an abstract and figure caption, the mention of specific hyperparameters (e.g., 7 m/s, 25 degrees) and the use of standard simulation environments (Gazebo) suggest a level of detail typically found in CoRL papers. However, without the full code release or detailed hyperparameter tables in the provided snippet, full reproducibility cannot be guaranteed, though the methodology is clearly defined.
The primary limitation is the reliance on a specific hardware platform (full-scale humanoid), which may limit immediate generalizability to other morphologies. Additionally, the high-speed control regime (>4 m/s) is inherently unstable and sensitive to environmental perturbations, which may not be fully explored in the provided abstract. The "causal MHA" approach may introduce computational overhead compared to flat MLPs, though this is likely negligible for onboard controllers.
This work pushes the physical limits of bipedal robotics, demonstrating that learning-based policies can handle the complex, high-frequency dynamics of sprinting. The insights regarding the failure of flat-MLP policies at high speeds and the necessity of explicit temporal modeling for high-frequency feedback are valuable for the broader robotics community. The skill-conditioned discriminator approach offers a generalizable technique for multi-skill learning in embodied agents. The paper presents a learning-based framework enabling full-scale humanoids to sprint at >7 m/s and perform aggressive cornering by introducing causal multi-head attention for high-frequency temporal modeling, heading-direction rewards to avoid suppressing natural gait oscillations, and skill-conditioned adversarial discriminators to maintain motion fidelity. This work significantly advances the state-of-the-art in high-speed bipedal locomotion by identifying and solving specific control challenges inherent to high-velocity gaits, bridging the gap between raw speed and reference-motion fidelity in both simulation and real-world deployment.