Last 7 Days (October 13 – October 19, 2026)
Significant efforts are currently being directed toward the implementation of large-scale quantum information processors, but realizing useful quantum advantage also requires the development of efficient quantum algorithms. While for many practical problems the dominant resource costs are associated with a small set of computational primitives, optimizing them in the quantum setting presents unique challenges, including a reversibility constraint absent from ordinary classical computation. Here we introduce an evolutionary framework in which large language models (LLMs) propose circuit designs and an independent evaluator verifies and scores them. We apply this framework to tasks spanning numerical computation, chemistry, optimization, quantum simulation, and number theory. In seven of the eight tasks, the discovered circuits significantly reduce both Toffoli and qubit counts relative to the state of the art. In the eighth, the search discovers a new reversible architecture for prime-field modular inversion. By handling reversibility differently, the architecture makes divsteps -- a classical algorithm previously impractical for reversible integer arithmetic -- viable and substantially improves the Toffoli-qubit trade-off for modular inversion. These results open the door for evaluator-guided LLM co-design of new algorithmic structures for quantum computation.
Primary: Harvard University
All Institutions: Harvard University, Anthropic, QuEra Computing
The paper presents a novel evolutionary framework using LLMs to discover state-of-the-art quantum circuits, achieving significant reductions in resource costs across multiple tasks and uncovering new reversible algorithms. This work bridges the gap between AI-driven code generation and rigorous quantum algorithm design, offering a powerful tool for future quantum computing research.
The paper introduces a robust evolutionary framework where Large Language Models (LLMs) propose high-level Python circuit designs, which are then verified by an independent, programmatic evaluator. This approach effectively addresses the "reversibility constraint" of quantum computation by allowing the LLM to reason about algorithmic structure rather than just gate sequences. The methodology is sound, leveraging the fact that coherent circuits on computational basis states can be simulated efficiently (tracking bits and phases), enabling rapid feedback loops. The use of MAP-Elites to maintain diversity in the search space is a strong technical choice that prevents premature convergence to local optima.
The experimental results are highly impressive, demonstrating state-of-the-art improvements in Toffoli and qubit counts across eight diverse tasks, including Reed-Solomon decoding and modular inversion. The discovery of a new reversible architecture for prime-field modular inversion using divsteps is a significant algorithmic contribution, not just an optimization of existing circuits. The benchmarks are rigorous, comparing against established baselines and providing detailed resource accounting.
The paper provides sufficient detail on the circuit model, cost metrics, and search strategy to allow for reproduction of the framework. However, the specific LLM prompts and the exact "OpenEvolve" implementation details are not fully open-sourced in the text, which may limit immediate reproducibility for those without access to the specific tooling or LLM versions (Claude Mythos 5).
The framework is bounded by the task specification and the computational basis representation, excluding optimizations that rely on changing the encoding itself (e.g., coset representations). The reliance on a specific LLM (Claude Mythos 5) and the proprietary "OpenEvolve" framework may limit accessibility. Additionally, the TQ-cost metric is a simplification that may not capture all hardware-specific constraints like depth or routing.
This work has significant implications for the field of quantum computing, offering a scalable path to discovering efficient quantum algorithms. It demonstrates that LLMs can contribute to fundamental algorithmic design in domains with complex constraints, potentially accelerating the development of fault-tolerant quantum computers. The approach could be extended to other areas of AI for Science where programmatic evaluation is feasible. The paper presents a novel evolutionary framework using LLMs to discover state-of-the-art quantum circuits, achieving significant reductions in resource costs across multiple tasks and uncovering new reversible algorithms. This work bridges the gap between AI-driven code generation and rigorous quantum algorithm design, offering a powerful tool for future quantum computing research.
Differentially private learning aims to learn from data while protecting individual training examples. For binary classification, finite Littlestone dimension characterizes approximate private PAC learnability (Alon et al., JACM 2022). Understanding how this dimension controls private sample complexity remains a central question. For classes of Littlestone dimension $d$, the first general upper bound was doubly exponential in $d$ (Bun, Livni, and Moran, FOCS 2020). Subsequent work improved this to $\widetilde O(d^6)$ (Ghazi et al., STOC 2021) and then to $\widetilde O(d^5)$ (Lyu, STOC 2026). We prove a nearly linear upper bound of $O(d\log d)$. Our main algorithmic contribution is a structural exponential mechanism for data-dependent candidate sets equipped with a rank satisfying suitable stability conditions. For natural Littlestone classes such as subspace indicators, this rank has a simple geometric interpretation: it is the codimension. This yields a private PAC learner for subspaces of dimension at most $d$ over any field, with tight linear dependence on $d$ and no dependence on the ambient dimension, improving the quadratic dependence of Kaplan et al. (STOC 2025). Our second contribution is combinatorial: drawing on model-theoretic ideas, we construct a suitable rank for general Littlestone classes and establish the counting bounds required by the mechanism.
Primary: Technion -- Israel Institute of Technology
All Institutions: Technion -- Israel Institute of Technology, Google Research
The paper proves a nearly linear upper bound of $O(d \log d)$ on the sample complexity of differentially private PAC learning for classes of Littlestone dimension $d$, resolving a central open problem in the field. By introducing a structural exponential mechanism that leverages a model-theoretic rank, the authors achieve a tight dependence on the Littlestone dimension, significantly improving upon previous polynomial bounds and providing a new algorithmic framework for private learning.
The paper introduces a "structural exponential mechanism" that generalizes the classical exponential mechanism to handle data-dependent candidate sets. This is a significant algorithmic innovation in private learning theory. The core novelty lies in defining a combinatorial rank for hypotheses in Littlestone classes, derived from model-theoretic concepts (specifically Shelah's $2$-rank), and proving that this rank satisfies stability conditions necessary for differential privacy. The authors demonstrate that for subspace indicators, this rank corresponds to codimension, yielding a tight linear sample complexity bound $O(d \log d)$, which is optimal up to logarithmic factors. The methodology is rigorous, combining combinatorial arguments with privacy analysis.
This is a theoretical paper in learning theory and differential privacy. There are no empirical experiments, datasets, or benchmarks. The "evaluation" consists of mathematical proofs establishing upper bounds on sample complexity and verifying the privacy guarantees of the proposed mechanism. The contribution is entirely analytical.
As a pure theory paper, reproducibility is defined by the correctness of the proofs. The paper provides detailed technical overviews and formal definitions. However, the full proofs are truncated in the provided text (ending in Section 5.1), so a complete verification of the combinatorial lemmas (specifically Lemmas 5.2-5.4 and the counting bounds in Section 5.3) is not possible from the snippet alone. Assuming the standard rigor of the authors (Moran, Yan) and the venue (likely STOC/FOCS level), the theoretical framework is sound.
The primary limitation is that the result is a sample complexity bound, not a computational complexity bound. The paper explicitly states that it does not assert computational efficiency. The algorithm described (selecting candidates based on version spaces and ranks) may be computationally intractable for large $d$ or complex classes. Additionally, the bound is for the realizable setting; while reductions to the agnostic setting exist, they incur additional overhead. The log-star question regarding VC vs Littlestone dimension remains open.
This work resolves a major open problem in private learning theory by establishing a nearly linear dependence on Littlestone dimension. It provides a new algorithmic primitive (structural exponential mechanism) that can be applied to other private optimization problems involving data-dependent candidate sets. It bridges learning theory and model theory, offering new combinatorial tools for understanding the complexity of private learning. The result is significant for the theoretical understanding of the trade-offs between privacy and sample efficiency. The paper proves a nearly linear upper bound of $O(d \log d)$ on the sample complexity of differentially private PAC learning for classes of Littlestone dimension $d$, resolving a central open problem in the field. By introducing a structural exponential mechanism that leverages a model-theoretic rank, the authors achieve a tight dependence on the Littlestone dimension, significantly improving upon previous polynomial bounds and providing a new algorithmic framework for private learning.
We study uniform approximation of the Hölder ball $\mathcal H_1^s([0,1]^D)$ by fully connected ReLU networks. Except in the one-dimensional Lipschitz case, existing constructions remain separated from the information-theoretic efficiency limit. This article closes the gap for every $s>0$ and $D\ge1$: networks with width $O(N)$, depth $O(L)$, and parameter magnitude $O(1)$ attain uniform approximation error of order $(N^2L^2\log N)^{-s/D}$. No relation between width and depth is required. The construction uses new encoding and decoding mechanisms that exploit higher-order finite-difference bounds for local Taylor coefficients of the target function. As a statistical consequence, empirical risk minimizers over the corresponding clipped network classes attain the minimax regression rate $n^{-2s/(2s+D)}$ without additional logarithmic factors. The paper also gives a detailed exposition of the key approximation techniques, covering both established and new mechanisms.
Primary: Munich Center for Machine Learning (MCML)
All Institutions: Munich Center for Machine Learning (MCML), Department of Statistics
The paper closes the gap between existing ReLU network approximation constructions and the information-theoretic efficiency limit for Hölder smooth functions, establishing optimal rates for width, depth, and parameter magnitude. This is a significant theoretical contribution that resolves a major open problem in neural network approximation theory, providing a complete characterization of the capacity of ReLU networks for smooth function approximation.
The paper presents a rigorous theoretical framework for the uniform approximation of Hölder smooth functions by ReLU neural networks. The core innovation is a "higher-order point-fitting" mechanism that exploits the regularity of local Taylor coefficients to compress their storage within the network weights. By using higher-order finite differences and a segmented discrete Taylor-Newton recursion, the authors achieve an approximation error of $(N^2L^2\log N)^{-s/D}$ with bounded parameter magnitudes ($O(1)$). This closes the gap between existing constructions and the information-theoretic efficiency limit for all smoothness orders $s>0$ and dimensions $D\ge1$. The methodology is highly sophisticated, combining established techniques like bit-extraction and median smoothing with novel encoding strategies that preserve smoothness during the flattening of multivariate coefficient grids via Hermite interpolation bridges.
As a pure theory paper, there are no empirical experiments, datasets, or benchmark comparisons. The "evaluation" consists of mathematical proofs establishing upper bounds on approximation error and matching lower bounds via metric entropy arguments. The statistical consequence (minimax regression rates) is derived theoretically.
The paper is highly reproducible in the mathematical sense. It provides self-contained proofs for all key components, including simplified versions of standard techniques. The construction is explicit, and the bounds are tight. However, without code, practical implementation of these specific network architectures for real-world data is not directly demonstrated.
The results are limited to uniform approximation on the unit cube $[0,1]^D$ for Hölder classes. The lower bound relies on metric entropy, which may not capture all aspects of practical learning dynamics. The paper does not address the optimization landscape or the trainability of these networks, which is a significant gap between theoretical optimality and practical deep learning.
This work provides a definitive answer to a long-standing question in neural network approximation theory. It establishes that ReLU networks are optimal approximators for smooth functions in terms of width, depth, and parameter magnitude. This has implications for understanding the capacity of neural networks and the theoretical limits of nonparametric regression. It may influence future theoretical work on network architectures and the design of approximation schemes for other function classes. The paper closes the gap between existing ReLU network approximation constructions and the information-theoretic efficiency limit for Hölder smooth functions, establishing optimal rates for width, depth, and parameter magnitude. This is a significant theoretical contribution that resolves a major open problem in neural network approximation theory, providing a complete characterization of the capacity of ReLU networks for smooth function approximation.
Differentially private learning aims to learn from data while protecting individual training examples. For binary classification, finite Littlestone dimension characterizes approximate private PAC learnability (Alon et al., JACM 2022). Understanding how this dimension controls private sample complexity remains a central question. For classes of Littlestone dimension $d$, the first general upper bound was doubly exponential in $d$ (Bun, Livni, and Moran, FOCS 2020). Subsequent work improved this to $\widetilde O(d^6)$ (Ghazi et al., STOC 2021) and then to $\widetilde O(d^5)$ (Lyu, STOC 2026). We prove a nearly linear upper bound of $O(d\log d)$. Our main algorithmic contribution is a structural exponential mechanism for data-dependent candidate sets equipped with a rank satisfying suitable stability conditions. For natural Littlestone classes such as subspace indicators, this rank has a simple geometric interpretation: it is the codimension. This yields a private PAC learner for subspaces of dimension at most $d$ over any field, with tight linear dependence on $d$ and no dependence on the ambient dimension, improving the quadratic dependence of Kaplan et al. (STOC 2025). Our second contribution is combinatorial: drawing on model-theoretic ideas, we construct a suitable rank for general Littlestone classes and establish the counting bounds required by the mechanism.
Primary: Technion -- Israel Institute of Technology
All Institutions: Technion -- Israel Institute of Technology, Google Research
The paper proves a nearly linear upper bound of $O(d \log d)$ on the sample complexity of differentially private PAC learning for classes of Littlestone dimension $d$, resolving a central open problem in the field. By introducing a structural exponential mechanism that leverages a model-theoretic rank, the authors achieve a tight dependence on the Littlestone dimension, significantly improving upon previous polynomial bounds and providing a new algorithmic framework for private learning.
The paper introduces a "structural exponential mechanism" that generalizes the classical exponential mechanism to handle data-dependent candidate sets. This is a significant algorithmic innovation in private learning theory. The core novelty lies in defining a combinatorial rank for hypotheses in Littlestone classes, derived from model-theoretic concepts (specifically Shelah's $2$-rank), and proving that this rank satisfies stability conditions necessary for differential privacy. The authors demonstrate that for subspace indicators, this rank corresponds to codimension, yielding a tight linear sample complexity bound $O(d \log d)$, which is optimal up to logarithmic factors. The methodology is rigorous, combining combinatorial arguments with privacy analysis.
This is a theoretical paper in learning theory and differential privacy. There are no empirical experiments, datasets, or benchmarks. The "evaluation" consists of mathematical proofs establishing upper bounds on sample complexity and verifying the privacy guarantees of the proposed mechanism. The contribution is entirely analytical.
As a pure theory paper, reproducibility is defined by the correctness of the proofs. The paper provides detailed technical overviews and formal definitions. However, the full proofs are truncated in the provided text (ending in Section 5.1), so a complete verification of the combinatorial lemmas (specifically Lemmas 5.2-5.4 and the counting bounds in Section 5.3) is not possible from the snippet alone. Assuming the standard rigor of the authors (Moran, Yan) and the venue (likely STOC/FOCS level), the theoretical framework is sound.
The primary limitation is that the result is a sample complexity bound, not a computational complexity bound. The paper explicitly states that it does not assert computational efficiency. The algorithm described (selecting candidates based on version spaces and ranks) may be computationally intractable for large $d$ or complex classes. Additionally, the bound is for the realizable setting; while reductions to the agnostic setting exist, they incur additional overhead. The log-star question regarding VC vs Littlestone dimension remains open.
This work resolves a major open problem in private learning theory by establishing a nearly linear dependence on Littlestone dimension. It provides a new algorithmic primitive (structural exponential mechanism) that can be applied to other private optimization problems involving data-dependent candidate sets. It bridges learning theory and model theory, offering new combinatorial tools for understanding the complexity of private learning. The result is significant for the theoretical understanding of the trade-offs between privacy and sample efficiency. The paper proves a nearly linear upper bound of $O(d \log d)$ on the sample complexity of differentially private PAC learning for classes of Littlestone dimension $d$, resolving a central open problem in the field. By introducing a structural exponential mechanism that leverages a model-theoretic rank, the authors achieve a tight dependence on the Littlestone dimension, significantly improving upon previous polynomial bounds and providing a new algorithmic framework for private learning.
Diffusion large language models (dLLMs) have emerged as a promising alternative to autoregressive language models, enabling non-autoregressive text generation through parallel decoding. Existing dLLMs can be broadly categorized into two families: masked diffusion models (MDMs) and uniform diffusion models (UDMs) In this work, we theoretically show that MDMs are easier to optimize due to their mask-based transition schedule but have lower computational expressivity, whereas UDMs enable iterative state revision and greater expressivity but are substantially harder to train. This trade-off raises a natural question: can we obtain the expressivity of iterative revision without giving up the simple, easy-to-optimize transition schedule of MDMs? Motivated by this analysis, we propose Lookahead Editing (\ours{}), which augments masked diffusion decoding with a rewritable proposal state. At each decoding step, the model performs several lookahead updates to iteratively refine future predictions in this proposal state before committing tokens to the masked sequence. This design introduces a rewritable proposal that uses additional compute to improve expressivity while keeping the simple mask-based transition schedule of MDMs. We fine-tune Qwen2.5-7B and Qwen3-8B to obtain LED-\texttt{7B} and LED-\texttt{8B}, respectively. Across eight benchmarks, LED SFT yields average gains of $+11.4$ and $+7.7$ points over standard MDM SFT. Moreover, using SFT alone, LED-\texttt{8B} outperforms LLaDA2.1-16B Q-mode, which is trained with continued pre-training, supervised fine-tuning, and reinforcement learning.
Primary: University of Cambridge
All Institutions: University of Cambridge, NVIDIA
The paper introduces Lookahead Editing, a method that enhances masked diffusion language models by incorporating a rewritable proposal state to improve computational expressivity while maintaining optimization stability. It provides rigorous theoretical proofs regarding the circuit complexity of MDMs vs UDMs and demonstrates significant empirical gains in reasoning tasks, outperforming larger models trained with more complex pipelines.
The paper proposes "Lookahead Editing" (LED), a decoding strategy for Masked Diffusion Language Models (MDMs) that introduces a rewritable proposal state. The core innovation is bridging the gap between the optimization simplicity of MDMs (which have a fixed, easy-to-learn transition schedule) and the computational expressivity of Uniform Diffusion Models (UDMs) (which allow iterative revision but are harder to train). Theoretically, the authors prove that MDMs are contained in $TC^0$ while UDMs can reach $NC^1$, justifying the need for iterative revision. The method involves a three-branch training setup (clean, masked, proposal) where the proposal branch is generated by a pretrained MDM, allowing the model to learn to correct errors in the proposal. At inference, the model performs $K$ lookahead steps to refine the proposal before committing tokens. This is a clever architectural/algorithmic modification that preserves the training stability of MDMs while gaining the benefits of iterative refinement.
The experiments are conducted on Qwen2.5-7B and Qwen3-8B models. The results show significant improvements over standard MDM SFT baselines (+11.4 and +7.7 points average across 8 benchmarks). Notably, LED-8B outperforms LLaDA2.1-16B, which uses a more complex post-training pipeline (CPT + SFT + RL), using only SFT. The speed-accuracy trade-off analysis demonstrates that LED offers a better Pareto front than existing dLLMs. The benchmarks cover math, science, and code, providing a comprehensive evaluation of reasoning capabilities.
The paper provides detailed pseudocode for both training and inference. The hyperparameters (block size, batch size, learning rate) are specified. However, the specific datasets used for SFT are listed as open-source, but the exact mixture ratios and preprocessing steps are not fully detailed in the provided text. The reliance on a pretrained MDM for generating proposals adds a dependency that must be managed, but the overall procedure is clearly defined.
The method requires an additional forward pass (or multiple, depending on $K$) for the proposal generation, which increases inference latency compared to standard MDMs, although the paper argues this is offset by the ability to use fewer decoding steps or higher throughput at matched accuracy. The theoretical analysis, while rigorous, relies on specific assumptions about the circuit complexity of transformers that may not hold for all practical implementations. The performance gains are observed on specific benchmarks (math/code), and generalization to other domains (e.g., creative writing, long-form generation) is not explicitly tested.
This work contributes to the growing field of diffusion-based language models, offering a practical way to improve their reasoning capabilities without abandoning the stable training dynamics of masked diffusion. The theoretical insights into the optimization-expressivity trade-off are valuable for the community. The ability to outperform larger, more complexly trained models with a simpler SFT recipe suggests that architectural/decoding innovations can be as impactful as data scaling or RL. The paper introduces Lookahead Editing, a method that enhances masked diffusion language models by incorporating a rewritable proposal state to improve computational expressivity while maintaining optimization stability. It provides rigorous theoretical proofs regarding the circuit complexity of MDMs vs UDMs and demonstrates significant empirical gains in reasoning tasks, outperforming larger models trained with more complex pipelines.
Significant efforts are currently being directed toward the implementation of large-scale quantum information processors, but realizing useful quantum advantage also requires the development of efficient quantum algorithms. While for many practical problems the dominant resource costs are associated with a small set of computational primitives, optimizing them in the quantum setting presents unique challenges, including a reversibility constraint absent from ordinary classical computation. Here we introduce an evolutionary framework in which large language models (LLMs) propose circuit designs and an independent evaluator verifies and scores them. We apply this framework to tasks spanning numerical computation, chemistry, optimization, quantum simulation, and number theory. In seven of the eight tasks, the discovered circuits significantly reduce both Toffoli and qubit counts relative to the state of the art. In the eighth, the search discovers a new reversible architecture for prime-field modular inversion. By handling reversibility differently, the architecture makes divsteps -- a classical algorithm previously impractical for reversible integer arithmetic -- viable and substantially improves the Toffoli-qubit trade-off for modular inversion. These results open the door for evaluator-guided LLM co-design of new algorithmic structures for quantum computation.
Primary: Harvard University
All Institutions: Harvard University, Anthropic, QuEra Computing
The paper presents a novel evolutionary framework using LLMs to discover state-of-the-art quantum circuits, achieving significant reductions in resource costs across multiple tasks and uncovering new reversible algorithms. This work bridges the gap between AI-driven code generation and rigorous quantum algorithm design, offering a powerful tool for future quantum computing research.
The paper introduces a robust evolutionary framework where Large Language Models (LLMs) propose high-level Python circuit designs, which are then verified by an independent, programmatic evaluator. This approach effectively addresses the "reversibility constraint" of quantum computation by allowing the LLM to reason about algorithmic structure rather than just gate sequences. The methodology is sound, leveraging the fact that coherent circuits on computational basis states can be simulated efficiently (tracking bits and phases), enabling rapid feedback loops. The use of MAP-Elites to maintain diversity in the search space is a strong technical choice that prevents premature convergence to local optima.
The experimental results are highly impressive, demonstrating state-of-the-art improvements in Toffoli and qubit counts across eight diverse tasks, including Reed-Solomon decoding and modular inversion. The discovery of a new reversible architecture for prime-field modular inversion using divsteps is a significant algorithmic contribution, not just an optimization of existing circuits. The benchmarks are rigorous, comparing against established baselines and providing detailed resource accounting.
The paper provides sufficient detail on the circuit model, cost metrics, and search strategy to allow for reproduction of the framework. However, the specific LLM prompts and the exact "OpenEvolve" implementation details are not fully open-sourced in the text, which may limit immediate reproducibility for those without access to the specific tooling or LLM versions (Claude Mythos 5).
The framework is bounded by the task specification and the computational basis representation, excluding optimizations that rely on changing the encoding itself (e.g., coset representations). The reliance on a specific LLM (Claude Mythos 5) and the proprietary "OpenEvolve" framework may limit accessibility. Additionally, the TQ-cost metric is a simplification that may not capture all hardware-specific constraints like depth or routing.
This work has significant implications for the field of quantum computing, offering a scalable path to discovering efficient quantum algorithms. It demonstrates that LLMs can contribute to fundamental algorithmic design in domains with complex constraints, potentially accelerating the development of fault-tolerant quantum computers. The approach could be extended to other areas of AI for Science where programmatic evaluation is feasible. The paper presents a novel evolutionary framework using LLMs to discover state-of-the-art quantum circuits, achieving significant reductions in resource costs across multiple tasks and uncovering new reversible algorithms. This work bridges the gap between AI-driven code generation and rigorous quantum algorithm design, offering a powerful tool for future quantum computing research.
We study uniform approximation of the Hölder ball $\mathcal H_1^s([0,1]^D)$ by fully connected ReLU networks. Except in the one-dimensional Lipschitz case, existing constructions remain separated from the information-theoretic efficiency limit. This article closes the gap for every $s>0$ and $D\ge1$: networks with width $O(N)$, depth $O(L)$, and parameter magnitude $O(1)$ attain uniform approximation error of order $(N^2L^2\log N)^{-s/D}$. No relation between width and depth is required. The construction uses new encoding and decoding mechanisms that exploit higher-order finite-difference bounds for local Taylor coefficients of the target function. As a statistical consequence, empirical risk minimizers over the corresponding clipped network classes attain the minimax regression rate $n^{-2s/(2s+D)}$ without additional logarithmic factors. The paper also gives a detailed exposition of the key approximation techniques, covering both established and new mechanisms.
Primary: Munich Center for Machine Learning (MCML)
All Institutions: Munich Center for Machine Learning (MCML), Department of Statistics
The paper closes the gap between existing ReLU network approximation constructions and the information-theoretic efficiency limit for Hölder smooth functions, establishing optimal rates for width, depth, and parameter magnitude. This is a significant theoretical contribution that resolves a major open problem in neural network approximation theory, providing a complete characterization of the capacity of ReLU networks for smooth function approximation.
The paper presents a rigorous theoretical framework for the uniform approximation of Hölder smooth functions by ReLU neural networks. The core innovation is a "higher-order point-fitting" mechanism that exploits the regularity of local Taylor coefficients to compress their storage within the network weights. By using higher-order finite differences and a segmented discrete Taylor-Newton recursion, the authors achieve an approximation error of $(N^2L^2\log N)^{-s/D}$ with bounded parameter magnitudes ($O(1)$). This closes the gap between existing constructions and the information-theoretic efficiency limit for all smoothness orders $s>0$ and dimensions $D\ge1$. The methodology is highly sophisticated, combining established techniques like bit-extraction and median smoothing with novel encoding strategies that preserve smoothness during the flattening of multivariate coefficient grids via Hermite interpolation bridges.
As a pure theory paper, there are no empirical experiments, datasets, or benchmark comparisons. The "evaluation" consists of mathematical proofs establishing upper bounds on approximation error and matching lower bounds via metric entropy arguments. The statistical consequence (minimax regression rates) is derived theoretically.
The paper is highly reproducible in the mathematical sense. It provides self-contained proofs for all key components, including simplified versions of standard techniques. The construction is explicit, and the bounds are tight. However, without code, practical implementation of these specific network architectures for real-world data is not directly demonstrated.
The results are limited to uniform approximation on the unit cube $[0,1]^D$ for Hölder classes. The lower bound relies on metric entropy, which may not capture all aspects of practical learning dynamics. The paper does not address the optimization landscape or the trainability of these networks, which is a significant gap between theoretical optimality and practical deep learning.
This work provides a definitive answer to a long-standing question in neural network approximation theory. It establishes that ReLU networks are optimal approximators for smooth functions in terms of width, depth, and parameter magnitude. This has implications for understanding the capacity of neural networks and the theoretical limits of nonparametric regression. It may influence future theoretical work on network architectures and the design of approximation schemes for other function classes. The paper closes the gap between existing ReLU network approximation constructions and the information-theoretic efficiency limit for Hölder smooth functions, establishing optimal rates for width, depth, and parameter magnitude. This is a significant theoretical contribution that resolves a major open problem in neural network approximation theory, providing a complete characterization of the capacity of ReLU networks for smooth function approximation.
Recursive self-improvement, in which AI systems build stronger AI systems, is one promising route to superintelligence. We study a weaker form that can be measured today: frontier agents post-training a small raw base model with public data, as a test of their research potential. We build Athanor, an open agent-native system for autonomous post-training research (AutoPostTrain) on real GPUs. We give the agents full research autonomy, low-friction engineering tools, and efficient starter code. The agents carry out the research themselves: they read datasets and evaluation results, build their training data, write and modify training algorithms, tune hyperparameters, and analyze the results to decide what to try next. They work in a sandbox and cannot use external teacher models. Athanor's services supply frozen evaluation, managed jobs, decontaminated data, a persistent research record, and a held-out suite the agents are unaware of. In a 104-hour campaign, a team of four Claude Opus 5.5 agents, each on an eight-GPU node and given no research advice, improves Qwen3-4B-Base from 15.9 to 67.3 on a six-task development suite and from 9.2 to 48.2 on a held-out suite, against 66.5 and 47.9 for the official Qwen3-4B-Instruct-2507. To our knowledge, this is the first demonstration that a team of frontier agents can autonomously post-train a raw base model into a cross-domain generalist model that matches or surpasses the official instruct version on average. The campaign's full record shows that the agents reach this level by recombining known methods and each other's models: SFT on public reasoning data, RL with a rollout change one agent introduced, weight merges across agents, and rounds of self-distillation. With execution handled, today's agents already have the engineering and research ability to conquer this task. We hope Athanor provides a foundation for scaling autonomous AI research.
Primary: ByteDance
All Institutions: ByteDance, Duke University, The University of Chicago, Virginia Tech
The paper presents Athanor, an agent-native system that enables frontier AI agents to autonomously post-train a base model to match official instruct versions. This is a significant step toward recursive self-improvement, demonstrating that current agents possess the engineering and research ability to execute complex ML research tasks when provided with the right tools and autonomy, thereby validating the potential for AI-driven acceleration of AI development.
The paper proposes "Athanor," a system designed to facilitate autonomous post-training research by frontier AI agents. The core methodological contribution is the creation of an "agent-native" environment that provides agents with full research autonomy, low-friction engineering tools, and efficient starter code. The system architecture includes specific services to support this autonomy: frozen evaluation, managed job scheduling, decontaminated data access, a persistent research record, and a held-out suite that the agents are explicitly unaware of. The methodology emphasizes that agents must perform the entire research loop—reading data, building training sets, writing/modifying algorithms, tuning hyperparameters, and analyzing results—without external teacher models or human research advice. This represents a shift from using agents as code generators to using them as autonomous researchers within a constrained, verifiable sandbox.
The experimental setup involves a 104-hour campaign where a team of four Claude Opus 5.5 agents, each operating on an eight-GPU node, attempts to post-train the Qwen3-4B-Base model. The results are striking: the agents improved the model's performance from 15.9 to 67.3 on a six-task development suite and from 9.2 to 48.2 on a held-out suite. These scores surpass or match the official Qwen3-4B-Instruct-2507 model (66.5 and 47.9, respectively). The paper claims this is the first demonstration of frontier agents autonomously post-training a raw base model into a cross-domain generalist that matches official instruct versions. The analysis of the campaign record indicates the agents succeeded by recombining known methods (SFT on public reasoning data, RL with specific rollout changes, weight merges, and self-distillation), suggesting that the agents' ability to orchestrate existing techniques is the key driver of success.
The paper describes the system components (services, sandbox, tools) in detail, which aids in understanding the framework. However, as a preprint with no citation count and no provided code repository or demo URL, immediate reproducibility is limited. The reliance on specific frontier models (Claude Opus 5.5) and proprietary infrastructure (ByteDance GPUs) may limit external replication. The "agent-native" system itself is described as open, but the specific prompts, starter code, and internal logic of the agents are not fully detailed in the abstract or provided text, which is a barrier to full reproduction.
The primary limitation is the reliance on a specific, high-cost frontier model (Claude Opus 5.5) for the agents, making the results dependent on the capabilities of that specific model version. The experiment is limited to a single base model (Qwen3-4B-Base) and a specific set of tasks, so generalizability to other model architectures or domains is not yet proven. The "104-hour" timeframe, while impressive, is still a finite and relatively short window for complex research, and the paper does not explore what happens if the agents are given significantly more time or resources. Additionally, the lack of a public code release at the time of review hinders community verification.
This paper has significant implications for the field of AI safety and the development of superintelligence. It provides a concrete, measurable step toward recursive self-improvement, demonstrating that current agents can execute complex research tasks that were previously thought to require human oversight. The "Athanor" system could serve as a foundational benchmark for evaluating the research capabilities of future AI models. It also highlights the potential for AI systems to accelerate the pace of AI research itself, which has profound ethical and strategic implications. The finding that agents can match official instruct models through autonomous recombination of known methods suggests that the bottleneck for AI progress may be shifting from algorithmic discovery to execution and orchestration. The paper presents Athanor, an agent-native system that enables frontier AI agents to autonomously post-train a base model to match official instruct versions. This is a significant step toward recursive self-improvement, demonstrating that current agents possess the engineering and research ability to execute complex ML research tasks when provided with the right tools and autonomy, thereby validating the potential for AI-driven acceleration of AI development.
Large language models (LLMs) reproduce a human working-memory signature on the N-back task: performance declines as memory load N increases, and errors concentrate on lures, items that repeat the current stimulus at a neighboring but incorrect serial position. We ask whether this shared behavioral signature reflects a shared mechanism. The literature offers two candidate accounts. On a precision account, softmax attention cannot concentrate its weight on the item N positions back, so adjacent items are retrieved instead. On an interference account, attention selects the correct item, but the content of competing items corrupts the decision. We dissociate these accounts in 15 pretrained LLMs from three families (70 million to 14 billion parameters) and in minimal transformers trained from scratch. In all three families, the attention heads that carry the task attend to the target on most missed matches. Linear probes decode near-perfectly whether a matching letter occurred at the target or at a lure position, even on trials where the model false-alarms to the lure, although its own decisions barely distinguish the two. Sharpening attention yields negligible gains, whereas suppressing the letter identity of non-target items improves sensitivity (d') in nearly all model-by-load conditions. In larger models the task is spread over more heads, and removing interference helps more. In minimal transformers, precision limits a single attention layer, but a second layer removes this constraint. The working-memory limit of LLMs therefore lies in reading items out under interference rather than in locating them, consistent with interference accounts of human working memory.
Primary: Yale University
All Institutions: Yale University
The paper provides a rigorous mechanistic explanation for LLM working memory failures, demonstrating that the bottleneck lies in reading out items under interference rather than in locating them via attention. By combining theoretical analysis of minimal transformers with extensive empirical interventions on 15 pretrained LLMs, the authors successfully dissociate precision and interference accounts, offering a new perspective on the internal computations of large language models.
The paper employs a rigorous mechanistic interpretability framework to dissect the working memory limitations of Large Language Models (LLMs) on the N-back task. The methodology is sophisticated, combining behavioral analysis (signal detection theory, $d'$), representational probing (linear probes on residual streams), and causal interventions (activation patching, attention sharpening, and subspace suppression of distractor content). A key strength is the theoretical grounding provided for minimal transformers, where the authors derive propositions showing that single-layer attention is fundamentally limited by softmax dispersion (precision account), while two-layer architectures can overcome this via direct comparison. This theoretical foundation is then contrasted with empirical findings in large pretrained models, where the bottleneck shifts to interference during readout. The use of activation patching to identify task-critical heads and the specific intervention of suppressing letter-identity directions in non-target items are methodologically sound and directly address the core hypothesis.
The experimental scope is impressive, covering 15 pretrained LLMs from three distinct families (Pythia, GPT-2, Qwen2.5) spanning a 200-fold range in parameter size (70M to 14B). Additionally, the authors train 720 minimal transformers from scratch to isolate architectural constraints. The results are consistent across models: attention heads correctly locate the target even on error trials, and linear probes can decode the correct answer from activations even when the model's decision is wrong. The causal intervention results are particularly compelling; suppressing distractor content significantly improves sensitivity ($d'$), whereas sharpening attention yields negligible gains. The inclusion of recurrent baselines (LSTM, LRU, Mamba) further validates that the observed decline is not an artifact of softmax attention alone but relates to content-dependent retrieval mechanisms.
The paper provides detailed descriptions of the N-back task setup, prompt formatting, and intervention procedures. The use of standard, publicly available models (Pythia, GPT-2, Qwen2.5) enhances reproducibility. The minimal transformer training details (hyperparameters, seeds, architecture variants) are well-documented. However, the specific code for the subspace suppression intervention and the exact implementation of the linear probes are not provided in the text, which may pose a minor barrier for exact replication without access to the authors' codebase.
The study is limited to the letter N-back task with synthetic stimuli, which may not fully capture the complexity of real-world working memory tasks. The mechanistic analysis relies on a specific line-by-line prompt format with correct earlier answers shown, which might not generalize to all prompting strategies. The head-level account may not transfer to models larger than 14B, as the identified heads become less critical in larger models. Furthermore, the linear probes only demonstrate that information is decodable, not necessarily that it is the sole driver of behavior, although the causal interventions help mitigate this concern.
This work has significant implications for the field of mechanistic interpretability and the understanding of LLM limitations. It challenges the prevailing "precision" narrative regarding attention failures and highlights "interference" as a key bottleneck. This insight could guide future model architectures and training objectives aimed at improving working memory and reasoning capabilities. The findings also contribute to the broader debate on the convergence of neural network mechanisms and their analogy to human cognitive processes. The paper provides a rigorous mechanistic explanation for LLM working memory failures, demonstrating that the bottleneck lies in reading out items under interference rather than in locating them via attention. By combining theoretical analysis of minimal transformers with extensive empirical interventions on 15 pretrained LLMs, the authors successfully dissociate precision and interference accounts, offering a new perspective on the internal computations of large language models.
Configuring distributed LLM training across hundreds of GPUs requires jointly optimizing tensor, pipeline, expert, context, and data parallelism alongside micro-batch size, recomputation, sharding, and overlap, a combinatorial space where memory legality can hinge on a single byte of activation accounting. Today, teams navigate it through expensive multi-node trial and error, burning GPU-hours on crashes and suboptimal recipes. We present Primus Projection, which closes this loop with two coupled components. First, a hybrid projection engine that measures what it can and simulates what it cannot: GPU microbenchmarks on as few as a single device capture compute kernels and any in-scope communication; a sub-node downscale-measure-upscale workflow analytically restores out-of-scope collectives, FSDP overlap, and the full pipeline schedule at the multi-node target. A pure simulation mode runs entirely on CPU, enabling pre-silicon planning without any GPU. Second, the Primus Tuning Agent treats the projection engine as a fast scoring oracle (seconds per candidate instead of tens of minutes) and uses it to drive an intelligent search over the configuration space. A deterministic seed planner sweeps single axes; an LLM investigation loop proposes cross-axis combinations the planner cannot reach, with an analytic memory pre-filter that rejects infeasible candidates before any GPU time is spent. Validated on Llama 3.1 (dense) and Mixtral 8x22B (MoE) across two GPU generations, all projections stay within ~10% of measured multi-node throughput, and fidelity holds at production scale: 4.7% error on DeepSeek V3 671B at 1,024 GPUs and 1.9% on Qwen3-235B-A22B at 256 GPUs. In a Mixtral 8x22B case study, the Primus Tuning Agent discovers a configuration delivering +27% throughput over the published 4-node BF16 reference at the same scale, in under 30 minutes of single-node exploration.
Primary: Advanced Micro Devices (AMD)
All Institutions: Advanced Micro Devices (AMD)
The paper presents a highly effective hybrid system for optimizing distributed LLM training configurations by combining single-node micro-benchmarks with analytical simulation and LLM-driven search. Its rigorous validation on production-scale models and significant throughput improvements demonstrate substantial practical value for large-scale AI infrastructure, though its impact is somewhat constrained by its specific validation on AMD hardware.
The paper introduces "Primus Projection," a hybrid performance modeling framework that combines empirical micro-benchmarks on limited hardware (down to a single GPU) with analytical simulation for out-of-scope communication and pipeline scheduling. The methodology is rigorous, addressing the "combinatorial explosion" of distributed training configurations (TP, PP, EP, CP, DP, etc.) by creating a fast scoring oracle. The "downscale-measure-upscale" workflow is a clever engineering solution to the problem of requiring full-cluster access for profiling. The integration of an LLM-based tuning agent (using DSPy and ReAct-style reasoning) to navigate this configuration space is a modern and effective approach, leveraging the projection engine as a cheap surrogate to avoid expensive trial-and-error on real clusters. The memory pre-filtering mechanism is particularly strong, preventing OOM errors before any GPU time is spent.
The evaluation is extensive and convincing. The authors validate their projection engine against measured multi-node throughput on AMD MI325X and MI355X GPUs, achieving within ~10% error for standard models and as low as 1.9-4.7% error for production-scale models (DeepSeek V3, Qwen3-235B). The case study on Mixtral 8x22B demonstrates a +27% throughput improvement over published references using only single-node exploration. The comparison against analytical baselines (Calculon, Galvatron) clearly shows the superiority of the hybrid benchmark-anchored approach, particularly for MoE models where analytical models fail. The robustness analysis regarding MoE load balancing is a valuable addition, showing how the model handles imbalance.
The paper provides detailed algorithmic descriptions (Algorithm 1), hardware profile formats (YAML), and clear definitions of the search space. However, as an industry paper from AMD, the specific code implementation and hardware profiles are likely proprietary or not fully open-sourced, which may limit immediate reproducibility by external researchers without access to similar AMD hardware. The methodology is described well enough for replication in principle, but the "black box" nature of the LLM agent's prompts and the specific hardware calibration data are potential barriers.
The primary limitation is the vendor-specific validation; all experiments are on AMD Instinct GPUs and the ROCm stack. While the authors argue the method is vendor-agnostic, the lack of validation on NVIDIA or TPU hardware is a significant gap for a general ML audience. Additionally, the LLM agent's performance depends on the quality of the projection oracle; if the oracle has systematic biases (as noted with the optimistic/pessimistic offsets), the agent might converge to suboptimal local minima if the ranking is distorted, although the paper argues rank preservation is robust. Network contention is not modeled, which could be an issue in highly congested clusters.
This work has high practical impact for organizations training large-scale LLMs, as it reduces the cost and time of cluster configuration. It bridges the gap between theoretical performance modeling and practical engineering constraints. The use of LLMs for systems optimization is a growing trend, and this paper provides a concrete, high-fidelity example of its application. It could influence how future distributed training frameworks are designed, potentially integrating such projection oracles natively. The paper presents a highly effective hybrid system for optimizing distributed LLM training configurations by combining single-node micro-benchmarks with analytical simulation and LLM-driven search. Its rigorous validation on production-scale models and significant throughput improvements demonstrate substantial practical value for large-scale AI infrastructure, though its impact is somewhat constrained by its specific validation on AMD hardware.
Language captures only a subset of the perceptions, intuitions and intents that form meaning between speakers. What, then, are the limits of language? We take steps towards measuring this in LLMs as follows: for a set of concepts, we measure how well each concept can be transmitted between two LLMs as their capabilities scale. This requires novel measurement tools: (1) optimal messages: we introduce a prompt optimization algorithm that beats existing algorithms by up to 14% on benchmarks and accurately estimates prompt upper bounds. (2) A transmission measure: the sender is asked to label text; the receiver then tries to recover the sender's prompt, and recovery is measured; we find this correlates strongly with human label error. With these measures, we use observational scaling laws to study how transmissibility of a concept, conveyed with optimal messages, increases as transmitter and receiver capacity grows: does it asymptote below or above a threshold? Measuring 1.2k concepts in eight domains, we find concepts that (a) can be fully articulated; these concepts usually depend on local patterns, e.g. "joke ends with surprise"; (b) might be articulated by future, stronger models; these rely on whole-text structural patterns, e.g. "story's worldbuilding coherence"; (c) concepts that face articulability limits; these concepts' meaning, we find, is tied to identity and social-group awareness, e.g. "comic voice". Interesting, these same concepts are not all challenging for humans: humans have moderate-to-high levels of agreement in 76% of concepts that LLMs struggle with; articulation limits, we find, are executor relative. Finally, we characterize the nature of improvement in each case: while recoverable concepts can be articulated through definitions and explanations, irrecoverable concepts cannot be transmitted, even with examples, and might require novel learning mechanisms.
Primary: Stanford University
All Institutions: Stanford University
The paper introduces a rigorous framework for measuring the limits of language articulation in LLMs, combining novel prompt optimization with a transmission metric to categorize concepts by their articulability. By demonstrating that certain social and identity-based concepts are fundamentally harder for LLMs to transmit than others, despite high human agreement, the work provides critical insights into the boundaries of language-based AI control and alignment.
The paper introduces a novel framework for measuring the "articulability" of concepts in LLMs, decomposing the problem into message optimization and transmission measurement. The core methodological contribution is $\epsilon$-estimation, a prompt optimization algorithm that uses capture-recapture statistics to estimate the upper bound of prompt performance by modeling the space of "semantic units." This is a creative application of ecological statistics to prompt engineering. The second contribution is the "reconstruction bottleneck," a transmission metric where a receiver LLM attempts to reconstruct the sender's prompt from labeled examples. The methodology is rigorous, with clear definitions of upper/lower bounds (planted rules, permutation nulls) and a strong theoretical grounding in information theory (mutual information). The approach effectively separates the capability of the model from the clarity of the language used to convey a concept.
The experiments are extensive, covering 1.2k concepts across eight diverse domains (humor, math, code, etc.). The paper compares $\epsilon$-estimation against SOTA prompt optimizers like GEPA, showing up to 14% improvement. The scaling laws analysis is particularly strong, categorizing concepts into three regimes: fully articulable, potentially articulable with scaling, and fundamentally limited. The finding that "identity and social-group awareness" concepts are the hardest to articulate, while humans show high agreement on them, is a significant and surprising empirical result. The use of human inter-rater agreement (Krippendorff's alpha) as a ground truth for concept clarity adds robustness.
The paper provides detailed descriptions of the unit mining process, the capture-recapture estimator, and the reconstruction protocol. The use of standard benchmarks (AIME, HotpotQA, IFBench) and public datasets (McSweeney's, Reddit) aids reproducibility. However, the specific implementation details of the "unit" extraction and the exact prompt templates for the reconstruction bottleneck are not fully specified in the provided text, which may hinder exact replication. The code availability is not explicitly stated in the snippet, but the level of detail suggests it is likely released.
The primary limitation is the reliance on LLMs as both transmitters and receivers, which may not fully capture human-like articulation limits. The "unit" decomposition assumes that prompts can be atomized into independent semantic units, which may not hold for highly contextual or stylistic concepts. The capture-recapture estimator assumes exchangeability of units, which is a strong assumption. Additionally, the paper focuses on binary labeling tasks, which may not generalize to more complex, multi-class, or generative articulation tasks.
This paper has significant implications for AI alignment and interpretability. By identifying which concepts are "inarticulable" to LLMs, it highlights the limits of using language as a medium for control and evaluation. This is crucial for developing AI systems that operate on nuanced, subjective, or socially constructed concepts. The framework could be used to design better rubrics for LLM judges and to identify areas where human feedback is essential. It also provides a new lens for understanding the gap between human and machine understanding of language. The paper introduces a rigorous framework for measuring the limits of language articulation in LLMs, combining novel prompt optimization with a transmission metric to categorize concepts by their articulability. By demonstrating that certain social and identity-based concepts are fundamentally harder for LLMs to transmit than others, despite high human agreement, the work provides critical insights into the boundaries of language-based AI control and alignment.
Recent work has sought to transfer first-order nonconvex optimization guarantees to zeroth-order methods through gradient estimators, typically incurring a factor linear in $d$. \citet{kornowski2024} obtained an $\mathcal{O}(dδ^{-1}ε^{-3})$ oracle complexity for nonsmooth nonconvex optimization and asked whether its dependence on all three parameters is jointly optimal. Although the dimension dependence and the first-order dependence on $(δ,ε)$ are separately known to be unavoidable, a matching zeroth-order lower bound has remained open. We answer this question in the unrestricted full-space setting. For smooth stochastic nonconvex optimization, we establish a scalar-oracle lower bound of $Ω\!\left(dL_0^2\min\{ΔH,L_0^2\}/ε^4\right)$ assuming a smooth, globally lower-bounded population objective and a second-moment bound on sample Lipschitz constants. A smooth reduction then yields a lower bound for Goldstein stationarity, proving the joint minimax optimality of the dependence on $(d,δ,ε)$ under explicit high-dimensional and small-accuracy conditions. Crucially, our results allow algorithms to query arbitrary points in $\mathbb{R}^d$, adaptively select sample-function labels, reuse the same sampled function at any number of locations, and output points without spatial restrictions. Thus, unlike lower bounds based on bounded query regions or fresh evaluation noise, our guarantees apply to a fully adaptive, reusable-sample scalar oracle. We also establish a separate noiseless exact-value lower bound against randomized algorithms, and hence also against deterministic methods. Our proof combines a globally defined sequential nonconvex construction with an exact Gaussian scalar channel, multistage random-field estimates, and an antithetic finite-sample realization that preserves information hardness under adaptive reuse.
Primary: Nanjing University
All Institutions: Nanjing University, State Key Laboratory for Novel Software Technology
The paper establishes a joint minimax lower bound for stochastic zeroth-order nonconvex optimization, proving that the dependence on dimension, smoothness, and accuracy is jointly optimal under a fully adaptive, reusable-sample scalar oracle model. By constructing a global zero-chain with hidden workspace directions and employing information-theoretic tools to control adaptive sample reuse, the authors demonstrate that the oracle complexity scales as $\Omega(d L_0^2 \min\{\Delta H, L_0^2\} / \epsilon^4)$, resolving a key open question in the field and providing a rigorous theoretical limit for gradient-free optimization methods.
The paper addresses a significant gap in zeroth-order optimization theory: establishing a joint lower bound for the dependence on dimension ($d$), smoothness ($H$), and accuracy ($\epsilon$) in the stochastic nonconvex setting. The authors propose a novel construction that combines a global zero-chain geometry with a "moving well" mechanism to force sequential information acquisition. The core technical contribution is the proof that even with a fully adaptive, reusable-sample scalar oracle (where algorithms can query any point and reuse sample functions), the complexity scales as $\Omega(d L_0^2 \min\{\Delta H, L_0^2\} / \epsilon^4)$. The methodology is rigorous, utilizing Gaussian process representations, antithetic sampling to create a finite static oracle, and information-theoretic arguments (specifically spherical concentration and log-Sobolev inequalities) to bound the information gain per query. The construction is complex, involving careful management of global Lipschitz moments and spatial cutoffs to ensure the hardness persists globally rather than just locally.
As a pure theory paper, there are no empirical experiments. The "evaluation" is the mathematical proof of the lower bound. The rigor of the proof is the primary metric of quality here. The paper provides detailed appendices (referenced but not fully visible in the snippet, though the main text outlines the structure) to support the claims. The lack of empirical validation is standard for this subcategory of optimization theory.
The paper is highly reproducible in the sense that the mathematical results are deterministic and verifiable by other theorists. There is no code to reproduce. The constants and conditions are explicitly stated, allowing for independent verification of the bounds.
The lower bound holds under specific high-dimensional and small-accuracy conditions (explicitly stated as $d \ge C(B/\epsilon^2)(2+BL_0^2/\epsilon^4)$). The bound is for the smooth stochastic case, and the extension to nonsmooth Goldstein stationarity is via a reduction, which may not be tight in all regimes. The construction is quite intricate, making it difficult to generalize to other oracle models without significant modification. The constants involved are conservative (e.g., $2^{128}$), which limits immediate practical utility for tight algorithm design but is typical for lower bound constructions.
This paper settles an open question regarding the joint optimality of zeroth-order optimization complexities. It provides a definitive answer to whether the linear dependence on dimension $d$ is unavoidable in the stochastic nonconvex setting when using scalar oracles. This has implications for the design of gradient-free optimization algorithms, confirming that one cannot hope for sub-linear dimension dependence in the worst case under these assumptions. It strengthens the theoretical foundation for understanding the limitations of zeroth-order methods compared to first-order methods. The paper establishes a joint minimax lower bound for stochastic zeroth-order nonconvex optimization, proving that the dependence on dimension, smoothness, and accuracy is jointly optimal under a fully adaptive, reusable-sample scalar oracle model. By constructing a global zero-chain with hidden workspace directions and employing information-theoretic tools to control adaptive sample reuse, the authors demonstrate that the oracle complexity scales as $\Omega(d L_0^2 \min\{\Delta H, L_0^2\} / \epsilon^4)$, resolving a key open question in the field and providing a rigorous theoretical limit for gradient-free optimization methods.
We introduce an approach for self-supervised representation learning designed to be used as a tokenizer for generative modeling. Our objective supervises a representation loss by factorizing the pointwise mutual information of the data, which we estimate by taking information differences between conditional predictions from a learned generative model. We justify our algorithm theoretically and empirically, demonstrating that our generative approach to representation learning avoids dimensional collapse and feature suppression in cases where commonly used methods like DINO and LeJEPA fail. Moreover, we show our representation learning technique is scalable; expending more compute leads to predictably better latent tokenization. For the first time, we perform a careful analysis of the combined flop efficiency of tokenizer and generative model and demonstrate that our approach is a compute-optimal improvement over direct diffusion for pixel-space generation, obtaining an FID-50K of 2.18 with JiT-L. Applied to ImageNet and the text-to-image dataset GPIC, we show tokenizer encoder scaling laws for diffusability, where extrapolating our approach achieves representations in 7x fewer FLOPs than DINOv2 on ImageNet and 8x fewer FLOPs than WebSSL on GPIC.
Primary: Massachusetts Institute of Technology
All Institutions: Massachusetts Institute of Technology
[One sentence main contribution]. The paper introduces RePR, a generative representation learning method that factorizes Pointwise Mutual Information to create scalable, feature-suppression-robust tokenizers, demonstrating that this approach yields compute-optimal improvements over direct pixel diffusion for image generation.
The paper proposes RePR (Relative Prediction Representations), a self-supervised representation learning method that explicitly factorizes the Pointwise Mutual Information (PMI) of the data using a generative model to estimate PMI. The core innovation is replacing the discriminative InfoNCE loss (which relies on negative sampling and is prone to feature suppression) with a generative objective that estimates PMI via the ELBO of a diffusion model. The method introduces "Timestep Weighting" to bias the PMI estimation toward semantically relevant noise scales and "Bootstrapped PMI Estimation" to make the likelihood estimation tractable for high-dimensional inputs by decomposing the PMI into incremental token predictions. The architecture uses a Masked Autoencoder (MAE) variant with a diffusion head for the prediction model and a separate representation encoder. The theoretical grounding is strong, linking the method directly to the information-theoretic properties of contrastive learning while avoiding its practical pitfalls.
The experiments are rigorous and address a critical gap in the field: the compute efficiency of tokenizers. The authors demonstrate that RePR encoders exhibit predictable scaling laws for "diffusability" (how well a generative model can learn from the latent space). They show that RePR achieves DINOv2-level performance with 7x fewer FLOPs on ImageNet and 8x fewer than WebSSL on GPIC. A key result is the demonstration that using RePR with Latent Forcing yields a better FID-50K (2.18) than direct pixel diffusion (JiT-L) at equal total training FLOPs, proving that representation learning is a compute-optimal component of the generative pipeline. The ablation studies on masking strategies, loss weights, and iterative feature prediction are thorough. The feature suppression experiment using watermarked images effectively highlights the robustness of the generative PMI objective compared to contrastive baselines like DINOv3 and LeJEPA.
The paper provides a GitHub link and detailed implementation notes in the appendix, including architectural specifics (RoPE, register tokens, layer splits) and training hyperparameters. The use of standard components (ViT, MAE, Diffusion) aids reproducibility. However, the specific "Latent Forcing" method is cited as a separate work (Baade et al.), which may introduce dependency on external codebases not fully detailed here. The FLOP calculations are explicitly defined, which is crucial for the claims made.
The method relies on a diffusion model for PMI estimation, which adds computational overhead during the representation learning phase, although the paper argues this is offset by downstream efficiency. The scaling laws are extrapolated, and the "7x/8x" claims depend on the accuracy of these power-law fits. The method is primarily evaluated on image generation; applicability to other modalities (audio, video) is mentioned but not extensively tested. The reliance on a specific masking strategy and timestep weighting scheme requires careful tuning.
This paper has high potential impact by providing a theoretically grounded, scalable alternative to contrastive representation learning for generative tasks. It challenges the prevailing assumption that discriminative objectives are sufficient for tokenizers and offers a path to more compute-efficient generative models. The demonstration of compute-optimal generation via learned representations could influence the design of future foundation models, encouraging a shift towards generative self-supervision for representation learning. [One sentence main contribution]. The paper introduces RePR, a generative representation learning method that factorizes Pointwise Mutual Information to create scalable, feature-suppression-robust tokenizers, demonstrating that this approach yields compute-optimal improvements over direct pixel diffusion for image generation.
Textual reasoning before acting has improved vision-language-action (VLA) models, but at the cost of latency: every action conditioned on the reasoning text must wait for autoregressive decoding. We show that actions do not need to wait for or condition on the generated reasoning to benefit from it, because the reasoning supervision already improves the backbone representation. We introduce TacitVLA, a reasoning VLA whose action expert never attends to the reasoning it generates for the current call. On long-horizon tasks where the reasoning text also serves as episodic memory, it can generate each entry during action execution rather than before acting. We evaluate it in a controlled study on four benchmarks spanning manipulation and autonomous driving, memory and non-memory tasks: RoboMME, RMBench, LIBERO, and PhysicalAI-AV. On all four, TacitVLA matches the performance of its explicit-reasoning counterpart at up to 3.4x lower latency, and matches the latency of the same architecture trained without reasoning at up to +17.1% success. In absolute terms, it surpasses the best previously published results on the RoboMME Counting suite (82.7% vs. 66.8%) and RMBench (77.5% vs. 56.5%), and reaches 72.9% on LIBERO-Plus and a 52.5% scene score on the public split of the PhysicalAI-AV closed-loop driving challenge. Project page: https://tacitvla.github.io.
Primary: NVIDIA
All Institutions: NVIDIA
TacitVLA demonstrates that reasoning supervision improves VLA performance through representation learning rather than explicit inference-time conditioning, enabling real-time deployment with up to 3.4x lower latency. This paper provides a rigorous, controlled study across manipulation and driving benchmarks, showing that masking the action expert's attention to reasoning tokens retains the performance benefits of reasoning while eliminating the autoregressive decoding bottleneck, a significant contribution to the efficiency of embodied AI systems.
The paper proposes TacitVLA, a method that decouples the *generation* of reasoning text from the *conditioning* of actions on that text. The core insight is that the benefit of reasoning in Vision-Language-Action (VLA) models primarily stems from the representation learning induced by the reasoning supervision signal (cross-entropy loss on text tokens) during training, rather than the explicit attention to the generated text tokens during inference. By masking the action expert's attention to the reasoning tokens during both training and inference, the model retains the improved backbone representations without incurring the autoregressive decoding latency. For memory tasks, the paper introduces an asynchronous generation strategy where the reasoning text (ledger entry) is generated during the execution of the current action chunk, ensuring it is available for the next call without blocking the current action. This is a clever architectural optimization that leverages the existing Mixture-of-Transformers structure.
The evaluation is rigorous and comprehensive, covering four distinct benchmarks: RoboMME (memory/counting), RMBench (long-horizon manipulation), LIBERO/LIBERO-Plus (standard manipulation/robustness), and PhysicalAI-AV (autonomous driving). The controlled study comparing Explicit Reasoning, TacitVLA, and No Reasoning is well-designed, isolating the effect of attention masking. The results show that TacitVLA matches the performance of Explicit Reasoning while achieving the latency of the No Reasoning baseline (or slightly higher due to deferred generation on memory tasks). The latency improvements are significant (up to 3.4x), which is critical for real-time deployment. The performance gains over the No Reasoning baseline (up to +17.1% success) confirm the value of the reasoning supervision.
The paper provides detailed implementation details in the appendix, including architecture specifics, training hyperparameters, data sources, and evaluation protocols. The use of standard benchmarks (LIBERO, RoboMME, RMBench) and public datasets (PhysicalAI-AV public split) enhances reproducibility. The code and project page are available. However, the driving dataset is proprietary (3.8M events), which limits full reproducibility for that specific benchmark, though the methodology is clear.
The primary limitation is the reliance on high-quality reasoning annotations for training. If the reasoning data is noisy or incorrect, the representation learning benefit may diminish. Additionally, the asynchronous generation on memory tasks assumes that the generation time fits within the action execution window; if the reasoning text becomes very long or the hardware is slower, this assumption may break, leading to latency spikes. The paper also notes that on non-memory tasks, the reasoning text is discarded, which might miss opportunities for interpretability or safety checks that could be performed post-hoc.
This work has significant implications for the deployment of VLA models in real-world robotics and autonomous driving, where latency is a critical constraint. By demonstrating that reasoning can be "internalized" into the model's representations without the inference cost, it paves the way for more efficient and responsive embodied AI systems. The approach is generalizable to other architectures that use reasoning as a training target, potentially influencing future designs of foundation models for robotics. TacitVLA demonstrates that reasoning supervision improves VLA performance through representation learning rather than explicit inference-time conditioning, enabling real-time deployment with up to 3.4x lower latency. This paper provides a rigorous, controlled study across manipulation and driving benchmarks, showing that masking the action expert's attention to reasoning tokens retains the performance benefits of reasoning while eliminating the autoregressive decoding bottleneck, a significant contribution to the efficiency of embodied AI systems.
Generating information-rich visual artifacts such as slides, posters, and dashboards as code is appealing because the outputs are structured, editable, and verifiable. Yet code models cannot see how their choices of layout and color come together on the canvas, often limiting spatial planning and aesthetic quality. Image generators excel at visual design but can misrender requested content, making their outputs unreliable targets for direct reproduction. We therefore introduce AesCode, which generates code grounded in the content prompt while using an image generated from the same prompt as an aesthetic guidance. To do so, AesCode represents the target as a design graph that describes the whole canvas and makes each property attributable. From this graph, we derive complementary cross-modal rewards: deterministic verifiers score properties parsable from code and rendering, while a VLM judge assesses the non-parsable ones with a sample-specific Visual Graph Rubric (VGR) tied to graph elements and relations. We optimize these decoupled rewards through reinforcement learning with per-channel normalization to jointly improve spatial planning, aesthetic quality, and verifiable correctness. To evaluate this setting, we build a benchmark that scores verifiable correctness and visual quality separately, with visual scores highly consistent with human judgments. We find that reference images substantially improve the visual quality of code generated by existing models, and ablations show that AesCode benefits from references both as model input and as a source of visual rewards. Relative to same-scale Qwen3-VL baselines given the same references, AesCode-8B and AesCode-32B improve visual quality by 31.1 and 22.4 percentage points, reaching scores comparable to those of closed-source frontier models such as GPT-5.5 and Claude Opus 4.8.
Primary: Microsoft
All Institutions: Microsoft, Yale University, Nanyang Technological University, University of Pittsburgh, Shanghai Jiao Tong University, Fudan University
AesCode introduces a decoupled cross-modal reward system using a Design Graph representation to optimize aesthetic code generation via reinforcement learning. The paper presents a rigorous methodology that effectively combines deterministic verification with VLM-based aesthetic judgment, achieving state-of-the-art results that rival frontier closed-source models, thereby validating the potential of code-based approaches for high-fidelity visual design.
The paper proposes AesCode, a framework for generating visual artifacts (slides, posters) as code. The core innovation is the "Design Graph" representation, which decomposes the visual canvas into attributable properties. This allows for a decoupled reward system: deterministic verifiers check parsable properties (e.g., text presence, layout constraints), while a Vision-Language Model (VLM) judge assesses non-parsable aesthetic qualities using a sample-specific Visual Graph Rubric (VGR). The training utilizes Reinforcement Learning (RL) with per-channel normalization to optimize these distinct reward signals simultaneously. This approach effectively bridges the gap between the structural reliability of code generation and the aesthetic fidelity of image generation.
The authors introduce a new benchmark that separates verifiable correctness from visual quality, noting high consistency with human judgments. Experiments show that AesCode-8B and AesCode-32B outperform same-scale Qwen3-VL baselines by 31.1 and 22.4 percentage points in visual quality, respectively, when given reference images. The results claim parity with closed-source frontier models like GPT-5.5 and Claude Opus 4.8. The ablation studies confirm the utility of reference images both as input context and as a source for visual rewards.
The paper provides a project page and likely code release (implied by the GitHub URL structure). The methodology is detailed enough to understand the RL setup and reward decomposition. However, the reliance on a VLM judge for aesthetic scoring introduces a dependency on the specific VLM used, which may vary in robustness.
The primary limitation is the dependency on high-quality reference images for optimal performance; without them, the aesthetic guidance is lost. Additionally, the "Design Graph" construction may be complex for highly unstructured or free-form artistic layouts, potentially limiting generalizability to strict UI/dashboard design. The evaluation relies heavily on VLM-based scoring, which, while consistent with humans, is not a ground-truth metric.
This work has significant implications for the "code-as-design" paradigm, making it viable for professional-grade visual content creation. By decoupling aesthetic and functional rewards, it offers a template for other multimodal generation tasks where structural integrity and perceptual quality are distinct objectives. It could drive adoption of code-based design tools in enterprise settings where editability is crucial. AesCode introduces a decoupled cross-modal reward system using a Design Graph representation to optimize aesthetic code generation via reinforcement learning. The paper presents a rigorous methodology that effectively combines deterministic verification with VLM-based aesthetic judgment, achieving state-of-the-art results that rival frontier closed-source models, thereby validating the potential of code-based approaches for high-fidelity visual design.
Terminal agents have shown strong abilities in complex coding tasks. However, they often struggle to recover from errors during long-running execution (e.g., command errors, faulty implementations), eventually leading to task failures. To this end, we present ReTerm, a self-improving terminal agent trained by learning how to recover from its own failed terminal tasks. To support its training, we construct a customized data pipeline and produce ReTerm-10K, a high-quality dataset that provides effective recovery demonstrations for SFT cold start. Furthermore, we propose ReTerm-GRPO, an online RL algorithm that forks from the agent's own failure states to generate multiple independent self-recovery attempts. By learning from whether these attempts successfully complete the task, the agent self-improves through its own recovery experience. Built on Qwen3.5-9B, our ReTerm-9B achieves 34.8% Pass@1 on Terminal-Bench 2.0, improving over the base model by 14.6 points. Further analysis shows that our method achieves consistently higher recovery rates across diverse error categories compared with standard GRPO training. All code, data and model are released at https://github.com/twothsnd/ReTerm.
Primary: Shanghai Jiao Tong University
All Institutions: Fudan University, MMLab, Shanghai Jiao Tong University, The Chinese University of Hong Kong
ReTerm introduces a recovery-oriented training framework for terminal agents, combining a curated dataset of recovery trajectories with an online RL algorithm that learns from self-generated failure states. The paper demonstrates that explicitly training agents to recover from errors leads to substantial performance gains on terminal and coding benchmarks, offering a new perspective on agent self-improvement beyond simple task completion.
The paper proposes ReTerm, a framework for training terminal agents to recover from their own execution errors. The methodology consists of two main components: (1) ReTerm-10K, a dataset constructed by collecting failed trajectories from a base model (Qwen3.5-9B), restoring the environment state, and using a stronger teacher model (DeepSeek-V4-Pro) to generate recovery continuations, which are then filtered for quality; and (2) ReTerm-GRPO, an online reinforcement learning algorithm that forks from the agent's own failure states during training to generate multiple independent self-recovery attempts. The use of a "recovery task queue" to manage the distribution of failure states across training steps is a practical engineering contribution that addresses the sparsity of failure events in RL. The approach is logically sound, addressing the specific gap of "recovery" rather than just "task completion," and the distinction between SFT cold-start and online RL self-improvement is well-motivated.
The experiments are comprehensive, evaluating the model on Terminal-Bench 2.0, OpenThoughts-TBLite, SWE-bench Verified, and out-of-domain tasks like AIME and SpreadsheetBench. The results show significant improvements over the base model (Qwen3.5-9B) and standard GRPO baselines. The ablation study clearly isolates the contribution of the recovery-specific RL component. The error analysis section, which categorizes errors into planning, tool use, implementation, verification, and resource management, provides valuable insight into where the model improves. The case studies are detailed and illustrative of the recovery process. The performance gains are substantial (e.g., +14.6 points on Terminal-Bench 2.0), suggesting a meaningful impact on the state of the art for 9B-scale models.
The authors release code, data, and models, which is a strong positive for reproducibility. The paper provides detailed descriptions of the data pipeline, filtering criteria, and training hyperparameters. The use of specific model versions (Qwen3.5-9B, DeepSeek-V4-Pro) and benchmarks (Terminal-Bench 2.0) allows for clear replication. The prompt templates for data curation and error analysis are included in the appendix, further enhancing transparency.
The method relies on a stronger teacher model (DeepSeek-V4-Pro) for the SFT cold-start phase, which may limit scalability if such a teacher is not available. The recovery task queue mechanism adds complexity to the training loop. The evaluation is primarily on terminal/coding tasks; while out-of-domain generalization is tested, the core contribution is specific to interactive terminal environments. The paper does not extensively discuss the computational cost of maintaining and sampling from the recovery task queue compared to standard RL.
This work has significant implications for the development of autonomous coding agents. By focusing on recovery, it addresses a critical failure mode in long-horizon tasks. The techniques could be extended to other interactive domains where agents must correct their own mistakes, such as robotics or GUI agents. The release of the ReTerm-10K dataset and the ReTerm-GRPO algorithm provides a valuable resource for the community. ReTerm introduces a recovery-oriented training framework for terminal agents, combining a curated dataset of recovery trajectories with an online RL algorithm that learns from self-generated failure states. The paper demonstrates that explicitly training agents to recover from errors leads to substantial performance gains on terminal and coding benchmarks, offering a new perspective on agent self-improvement beyond simple task completion.
Video diffusion models (VDMs) trained on internet-scale data encode a rich prior on appearance, motion, and physics that is difficult to replicate with explicit 3D rendering and costly 3D data capture. We introduce Video Diffusion Avatars (VDAs), a method that creates photo-realistic, multi-view consistent avatars from a single image by fine-tuning a VDM for precise control over camera viewpoint, facial expression, and upper-body pose. Departing from 3D morphable models (3DMMs), VDAs use a learned expression space that lifts the expressiveness ceiling of 3DMMs, e.g., for the tongue and mouth interior. However, learned expression codes exhibit heavy viewpoint bias, as our DISTRACT benchmark reveals. We show that this bias can be removed through disentanglement, enabling their use as driving signals for multi-view consistent avatars. Camera and upper-body pose are controlled jointly via proxy-mesh renderings, giving independent control over expression, pose, and viewpoint. A camera-triplet training strategy makes the approach efficient: our models train for 34 hours on a single A100 using only public multi-view data. VDAs outperform state-of-the-art single-image 3D avatars on novel-view reenactment, reducing face LPIPS by 29% and improving perceptual quality (JOD) by 66% on BecomingLit. A static 3DGS reconstruction probe further confirms that our generations are the most multi-view consistent. Built on a video prior, VDAs produce effects beyond the reach of explicit renderers, including physically plausible hair dynamics, shadows, reflections, and refractions.
Primary: University of Tokyo
All Institutions: University of Tokyo, Toyota Motor Corporation
The paper introduces Video Diffusion Avatars (VDAs), a method that fine-tunes video diffusion models to create photo-realistic, multi-view consistent 3D avatars from a single image with precise control over expression, pose, and viewpoint. By leveraging the rich priors of VDMs and addressing viewpoint bias through disentanglement, VDAs outperform existing methods in perceptual quality and consistency, enabling physically plausible effects like hair dynamics and lighting that are difficult to achieve with explicit 3D rendering.
The paper proposes Video Diffusion Avatars (VDAs), a method that leverages the rich priors of video diffusion models (VDMs) to generate controllable, multi-view consistent 3D avatars from a single image. The core innovation lies in replacing explicit 3D morphable models (3DMMs) with a learned expression space, which allows for higher expressiveness (e.g., tongue and mouth interior) but introduces viewpoint bias. The authors address this via a disentanglement strategy, enabling the learned codes to serve as robust driving signals. Control over camera and pose is achieved through proxy-mesh renderings, and a "camera-triplet" training strategy ensures efficiency (34 hours on a single A100). This approach effectively bridges the gap between the photorealism of diffusion models and the geometric consistency required for 3D avatars.
The experiments demonstrate significant improvements over state-of-the-art single-image 3D avatar methods. On the BecomingLit dataset, VDAs reduce face LPIPS by 29% and improve perceptual quality (JOD) by 66%. A static 3D Gaussian Splatting (3DGS) reconstruction probe confirms superior multi-view consistency compared to baselines. The paper also highlights qualitative advantages such as physically plausible hair dynamics and lighting effects (shadows, reflections) that are difficult to achieve with explicit renderers. The introduction of the DISTRACT benchmark to reveal viewpoint bias in learned expression codes is a valuable diagnostic contribution.
The paper provides specific training details (34 hours on A100, public multi-view data), which aids reproducibility. However, without explicit links to code or detailed hyperparameter tables in the provided text, full reproduction may be challenging. The reliance on fine-tuning a VDM suggests that access to a strong base model is critical.
The method requires fine-tuning a video diffusion model, which may be computationally intensive for non-experts despite the claimed efficiency. The reliance on proxy-mesh renderings for pose control might limit the range of extreme poses not well-represented in the proxy. The learned expression space, while more expressive than 3DMMs, may still struggle with very rare or out-of-distribution expressions.
This work has significant implications for the field of 3D human modeling, potentially shifting the paradigm from explicit geometric modeling to implicit diffusion-based generation. It enables new applications in virtual reality, film production, and social media where high-fidelity, controllable avatars are needed. The ability to generate physically plausible effects like hair dynamics and lighting from a single image is a major step forward in content creation. The paper introduces Video Diffusion Avatars (VDAs), a method that fine-tunes video diffusion models to create photo-realistic, multi-view consistent 3D avatars from a single image with precise control over expression, pose, and viewpoint. By leveraging the rich priors of VDMs and addressing viewpoint bias through disentanglement, VDAs outperform existing methods in perceptual quality and consistency, enabling physically plausible effects like hair dynamics and lighting that are difficult to achieve with explicit 3D rendering.
Video world models that generate and explore environments at scale need a persistent 3D memory of what they have produced. Plane primitives offer a natural candidate: a compact set of oriented planes captures the dominant structural elements (e.g., walls, floors, facades) of indoor and outdoor scenes, while requiring only 7 parameters per primitive. We present PPMem, a video world model that adopts plane primitives as its persistent 3D memory. PPMem augments a video diffusion transformer with a planar prediction head and a Depth Token Fusion (DTF) module: each autoregressive chunk jointly produces RGB video and planar geometry, which merges into a persistent plane memory that conditions subsequent chunks via rendered depth. The resulting memory is three orders of magnitude more compact than per-pixel point clouds (~100K primitives, 5MB for a full scene), geometrically explicit (metric depth, normals, exportable surfaces), cross-view consistent (jointly predicted by the shared backbone), and adaptive (refined by the merging procedure as generation proceeds). On DL3DV and RealEstate10K, PPMem outperforms prior 3D-grounded world models in video quality while jointly producing metric-scale depth, surface normals, and exportable planar geometry, sustaining coherence across hundreds of frames under large viewpoint changes. Project page: https://hyzcluster.github.io/ppmem.
Primary: Tsinghua University
All Institutions: Tsinghua University, Ant Group
PPMem introduces a compact planar memory for video world models, enabling high-quality, geometrically consistent video generation with exportable 3D structures. The paper presents a novel integration of planar primitives into a video diffusion transformer, offering a significant improvement in memory efficiency and geometric fidelity over prior point-cloud-based approaches, with strong experimental results on standard benchmarks.
The paper proposes PPMem, a video world model that integrates planar primitives as a persistent 3D memory. The core architectural contribution is the addition of a planar prediction head to a video diffusion transformer, which jointly predicts RGB video and planar geometry (position, normal, size) for each autoregressive chunk. A key component is the Depth Token Fusion (DTF) module, which merges the newly predicted planes into a persistent memory bank. This memory is then rendered into a depth map to condition subsequent video generation steps. The approach leverages the fact that real-world scenes (especially indoor and urban) are dominated by planar structures, allowing for a highly compact representation (~100K primitives, 5MB) compared to dense point clouds. The method ensures cross-view consistency by deriving geometry from the same backbone that generates the video, avoiding the decoupling issues seen in separate geometry estimation pipelines.
The authors evaluate PPMem on the DL3DV and RealEstate10K datasets. The experiments demonstrate that PPMem outperforms prior 3D-grounded world models in video quality metrics (likely FVD or similar, though specific metrics are not detailed in the abstract, the claim of "outperforms" is made). Crucially, the paper highlights the efficiency of the memory representation, showing a three-order-of-magnitude reduction in storage compared to per-pixel point clouds. The results indicate sustained coherence across hundreds of frames, even under large viewpoint changes, which is a significant challenge for video world models. The joint production of metric-scale depth, surface normals, and exportable planar geometry is a strong practical advantage, as these outputs are directly usable for downstream 3D reconstruction or simulation tasks.
The paper provides a project page with a link, which likely contains code and additional visualizations. The description of the DTF module and the planar prediction head is sufficiently detailed in the abstract to suggest that the core components are well-defined. However, without access to the full code, the specific hyperparameters for the planar prediction head and the merging strategy for the memory bank are not fully transparent. The use of standard datasets (DL3DV, RealEstate10K) aids in reproducibility.
The reliance on planar primitives may limit the model's ability to accurately represent non-planar structures, such as organic shapes, complex foliage, or irregular objects. While the paper notes that planes capture dominant structural elements, scenes with high geometric complexity may suffer from artifacts or inaccuracies in the generated video and geometry. Additionally, the performance on outdoor scenes with less regular structure might be less consistent than on indoor scenes. The computational cost of rendering the depth map from the persistent memory for conditioning could be a bottleneck for real-time applications, although the compactness of the memory mitigates this to some extent.
This work has significant implications for the development of scalable video world models. By providing a compact, geometrically explicit, and exportable memory representation, PPMem bridges the gap between video generation and 3D scene understanding. This could enable new applications in virtual reality, robotics, and autonomous driving, where persistent and consistent 3D environments are required. The ability to export metric-scale geometry directly from the video generation process could streamline pipelines for 3D content creation and simulation. PPMem introduces a compact planar memory for video world models, enabling high-quality, geometrically consistent video generation with exportable 3D structures. The paper presents a novel integration of planar primitives into a video diffusion transformer, offering a significant improvement in memory efficiency and geometric fidelity over prior point-cloud-based approaches, with strong experimental results on standard benchmarks.
City-scale 3D grounding is commonly formulated as candidate ranking, which requires precomputed instance proposals and becomes inefficient in large outdoor scenes. We propose Urbex, an agentic vision-language framework that instead casts grounding as active spatial search. Urbex represents a city-scale 3D scene as an interactive multi-view environment, allowing a VLM agent to query landmarks, zoom into local regions, render oblique views, and commit a grounded bounding box. We optimize this tool-use policy with reinforcement learning on CityRefer, using smooth localization rewards and lightweight shaping rewards for efficient exploration. Experiments show that Urbex achieves the best Hit@1 and IoU on CityRefer, supports candidate-free grounding without ground-truth boxes, and is substantially faster than the strongest city-scale ranking baseline. Zero-shot results on STPLS3D-Refer further indicate improved cross-scene robustness.
Primary: Shanghai AI Laboratory
All Institutions: Shanghai AI Laboratory, Tsinghua University
The paper introduces Urbex, an agentic framework that reformulates city-scale 3D grounding as active spatial search using a VLM agent and reinforcement learning. It demonstrates superior accuracy and efficiency compared to candidate-ranking baselines, offering a scalable solution for large outdoor 3D scene understanding.
The paper proposes "Urbex," a framework that reframes city-scale 3D grounding from a static candidate ranking problem to an active spatial search problem. The core novelty lies in the design of an interactive multi-view environment where a Vision-Language Model (VLM) agent uses specific tools (Landmark Query, Zoom-in, Oblique Rendering) to navigate the scene. The use of Reinforcement Learning (specifically GRPO) to optimize this tool-use policy is a strong methodological contribution, addressing the challenge of credit assignment in multi-step agentic tasks. The reward design, combining terminal IoU with distance-based shaping and tool-level rewards, is well-motivated for sparse reward environments. The approach effectively leverages the reasoning capabilities of modern VLMs while constraining the action space to manageable geometric operations.
The experiments are rigorous and comprehensive. The authors evaluate on CityRefer (in-domain) and STPLS3D-Refer (zero-shot cross-scene), covering both "With GT" (candidate ranking) and "Without GT" (candidate-free) settings. The results demonstrate significant improvements in Hit@1 and IoU over strong baselines like CityAnchor and GPT-5.4/Qwen3.6-plus prompted baselines. The efficiency analysis is particularly compelling, showing a 10x speedup over the strongest ranking baseline by avoiding exhaustive scoring. Ablation studies clearly isolate the contributions of individual tools and reward components, providing strong evidence for the design choices. The zero-shot generalization results are a key strength, suggesting the learned policy captures reusable spatial search strategies rather than just memorizing the training distribution.
The paper provides detailed descriptions of the environment, tools, and RL objective. However, as an arXiv preprint, specific code availability is not confirmed in the text. The reliance on specific VLM backbones (Qwen, GPT) and the complexity of the RL training loop (GRPO) may pose barriers to reproduction for groups without significant compute resources. The hyperparameters for the RL training and the specific implementation of the view-to-world transforms are described but would benefit from open-source code for full verification.
The method relies on the availability of named landmarks or distinct visual features for the "Landmark Query" tool, which may limit its applicability to scenes with poor semantic labeling or highly repetitive structures. The RL training process is computationally expensive compared to supervised fine-tuning. The "Oblique Rendering" tool requires pre-computed or on-the-fly rendering, which adds latency and storage overhead, although the paper addresses this with caching strategies. The performance on "Without GT" settings, while better than baselines, still shows room for improvement in precise box localization (high area ratio variance).
This work has significant implications for embodied AI and autonomous driving, where efficient and accurate localization of objects in large-scale environments is critical. By demonstrating that agentic search can outperform and outpace exhaustive ranking, it offers a scalable paradigm for handling large 3D scenes. The framework could be extended to other tasks requiring long-range spatial reasoning, such as robot navigation or aerial inspection. The integration of RL with VLM tool-use is a trend that this paper advances meaningfully in the 3D domain. The paper introduces Urbex, an agentic framework that reformulates city-scale 3D grounding as active spatial search using a VLM agent and reinforcement learning. It demonstrates superior accuracy and efficiency compared to candidate-ranking baselines, offering a scalable solution for large outdoor 3D scene understanding.
Multimodal code intelligence connects the symbolic world of programs with the visual world their execution creates. Interest in this capability is burgeoning, as models are increasingly asked to produce charts, webpages, animations, and semi-structured documents. Recently, proprietary models have pushed far ahead, producing interactive artifacts even from a text description or a screenshot alone. Yet their data and training remain closed, and open progress is held back by the quantity and quality of available data and by the pipelines and methods for producing it. Quality is the harder of the two, since a sample must be correct as a program, well built, and above all visually right once rendered. No existing signal covers all three. Execution often returns just a pass or fail, code similarity does not map to the visual output, and task-specific verifiers do not generalize. To push open progress further, we start from the data side with PrismaForge, a hierarchical data pipeline centered on a synthesized rubric that decomposes these factors into checkable criteria. The same rubric serves as both filter and reward: it curates the supervised corpus, then drives reinforcement learning on visual-coding tasks. Leveraging PrismaForge for data synthesis and filtering, we build PrismaCoder-2M, the most diverse open dataset for multimodal code intelligence and the largest to date. On the model side, we train PrismaCoder-9B and PrismaCoder-35B on this data and release both. They lead the open state of the art by a wide margin and rival leading proprietary systems. Our analyses further show that the proposed rubric-based pipeline improves multimodal coding performance across model backbones. We release our code and data at https://qiushisun.github.io/PrismaCoder-Home/.
Primary: University of California, San Diego (UCSD)
All Institutions: University of California, San Diego, Meta AI
The paper introduces PrismaForge, a data pipeline using decomposed rubric synthesis to generate high-quality multimodal code data, resulting in PrismaCoder models that rival proprietary systems. This is a significant contribution to the field of multimodal code intelligence, addressing the critical need for verifiable, high-quality open data and demonstrating that data-centric approaches can close the gap with closed-source models.
The paper proposes PrismaForge, a hierarchical data pipeline that addresses the lack of comprehensive quality signals in multimodal code generation (combining program correctness, code structure, and visual fidelity). The core innovation is the "decomposed rubric synthesis," where a synthesized rubric breaks down complex visual-coding tasks into checkable criteria. This rubric serves a dual purpose: it acts as a filter to curate a high-quality supervised corpus (PrismaCoder-2M) and as a reward signal for reinforcement learning. The method effectively bridges the gap between symbolic code and visual output by creating a verifiable intermediate representation. The approach is sound and addresses a critical bottleneck in open-source multimodal code models: the scarcity of high-quality, verifiable data.
The authors train PrismaCoder-9B and PrismaCoder-35B on the synthesized data. The results claim to lead the open state of the art by a wide margin and rival proprietary systems. The evaluation likely includes standard benchmarks for code generation (e.g., HumanEval, MBPP) and multimodal-specific tasks (e.g., rendering accuracy, visual similarity). The claim of "rivaling leading proprietary systems" is significant and, if substantiated by the full text (which is summarized here), would be a major empirical finding. The analysis section reportedly shows the pipeline improves performance across different model backbones, suggesting the data quality is the key driver rather than a specific architectural quirk.
The paper releases both the code and the data (PrismaCoder-2M), which is a significant contribution to the open-source community. The release of a 2M sample dataset for multimodal code intelligence is substantial and will likely be widely used. The detailed release of the rubric synthesis pipeline further enhances reproducibility.
The reliance on synthesized rubrics may introduce biases if the rubric generation model has blind spots. The computational cost of generating and filtering 2M samples with complex visual verification is likely high. Additionally, the generalization to tasks outside the training distribution (e.g., highly abstract UIs or non-standard visualizations) may be limited.
This work significantly advances the open-source ecosystem for multimodal code intelligence. By providing a large, high-quality dataset and a robust training pipeline, it lowers the barrier for other researchers and developers to build competitive multimodal code models. It also highlights the importance of data quality and verification pipelines in the era of large language models. The paper introduces PrismaForge, a data pipeline using decomposed rubric synthesis to generate high-quality multimodal code data, resulting in PrismaCoder models that rival proprietary systems. This is a significant contribution to the field of multimodal code intelligence, addressing the critical need for verifiable, high-quality open data and demonstrating that data-centric approaches can close the gap with closed-source models.
Long-horizon agents have drawn growing attention for their ability to work autonomously -- from hours to days -- on economically valuable tasks, a capability essential to automated discovery and recursive self-improvement (RSI). Yet open-source models still struggle on long-horizon tasks, largely for lack of both training data and a working recipe, and scaling both is hard: long, complex tasks are difficult to curate and verify, and their rewards are sparse. We address both. First, we introduce Tev-Gym, the first long-horizon agent training dataset with verifiable, dense, progressive rewards -- 1,170 verified task environments across 7 domains. It is built on a latent-world-first approach that first constructs a latent world: a complete and correct hidden specification of the task domain, workflow, and expected state after each step. Each task is then progressively rendered from this specification into an agent-facing environment, enabling controllable complexity and a natural progress metric. Second, we turn this metric into a training recipe: via progress-potential reward shaping (PPRS), we convert progress into a dense per-step reward that improves training efficiency and makes otherwise-intractable tasks trainable. Training with this recipe yields Tev-27B, which achieves substantial gains on the Agents' Last Exam computing and math sciences domains (ALE-CM), matching GPT-6 Astra and Claude Opus 5 at Pass@5. Despite training only on ALE-related data, it generalizes to the Terminal-Bench (TB) 3.0 and 4.0 subsets, surpassing GLM 5.2 and Gemini 3.7 Flash, and reaches Claude Opus 4.8-level performance on middle-horizon tasks such as TB 2.1.
Primary: University of California
All Institutions: University of California
The paper introduces a latent-world-first synthesis pipeline and Progress-Potential Reward Shaping (PPRS) to enable effective RL training for long-horizon agents, resulting in open-source models that match frontier closed-source systems on complex agentic benchmarks. This is a significant contribution that addresses the critical data and reward sparsity bottlenecks in agentic AI, offering a reproducible and scalable recipe for generating high-quality training environments and training signals.
The paper introduces a "latent-world-first" synthesis pipeline for creating long-horizon agent environments. This is a significant methodological contribution because it addresses the core bottleneck in agentic RL: the lack of verifiable, dense rewards for complex, multi-step tasks. By constructing a complete, correct hidden specification (the latent world) first and then rendering the task and corruption from it, the authors ensure that ground truth is known by construction. This allows for the derivation of a "progress potential" function, which is used in a novel reward shaping scheme called Progress-Potential Reward Shaping (PPRS). PPRS is theoretically grounded in potential-based reward shaping but adapted for GRPO to prevent reward starvation in groups where no rollout succeeds. The methodology is rigorous, with clear stages for world construction, workflow graphing, artifact rendering, and controlled corruption. The inclusion of qualification gates (exploit resistance, progress validity, role independence) demonstrates a high level of care in preventing reward hacking and ensuring data quality.
The experiments are extensive and well-designed. The authors train two model sizes (9B and 27B) and evaluate them on the Agents' Last Exam (ALE-CM) and Terminal-Bench (TB) 3.0/4.0. The results show substantial gains, with the 27B model matching frontier closed-source models (GPT-6 Astra, Claude Opus 5) on Pass@5 for ALE-CM. Crucially, the paper demonstrates generalization to unseen benchmarks (TB), which is a strong indicator of learning robust agentic behaviors rather than overfitting to the training distribution. The ablation studies likely confirm the importance of both the dense reward and the specific data synthesis method. The comparison against prior open-source datasets (SWE-Smith, etc.) highlights the complexity gap, with Tev-Gym tasks being significantly longer and harder.
The paper provides detailed descriptions of the synthesis pipeline, the reward shaping formula, and the training setup (DPPO, group size, token budgets). The "latent world" concept is well-defined, and the qualification gates are described formally. However, without access to the code or the specific generators for the 1,170 environments, full reproduction is difficult. The reliance on specific base models (Qwen3.5/3.8) and the proprietary nature of some benchmark subsets (if any) could limit reproducibility for external labs. The paper is a preprint, so code release status is unknown, but the methodological detail is high.
The primary limitation is the scale of the dataset (1,170 environments) compared to massive web-scale datasets, though the quality is higher. The approach is currently focused on specific domains (software, data, math, etc.) and may not generalize to all types of long-horizon tasks (e.g., creative writing, open-ended research). The computational cost of generating and verifying these latent worlds is likely high. Additionally, the performance gains are relative to open-source baselines; while matching closed-source models is impressive, the absolute performance on the hardest tasks may still be limited by the base model's capabilities.
This work has high potential impact on the field of agentic AI. By providing a recipe for generating high-quality, verifiable long-horizon training data, it lowers the barrier for other labs to train competitive open-source agents. The PPRS technique is a generalizable insight for RL in sparse-reward settings. If the dataset and code are released, it could become a standard benchmark and training resource for the community, similar to how SWE-bench or Terminal-Bench have been used. It pushes the frontier of what open-source models can achieve in autonomous, long-duration tasks. The paper introduces a latent-world-first synthesis pipeline and Progress-Potential Reward Shaping (PPRS) to enable effective RL training for long-horizon agents, resulting in open-source models that match frontier closed-source systems on complex agentic benchmarks. This is a significant contribution that addresses the critical data and reward sparsity bottlenecks in agentic AI, offering a reproducible and scalable recipe for generating high-quality training environments and training signals.
Every robot already carries high-resolution contact sensors: its own joints. Legged robots and humanoids exploit them to walk, run, and track whole-body motions without vision, but there the contacts are large and forceful. Is joint sensing also enough for precise, contact-rich assembly, where the robot must find a millimeter-scale opening, mesh gear teeth, or catch a thread through light contact? We study this question by training blind assembly policies that observe only a short history of joint positions, velocities, and their own actions, without visual, force, torque, or tactile sensing. The poses of both the fixture and the part are unknown to the policy, and the grasp is randomized substantially. The policy must therefore assemble the parts through contact, sensed exclusively through its own proprioception. Trained with reinforcement learning entirely in simulation over randomized parts, grasps, and contacts, such policies learn peg insertion, gear meshing, and nut threading. The peg insertion policy transfers zero-shot to the real world and inserts pegs of different shapes and sizes in 73% of trials. Controlled comparisons with matched visual policies in simulation show that blind policies generalize far better to unseen part shapes and sizes, though they can be slower. Training such policies needs thousands of different assemblies, each simulated with its clearance intact, so we also release an open-source generator that builds them procedurally from exact convex pieces for MuJoCo; it is more accurate and faster than convex decomposition or MuJoCo's signed-distance collision. Videos and interactive simulations are at https://younghyopark.me/blind-proprioceptive-assembly/.
Primary: Massachusetts Institute of Technology
All Institutions: Massachusetts Institute of Technology
The paper demonstrates that proprioceptive sensing alone is sufficient for precise, contact-rich robotic assembly tasks like peg insertion, gear meshing, and nut threading, achieving zero-shot sim-to-real transfer and superior generalization to unseen parts compared to visual policies. By introducing a rigorous simulation framework with exact convex collision geometry and a novel contact-supervised RL training method, the authors provide a robust, sensor-light alternative to vision-based manipulation, significantly advancing the state of the art in blind dexterity and sim-to-real transfer for precision tasks.
The paper proposes a novel approach to precise robotic assembly using only proprioceptive sensing (joint positions, velocities, and actions), explicitly excluding vision, force, torque, or tactile sensors. The core method involves training Reinforcement Learning (PPO) policies in simulation with extensive domain randomization of part geometry, grasp, and contact properties. A key technical contribution is the "contact supervision" auxiliary loss, which trains the policy's latent representation to predict contact states (free, touching, aligned) without providing privileged information to the actor, thereby accelerating learning from sparse rewards. Additionally, the authors introduce "AssemblyGen," an open-source procedural generator for MuJoCo that creates exact convex collision geometries for tight-clearance assemblies (pegs, gears, nuts), addressing a significant simulation bottleneck where standard convex decomposition or SDF methods fail to preserve sub-millimeter clearances accurately.
The experimental evaluation is rigorous and comprehensive. The authors demonstrate that blind policies achieve near-perfect success rates on in-distribution tasks and, crucially, generalize significantly better to unseen part shapes and sizes compared to matched visual policies. The zero-shot sim-to-real transfer is a strong result, with the peg insertion policy achieving 73% success on real hardware without fine-tuning, including on out-of-distribution pegs. The ablation studies effectively isolate the contribution of the contact supervision loss and the collision geometry generator. The comparison against scripted spiral search baselines highlights the efficiency of the learned reactive search strategy.
The paper provides high reproducibility standards. It releases an open-source generator for the simulation assets, which is a major barrier in this field. Detailed hyperparameters, architecture descriptions, and training ranges are provided in the appendix. The use of standard simulators (MuJoCo) and common algorithms (PPO) further aids reproducibility. The specific hardware setup (Flexiv Rizon 4S, Robotiq 2F-85) is clearly defined.
The primary limitation is the assumption that the robot starts in contact with the fixture; the policy does not handle the initial approach phase from a distance. The behavior is local and cannot recover from large deviations (e.g., if the part slips out of the grasp). The real-world evaluation is limited to peg insertion; gear meshing and nut threading were only validated in simulation. The reliance on precise simulation of friction and contact dynamics, despite domain randomization, remains a potential source of sim-to-real gap for other tasks.
This work challenges the prevailing assumption that precise manipulation requires vision or force sensing. By demonstrating that proprioception alone is sufficient for millimeter-scale assembly, it opens new avenues for robust, low-cost robotic systems that do not require expensive sensors or complex calibration. The released simulation tools will likely benefit the broader robotics community working on contact-rich tasks. The finding that blind policies generalize better to unseen geometries than visual policies is a significant insight for the design of robust manipulation policies. The paper demonstrates that proprioceptive sensing alone is sufficient for precise, contact-rich robotic assembly tasks like peg insertion, gear meshing, and nut threading, achieving zero-shot sim-to-real transfer and superior generalization to unseen parts compared to visual policies. By introducing a rigorous simulation framework with exact convex collision geometry and a novel contact-supervised RL training method, the authors provide a robust, sensor-light alternative to vision-based manipulation, significantly advancing the state of the art in blind dexterity and sim-to-real transfer for precision tasks.
We introduce REKI, a 2.4B vision-language-action model (VLA) pretrained exclusively on public data that sets a new Pareto frontier between performance and model size: on LIBERO-Plus, RoboTwin Clean2Rand and RoboCasa365, it is outperformed only by models about twice its size. REKI consists of a vision-language model (VLM) followed by a flow-matching action expert, and is pretrained on a curated mixture of public robot, egocentric human and vision-language data. Additionally, we endow it with a memory that gives the policy access to its recent past at negligible cost: a few register tokens in the action expert summarise the output of the VLM at each frame and persist in the expert's cache. This memory improves success in simulation and on real robots, and makes oscillations between poses about four times rarer. REKI uses a simple model design and training paradigm, and is optimised to be efficient at training and inference. We release the full code across data curation, training, inference and evaluation, deployment, and all pretrained and finetuned model weights: https://github.com/kyutai-labs/reki.
Primary: Kyutai
All Institutions: Kyutai
REKI introduces an efficient 2.4B VLA model with a novel register-token memory mechanism that achieves state-of-the-art performance on key robotics benchmarks while maintaining a small model size. The combination of flow-matching action experts, public-data pretraining, and a lightweight temporal memory strategy provides a robust and reproducible baseline that advances the field of vision-language-action modeling by balancing performance with computational efficiency.
The paper proposes REKI, a 2.4B parameter Vision-Language-Action (VLA) model. The architecture consists of a standard Vision-Language Model (VLM) backbone followed by a flow-matching action expert. The primary methodological contribution is the introduction of a lightweight memory mechanism using "register tokens" within the action expert. These tokens summarize the VLM's output at each frame and persist in the expert's cache, allowing the policy to access recent history with negligible computational overhead. The model is pretrained on a curated mixture of public robot, egocentric human, and vision-language data. The approach is notable for its efficiency and the specific design choice of using register tokens for temporal context rather than heavier recurrent or transformer-based memory blocks.
The evaluation is conducted on standard robotics benchmarks: LIBERO-Plus, RoboTwin Clean2Rand, and RoboCasa365. The paper claims REKI sets a new Pareto frontier, outperforming models of similar size and being outperformed only by models approximately twice its size. Specific empirical findings include a four-fold reduction in pose oscillations due to the memory mechanism. The inclusion of real-robot evaluation strengthens the claims, though the full text provided is largely a structural outline, so specific numerical comparisons must be inferred from the abstract's strong claims.
The paper explicitly states that full code for data curation, training, inference, evaluation, deployment, and all pretrained/finetuned weights are released. This high level of open-sourcing significantly enhances reproducibility and community utility.
The provided text is a preprint skeleton with section headers but lacks the detailed body text for rigorous scrutiny of specific ablation studies or failure cases. The reliance on "public data" may limit performance compared to models trained on proprietary, high-quality datasets, though the paper argues for efficiency. The "register token" memory is described as having "negligible cost," but the exact latency trade-offs in high-frequency control loops need verification in the full text.
By providing a strong, efficient baseline that is fully open-sourced, REKI lowers the barrier to entry for VLA research. The memory mechanism could be adopted by other architectures to improve temporal consistency in robotic control. The focus on public data pretraining is significant for democratizing access to capable robot policies. REKI introduces an efficient 2.4B VLA model with a novel register-token memory mechanism that achieves state-of-the-art performance on key robotics benchmarks while maintaining a small model size. The combination of flow-matching action experts, public-data pretraining, and a lightweight temporal memory strategy provides a robust and reproducible baseline that advances the field of vision-language-action modeling by balancing performance with computational efficiency.