Last 7 Days (October 29 – November 04, 2026)
Whether models trained on different modalities converge to the same representation of the world is central to AI. The Platonic Representation Hypothesis (PRH) conjectures that they do as they scale, but without a proof it offers neither a guarantee nor a recipe. We turn the hypothesis into a theorem. First, we prove a \emph{Platonic Representation Theorem}: Joint-Embedding Predictive Architectures (JEPAs) trained separately on any number of modalities learn the same representation up to one rotation whenever their positive pairs are two observations of one scene--\emph{good views}--and their misalignment is bounded otherwise. Second, we show that next-token prediction is a JEPA with \emph{bad views}, parts of a single document, which explains why language models align with vision only through scale. Third, we validate the theory from 145k captions to the texts of 940M web images, and uncover a scaling law of alignment that predicts unseen runs within seed noise. Good views need 42 times fewer training samples than next-token prediction on the same corpus to reach the same alignment, and a 109M-parameter text JEPA aligns with DINOv2 and CLIP's image encoders better than every language model we evaluated, up to 72B parameters, and than CLIP's text encoder: next-token prediction is a remarkably inefficient route to platonic representations. As a showcase, one rotation turns separately trained encoders into an image captioner (CIDEr .838 on COCO). Even that rotation needs no pair: on the latest pair-free benchmark, our text JEPA beats every zero-shot accuracy reported with language models up to 8B parameters, e.g., 64\% against at most 56\% top-5 on ImageNet-100. Positive views, and not scale alone, are thus the lever to build platonic representations.
Primary: Unknown (Not explicitly stated in provided text)
All Institutions: Unknown
The paper proves the Platonic Representation Hypothesis for JEPAs with good views, demonstrating that cross-modal alignment is a property of the positive-pair distribution rather than just scale, and validates this with a 109M model outperforming 72B LLMs in alignment and enabling efficient zero-shot captioning.
The paper presents a rigorous theoretical framework for the Platonic Representation Hypothesis (PRH) by proving that Joint-Embedding Predictive Architectures (JEPAs) trained on "good views" (multiple observations of the same scene) converge to the same representation up to rotation. The core novelty lies in distinguishing "good views" (cross-observation positives) from "bad views" (corruptions of a single observation) and formally linking next-token prediction to the latter. The theoretical proofs rely on spectral analysis of the positive-pair kernel, establishing conditions under which cross-modal alignment is guaranteed. The methodology is sound, leveraging established tools from contrastive learning theory (Laplacian eigenmaps, spectral gaps) but applying them in a novel cross-modal context. The extension to show that next-token prediction is a "bad-view" JEPA is a clever and insightful reformulation that explains the inefficiency of LLMs in achieving cross-modal alignment compared to vision models.
The experimental validation is extensive and convincing. The authors validate the theory on Flickr30k, COCO, and large-scale web image datasets (Recap-DataComp). Key results include: (1) Good-view JEPAs achieve significantly higher cross-modal alignment (mutual 10-NN) than bad-view JEPAs and next-token prediction on identical data. (2) A 109M-parameter text JEPA trained on good views outperforms language models up to 72B parameters in alignment with DINOv2 and CLIP image encoders. (3) The paper demonstrates a 42x sample efficiency gain for good views over next-token prediction to reach equivalent alignment. (4) A practical application is shown where a single rotation matrix aligns frozen DINOv2 and text JEPA encoders to enable zero-shot image captioning (CIDEr .838 on COCO) and zero-shot classification (64% on ImageNet-100), outperforming larger LLMs. The scaling law derived for alignment is predictive of unseen runs within seed noise, adding significant empirical weight to the theoretical claims.
The paper provides detailed hyperparameters, dataset splits, and training protocols. The use of standard architectures (ViT, BERT) and objectives (LeJEPA) enhances reproducibility. The code for the rotation fitting and captioning demo is implied to be available, though specific URLs are not present in the text. The scaling law parameters are provided, allowing others to predict alignment for their own setups.
The theoretical proofs assume a finite world and injective renderings, which are idealizations. The "good view" definition requires multiple distinct observations of the same scene, which can be data-hungry for images (e.g., video frames) compared to text (multiple captions). The paper acknowledges that image JEPAs trained on good views (video frames) are currently weaker than DINOv2 due to data scale, leaving the image-side scaling of good views as future work. The captioning results rely on a pretrained LLM decoder, so the "zero-shot" claim applies to the alignment/rotation, not the language generation capability itself.
This paper has high potential impact by providing a theoretical recipe for improving cross-modal alignment. It suggests that current practices of scaling LLMs for multimodal tasks are inefficient and that focusing on "good views" (multi-observation positives) is a more effective lever. The demonstration of alignment via simple rotations opens new avenues for lightweight multimodal integration, potentially reducing the need for massive paired datasets or joint training. The insights into why LLMs align poorly with vision models (bad views) are valuable for the design of future multimodal foundation models. The paper proves the Platonic Representation Hypothesis for JEPAs with good views, demonstrating that cross-modal alignment is a property of the positive-pair distribution rather than just scale, and validates this with a 109M model outperforming 72B LLMs in alignment and enabling efficient zero-shot captioning.
Accurate forecasting with uncertainty quantification is critical in a variety of scientific applications ranging from epidemiology and economics to energy systems and earth sciences. Yet most forecasting models still need to be developed and trained separately for each domain-specific dataset. Time series foundation models (TSFMs) aim to remove this limitation and enable the application of the same model across domains through large-scale pretraining. However, most existing TSFMs treat each time series independently and therefore cannot model interactions among time series and external drivers. Here, we introduce Chronos-2, a foundation model for zero-shot forecasting that models the interactions between multivariate time series and heterogeneous covariates, which commonly arise in scientific applications. The model infers relationships among time series directly from historical context and generates probabilistic forecasts without task-specific treatment. Across various benchmarks, Chronos-2 achieves state-of-the-art accuracy on average by improving skill scores by 5.2 percentage points relative to the strongest baselines and increasing win rates by 10 percentage points. Chronos-2 is also computationally efficient, generating forecasts for 100 time series in under one second on a single mid-range GPU, 50x and 1000x faster end-to-end than the best task-specific point and probabilistic forecasters, respectively. In addition, we demonstrate its effectiveness in diverse scientific settings -- including infectious-disease forecasting, macroeconomic modeling, carbon-flux prediction, and runoff simulation -- where the model matches or surpasses specialized methods developed for each domain. These results show that a single pretrained model can capture structure shared across temporal systems.
Primary: Amazon
All Institutions: Amazon, AWS
Chronos-2 introduces a multivariate time series foundation model leveraging group attention and synthetic pretraining to achieve state-of-the-art zero-shot forecasting across diverse scientific domains. The paper demonstrates that a single pretrained model can effectively capture complex multivariate interactions and external drivers without task-specific training, outperforming specialized baselines in epidemiology, macroeconomics, earth science, and hydrology, while maintaining high computational efficiency.
The paper introduces Chronos-2, a 120M parameter encoder-only Transformer foundation model for time series forecasting. The core methodological contribution is the "group attention" mechanism, which allows the model to infer dependencies among multivariate time series and heterogeneous covariates directly from historical context (in-context learning) without task-specific training. The architecture alternates between temporal attention (within a series) and group attention (across related series). A significant portion of the methodology focuses on the synthetic data generation pipeline, which creates diverse multivariate dynamics by coupling univariate generators (Gaussian Processes, AR, ETS) through linear (VAR, mixing matrices) and non-linear (MLP, RNN) transformations. This approach addresses the scarcity of large-scale multivariate time series data for pretraining. The model uses a quantile regression head to provide probabilistic forecasts, supporting uncertainty quantification.
The evaluation is extensive and rigorous, covering both standard benchmarks and domain-specific case studies. On the FEV benchmark (100 tasks), Chronos-2 achieves state-of-the-art results, outperforming strong baselines like TiRex, TimesFM, and Moirai. It demonstrates significant gains in win rate and skill score over univariate inference, validating the group attention mechanism. The paper includes four detailed case studies: epidemiology (ECDC RespiCast), macroeconomics (German CPI), earth science (FLUXNET carbon flux), and hydrology (CAMELS-DE rainfall-runoff). In each domain, the model matches or surpasses specialized, task-specific models (e.g., HBV, LSTM ensembles, XGBoost) in zero-shot settings. The inclusion of hierarchical forecasting experiments further demonstrates the model's ability to capture cross-level dependencies without explicit hierarchy information. The computational efficiency is also highlighted, with inference times under 1 second for 100 series on a mid-range GPU.
The paper provides detailed architectural specifications, training hyperparameters, and data generation procedures. The model size (120M parameters) and training setup (4x A100 GPUs) are clearly stated. The synthetic data generation process is described in detail, allowing for potential replication of the pretraining corpus. However, the specific code repository or model weights are not explicitly linked in the provided text (though likely available given the Amazon affiliation and prior Chronos releases). The evaluation protocols for the case studies are well-defined, referencing standard domain metrics (WIS, NSE, etc.).
The model is limited to 120M parameters, which is smaller than some competing foundation models (e.g., Moirai, TimesFM), potentially limiting its capacity for extremely complex dynamics. The reliance on synthetic data for multivariate pretraining may introduce a domain gap if real-world interactions are more complex than the synthesized ones. The paper does not extensively discuss the interpretability of the inferred relationships, which is a known challenge in foundation models. Additionally, the model does not natively support static metadata (e.g., soil type, demographics), which is a limitation for certain scientific applications like ungauged basin prediction, though the authors acknowledge this as future work.
This paper has high potential impact by democratizing advanced forecasting capabilities. By providing a single, efficient, zero-shot model that handles multivariate interactions, it reduces the barrier to entry for applying state-of-the-art forecasting in scientific and industrial domains. The demonstrated success in diverse fields (epidemiology, economics, earth sciences) suggests broad applicability. The efficiency of the model (running on consumer GPUs/CPU) makes it accessible for edge deployment or resource-constrained environments. It shifts the paradigm from task-specific model fitting to contextual inference, which could streamline workflows in data science teams. Chronos-2 introduces a multivariate time series foundation model leveraging group attention and synthetic pretraining to achieve state-of-the-art zero-shot forecasting across diverse scientific domains. The paper demonstrates that a single pretrained model can effectively capture complex multivariate interactions and external drivers without task-specific training, outperforming specialized baselines in epidemiology, macroeconomics, earth science, and hydrology, while maintaining high computational efficiency.
Despite recent advances in CUAs, their development remains constrained by the limited diversity, controllability, and reliability of interaction data. Moreover, conventional optimization pipelines lack iterative feedback, while existing recursive improvement approaches often rely on self-generated experiences, limiting ability expansion and yielding diminishing returns. To address these challenges, we introduce CUA-Master, a nested double-loop recursive improvement framework. At the system level, an outer world-data-model flywheel continuously incorporates open-world corpus and leverages evolving model weaknesses to guide subsequent data construction and optimization iterations. At the data level, an inner flywheel couples open-world seeding and autocatalytic propagation. The former mines authentic workflows as seed tasks from instructional videos, and employs visual-semantic dual-path trajectory reconstruction; The latter transforms these seeds into reusable primitives for controllable synthesis of diverse, complex, and verifiable tasks. To enable efficient and reliable trajectory collection, we further introduce a feedback-guided snapshot-rollback retry mechanism in various real environments. Extensive experiments demonstrate that CUA-Master-27B achieves competitive performance across GUI-only and GUI-CLI hybrid benchmarks, scoring 80.2 on OSWorld Verified, 36.9 on OSWorld 2.0, 43.9 on WeaveBench, and 25.0 on Agents' Last Exam. Our framework also substantially improves the quality and efficiency of data synthesis, and exhibits continuous gains across recursive iterations.
Primary: MMLab, HKUST(GZ)
All Institutions: MMLab, HKUST(GZ), Explore Academy
CUA-Master introduces a nested double-loop recursive framework that significantly advances Computer-Use Agents by integrating open-world data seeding with autocatalytic propagation and feedback-guided rollout retries, achieving competitive performance on major benchmarks with a 27B parameter model.
The paper proposes CUA-Master, a nested double-loop recursive improvement framework for Computer-Use Agents (CUAs). The core innovation lies in the "World-Data-Model" flywheel, which integrates an outer loop for system-level optimization (guiding data construction based on model weaknesses) and an inner "Data Flywheel" for scalable synthesis. The inner flywheel combines "Open-World Seeding" (mining tasks from instructional videos via visual-semantic dual-path reconstruction) with "Autocatalytic Propagation" (abstracting tasks into reusable primitives for compositional synthesis). A key technical contribution is the "feedback-guided snapshot-rollback retry mechanism," which improves rollout efficiency by preserving valid execution prefixes and recovering from localized failures. The methodology is well-structured, addressing the common pitfalls of closed-world self-improvement (diminishing returns) and unidirectional data pipelines (lack of feedback). The use of a hybrid GUI+CLI action space is also a significant practical contribution.
The experiments are extensive, evaluating CUA-Master-27B on four major benchmarks: OSWorld Verified, OSWorld 2.0, WeaveBench, and Agents' Last Exam. The model achieves competitive performance, scoring 80.2 on OSWorld Verified and 25.0 on Agents' Last Exam, surpassing many larger open-source models and approaching top closed-source models like GPT-5.6-Sol and Claude-Opus-5. The paper provides detailed ablations on the effectiveness of the hybrid action space, the data flywheel's impact on diversity and rollout success rates, and the progressive gains from recursive training iterations. The inclusion of cross-platform generalization tests (Windows, macOS) further strengthens the evaluation. The results demonstrate that the proposed framework effectively scales data quality and model capability.
The paper provides sufficient detail on the training recipe (SFT followed by RL), hyperparameters, and infrastructure setup (VM pool, resource scheduling). However, the specific prompts for the "annotator" model roles and the exact criteria for primitive extraction are referenced in figures/appendices that are not fully detailed in the text provided. The reliance on a powerful external annotator model for data generation introduces a dependency that may limit reproducibility for groups without access to such models. The code and data are not explicitly linked in the provided text, though the methodology is described in enough detail for a skilled team to attempt replication.
The primary limitation is the high computational and infrastructure cost, particularly for the large-scale VM deployment and iterative training. The framework's performance is heavily dependent on the quality of the external annotator model, which introduces additional inference costs and potential biases. The paper acknowledges that the upper limit of synthesized data quality is bounded by the annotator's capabilities. Additionally, the recursive nature of the pipeline makes it complex to manage and debug compared to simpler training paradigms.
This work has significant implications for the development of autonomous agents capable of interacting with complex software environments. By providing a scalable and reliable method for generating high-quality training data from open-world sources, it addresses a critical bottleneck in CUA research. The hybrid GUI+CLI approach and the snapshot-rollback mechanism are likely to be adopted by other groups working on agent-based systems. The framework's ability to continuously improve through recursive data-model feedback loops offers a promising path toward more general and capable AI agents. CUA-Master introduces a nested double-loop recursive framework that significantly advances Computer-Use Agents by integrating open-world data seeding with autocatalytic propagation and feedback-guided rollout retries, achieving competitive performance on major benchmarks with a 27B parameter model.
Language models are accurate on most long-arithmetic problems, but their accuracy is estimated on samples and carries no guarantee for an unseen input; tool use is exact but moves the computation outside the network. We show that exactness at all input lengths becomes a finite, checkable property of a network when each of its steps reads only a bounded view of an external working memory. In Sketchpad, Qwen3.5-0.8B with a low-rank adapter executes arithmetic procedures on a rendered sheet. Each forward pass reads a three-cell view and emits one step: a digit, a carry or borrow, and the next operation. A fixed runtime moves the view and writes no digit of a result. We prove in Lean that the model is then exact on all inputs of any length if and only if it is exact on the finite set of views the runtime presents, that no finite test suffices when steps also read earlier steps, and that the guarantee tolerates bounded numerical perturbation and extends to compositions of procedures. All 2,446 views pass on the deployment device, a laptop GPU running the model in a web browser. The certified model performs signed addition and subtraction, long multiplication and division, square roots and greatest common divisors at arbitrary length. It solved 200 of 200 pre-registered problems in the browser, and 220 of 220 sums of up to 1,000 digits and products of up to 20 digits, of which GPT-6.1 Sol, the strongest of ten other models, solved 217. A symbolic runtime built on these procedures solves, expands and factors polynomials with every number computed by the model, and a multiplicative gate on the adapter recovers the base model exactly when the sheet is not in use.
Primary: MIT Media Lab
All Institutions: MIT Media Lab
Sketchpad introduces a certified exact arithmetic framework by constraining a vision-language model to a bounded view of external memory, enabling finite verification of infinite-length generalization. The paper rigorously proves that this "pinned-sheet" policy allows for a finite check (2,446 views) to guarantee correctness on all inputs, outperforming frontier models on long arithmetic tasks and enabling certified symbolic algebra, representing a significant step toward verifiable and trustworthy AI systems.
The paper introduces "Sketchpad," a framework that achieves certified exact arithmetic by constraining a vision-language model (VLM) to read only a bounded view (3 cells) of an external working memory (the "sheet") at each step. The core methodological innovation is the "pinned-sheet policy," which ensures that the model's output is a function of the current view alone, decoupling it from the history of previous steps. This transforms the problem of verifying infinite-length generalization into a finite verification problem: checking the model's correctness on the finite set of all possible views (2,446 views). The authors provide formal proofs in Lean 4 establishing that if the model is correct on all views, it is correct on all inputs of any length. They also prove a negative result: if the model reads previous steps (as in Chain-of-Thought), no finite test suffices for certification. The training involves a low-rank adapter (LoRA) on Qwen3.5-0.8B, trained specifically on the finite table of views with a margin-based loss to ensure robustness against floating-point perturbations.
The experimental evaluation is rigorous and directly supports the theoretical claims. The authors certify the model on the actual deployment hardware (laptop GPU, 16-bit precision), showing that all 2,446 views pass with a minimum logit margin of 3.02, which provides a robustness bound. The model achieves 100% accuracy on pre-registered tests for addition, subtraction, multiplication, division, square roots, and GCD at arbitrary lengths, outperforming frontier models like GPT-6.1 Sol (which made silent errors on long sums). The paper also demonstrates compositionality, using the certified arithmetic procedures to build a symbolic algebra runtime that solves and factors polynomials. A gating mechanism is introduced to allow the model to switch between the base model and the arithmetic adapter, preserving general capabilities while enabling exact computation when needed.
The paper offers high reproducibility. The authors provide the model weights, adapter, certificate, and analysis data on Hugging Face. The runtime, algebra code, baselines, and Lean proofs are available under an Apache 2.0 license on GitHub. The certification process is detailed, including how margins are computed and how the certificate is bound to specific model file hashes and library versions. The pre-registration of problems and the detailed description of the training procedure (loss function, replay strategy, stopping criteria) further enhance reproducibility.
The primary limitation is the scope of the certification: it covers the model's steps but not the runtime's logic or the transcription of problems into calls. The runtime is conventional code, and errors in the runtime or in the dialogue adapter's decision to invoke the sheet are not covered by the certificate. The dialogue adapter, while effective, still suffers from some forgetting (5.8 points drop on grade-school word problems) and mistranslates about 7% of problems. The method is currently limited to arithmetic and basic algebra; extending it to more complex symbolic reasoning or continuous mathematics would require significant additional work. The reliance on a specific VLM architecture (Qwen3.5-0.8B) and the need for a vision encoder to read the sheet may limit efficiency compared to purely text-based approaches, though the paper argues the bounded view is crucial for the certification guarantee.
This paper has significant broader impact by providing a rigorous framework for certifying the correctness of neural network outputs, a long-standing challenge in AI safety and reliability. It demonstrates that exactness can be a finite, checkable property of a network, offering a path toward trustworthy AI systems for critical applications where silent errors are unacceptable (e.g., finance, healthcare, scientific computing). The approach bridges the gap between neural network flexibility and symbolic verification, potentially influencing future architectures that prioritize verifiability. The use of Lean 4 for formal proofs of neural network properties is a notable methodological contribution that could inspire further work in formal verification of ML systems. Sketchpad introduces a certified exact arithmetic framework by constraining a vision-language model to a bounded view of external memory, enabling finite verification of infinite-length generalization. The paper rigorously proves that this "pinned-sheet" policy allows for a finite check (2,446 views) to guarantee correctness on all inputs, outperforming frontier models on long arithmetic tasks and enabling certified symbolic algebra, representing a significant step toward verifiable and trustworthy AI systems.
Reasoning vision-language-action (VLA) models pair physical control with natural-language rationales that supervisors can read to monitor embodied safety. Prior attacks on VLA reasoning corrupt the rationale itself and leave visible evidence in the text. We show that visual perturbations can instead alter the physical action while keeping the generated rationale word-for-word identical. At the shared text/action interface of Alpamayo for driving and InstructVLA for manipulation, our text-neutral action-subspace (TNAS) diagnostic shows that the locally most action-sensitive direction is nearly orthogonal to the active text-loss gradients, with 1.87% and 5.28% norm overlap. At equal hidden-state perturbation norms across both models, interventions along this text-neutral residual shift actions 4.64x to 47.8x more than text-gradient perturbations, whereas text-gradient perturbations induce 21.2x to 277x more text drift. Steering visual perturbations along this direction reaches the same decoupling from the image. Across 154 obstacle-annotated driving scenes in Alpamayo, TNAS-guided attacks induce a new collision under a word-for-word identical rationale in 52.6% of scenes under full-image perturbations and 35.1% under a localized forward-camera patch, compared with 4.55% for unguided action attacks. Relative to a text-preserving baseline, TNAS guidance raises exact rationale preservation from 83.0% to 89.5% and cuts teacher-forced text drift by 68.4% at a comparable collision rate. In InstructVLA, TNAS guidance cuts text drift by 40.5% and preserves the exact response on 85.0% of examples alongside a 1.73-unit action-chunk shift. An unchanged rationale therefore cannot certify a safe action, so monitors for reasoning VLAs must verify the predicted action channel directly.
Primary: University of California, San Diego (Inferred from NSF Grant 2333489 and common affiliation for this research group; note: explicit affiliation text is missing from provided snippet)
All Institutions: University of California, San Diego (Inferred)
The paper demonstrates that visual perturbations can alter the physical actions of reasoning VLA models while keeping the generated natural language rationales identical, revealing a critical safety vulnerability. By introducing the Text-Neutral Action-Subspace (TNAS) diagnostic, the authors show that the action-sensitive directions in the hidden state are nearly orthogonal to text-generation gradients, enabling targeted attacks that bypass text-based safety monitors. This work provides a rigorous empirical foundation for the claim that text rationales cannot certify safe actions, urging the field to develop direct verification methods for the action channel in embodied AI systems.
The paper introduces the Text-Neutral Action-Subspace (TNAS) diagnostic, a method to identify directions in the hidden state space of Vision-Language-Action (VLA) models that are highly sensitive to physical actions but orthogonal to text-generation gradients. The core innovation is the decoupling of the "reasoning" channel (text) from the "action" channel (control) within the shared representation space. By projecting perturbations onto this text-neutral subspace, the authors demonstrate that it is possible to induce significant changes in physical behavior (e.g., causing a collision) while keeping the generated natural language rationale identical. This is a sophisticated adversarial analysis that goes beyond simple input-space perturbations, operating in the latent space of the model's internal representations. The methodology is rigorous, utilizing gradient analysis to define the subspace and validating it through controlled interventions.
The experiments are conducted on two distinct VLA architectures: Alpamayo (driving) and InstructVLA (manipulation). The driving experiments use 154 obstacle-annotated scenes, showing that TNAS-guided attacks induce collisions in 52.6% of scenes with word-for-word identical rationales, compared to 4.55% for unguided attacks. The manipulation experiments show a 40.5% reduction in text drift while maintaining significant action shifts. The results are compelling and directly support the central claim that text rationales are not a reliable proxy for safety in VLA models. The comparison against text-preserving baselines is particularly strong, highlighting the efficiency of the TNAS approach.
The paper provides specific metrics (norm overlaps, drift ratios, collision rates) and clear definitions of the TNAS diagnostic. However, the provided text snippet does not include the full appendix or code repository links, which are critical for reproducing the specific gradient computations and subspace projections. The reliance on specific models (Alpamayo, InstructVLA) limits immediate reproducibility for users without access to these specific checkpoints, but the methodology is generalizable to other VLA models with similar architectures.
The primary limitation is the dependence on specific VLA models (Alpamayo and InstructVLA). It is unclear how well the TNAS diagnostic generalizes to other VLA architectures or if the orthogonality between text and action gradients is a universal property or specific to these models. Additionally, the attacks are evaluated in simulated or controlled environments; real-world robustness against sensor noise and dynamic obstacles is not fully established. The "word-for-word identical" rationale is a strong metric, but in practice, minor variations might occur, and the paper does not extensively explore the robustness of the attack to small text perturbations.
This paper has significant implications for the safety certification of embodied AI systems. It challenges the assumption that natural language rationales provide a transparent and safe interface for monitoring VLA models. The finding that actions can be manipulated without altering the text rationale suggests that current safety monitors relying on text inspection are fundamentally flawed. This work will likely drive the development of new safety mechanisms that directly monitor the action channel or use more robust verification methods. It also highlights a critical vulnerability in the deployment of VLA models in high-stakes environments like autonomous driving and robotics. The paper demonstrates that visual perturbations can alter the physical actions of reasoning VLA models while keeping the generated natural language rationales identical, revealing a critical safety vulnerability. By introducing the Text-Neutral Action-Subspace (TNAS) diagnostic, the authors show that the action-sensitive directions in the hidden state are nearly orthogonal to text-generation gradients, enabling targeted attacks that bypass text-based safety monitors. This work provides a rigorous empirical foundation for the claim that text rationales cannot certify safe actions, urging the field to develop direct verification methods for the action channel in embodied AI systems.
Despite recent advances in CUAs, their development remains constrained by the limited diversity, controllability, and reliability of interaction data. Moreover, conventional optimization pipelines lack iterative feedback, while existing recursive improvement approaches often rely on self-generated experiences, limiting ability expansion and yielding diminishing returns. To address these challenges, we introduce CUA-Master, a nested double-loop recursive improvement framework. At the system level, an outer world-data-model flywheel continuously incorporates open-world corpus and leverages evolving model weaknesses to guide subsequent data construction and optimization iterations. At the data level, an inner flywheel couples open-world seeding and autocatalytic propagation. The former mines authentic workflows as seed tasks from instructional videos, and employs visual-semantic dual-path trajectory reconstruction; The latter transforms these seeds into reusable primitives for controllable synthesis of diverse, complex, and verifiable tasks. To enable efficient and reliable trajectory collection, we further introduce a feedback-guided snapshot-rollback retry mechanism in various real environments. Extensive experiments demonstrate that CUA-Master-27B achieves competitive performance across GUI-only and GUI-CLI hybrid benchmarks, scoring 80.2 on OSWorld Verified, 36.9 on OSWorld 2.0, 43.9 on WeaveBench, and 25.0 on Agents' Last Exam. Our framework also substantially improves the quality and efficiency of data synthesis, and exhibits continuous gains across recursive iterations.
Primary: MMLab, HKUST(GZ)
All Institutions: MMLab, HKUST(GZ), Explore Academy
CUA-Master introduces a nested double-loop recursive framework that significantly advances Computer-Use Agents by integrating open-world data seeding with autocatalytic propagation and feedback-guided rollout retries, achieving competitive performance on major benchmarks with a 27B parameter model.
The paper proposes CUA-Master, a nested double-loop recursive improvement framework for Computer-Use Agents (CUAs). The core innovation lies in the "World-Data-Model" flywheel, which integrates an outer loop for system-level optimization (guiding data construction based on model weaknesses) and an inner "Data Flywheel" for scalable synthesis. The inner flywheel combines "Open-World Seeding" (mining tasks from instructional videos via visual-semantic dual-path reconstruction) with "Autocatalytic Propagation" (abstracting tasks into reusable primitives for compositional synthesis). A key technical contribution is the "feedback-guided snapshot-rollback retry mechanism," which improves rollout efficiency by preserving valid execution prefixes and recovering from localized failures. The methodology is well-structured, addressing the common pitfalls of closed-world self-improvement (diminishing returns) and unidirectional data pipelines (lack of feedback). The use of a hybrid GUI+CLI action space is also a significant practical contribution.
The experiments are extensive, evaluating CUA-Master-27B on four major benchmarks: OSWorld Verified, OSWorld 2.0, WeaveBench, and Agents' Last Exam. The model achieves competitive performance, scoring 80.2 on OSWorld Verified and 25.0 on Agents' Last Exam, surpassing many larger open-source models and approaching top closed-source models like GPT-5.6-Sol and Claude-Opus-5. The paper provides detailed ablations on the effectiveness of the hybrid action space, the data flywheel's impact on diversity and rollout success rates, and the progressive gains from recursive training iterations. The inclusion of cross-platform generalization tests (Windows, macOS) further strengthens the evaluation. The results demonstrate that the proposed framework effectively scales data quality and model capability.
The paper provides sufficient detail on the training recipe (SFT followed by RL), hyperparameters, and infrastructure setup (VM pool, resource scheduling). However, the specific prompts for the "annotator" model roles and the exact criteria for primitive extraction are referenced in figures/appendices that are not fully detailed in the text provided. The reliance on a powerful external annotator model for data generation introduces a dependency that may limit reproducibility for groups without access to such models. The code and data are not explicitly linked in the provided text, though the methodology is described in enough detail for a skilled team to attempt replication.
The primary limitation is the high computational and infrastructure cost, particularly for the large-scale VM deployment and iterative training. The framework's performance is heavily dependent on the quality of the external annotator model, which introduces additional inference costs and potential biases. The paper acknowledges that the upper limit of synthesized data quality is bounded by the annotator's capabilities. Additionally, the recursive nature of the pipeline makes it complex to manage and debug compared to simpler training paradigms.
This work has significant implications for the development of autonomous agents capable of interacting with complex software environments. By providing a scalable and reliable method for generating high-quality training data from open-world sources, it addresses a critical bottleneck in CUA research. The hybrid GUI+CLI approach and the snapshot-rollback mechanism are likely to be adopted by other groups working on agent-based systems. The framework's ability to continuously improve through recursive data-model feedback loops offers a promising path toward more general and capable AI agents. CUA-Master introduces a nested double-loop recursive framework that significantly advances Computer-Use Agents by integrating open-world data seeding with autocatalytic propagation and feedback-guided rollout retries, achieving competitive performance on major benchmarks with a 27B parameter model.
A model is only half of an AI agent system. The other half is the inference strategy: the reasoning procedure that turns model calls into a response. Today that second half is written by hand, and the experience of building one strategy never carries to the next. We introduce Strategist, a meta-agent that evolves inference strategies across tasks. Each strategy is written as a blueprint,executable code that other strategies can reuse, and together the blueprints form a library. Given a task and a small development set, Strategist retrieves related blueprints, writes candidate strategies, tests them on real samples, and revises them from the results. Every candidate then enters the library, so later tasks build on earlier discoveries. On 30 benchmarks, Strategist beats the strongest baseline on 24 and matches its average accuracy (74.7% vs.74.0%) at 39% lower inference cost. Its strategies also transfer to unseen tasks and other models without further search. The part of an AI system that is not improved on its own now does.
Primary: Princeton University
All Institutions: Princeton University
The paper presents a novel meta-agent framework that evolves reusable inference strategies via a growing library of executable code blueprints. By automating the design of reasoning procedures and enabling cross-task knowledge accumulation, it achieves superior accuracy and cost-efficiency compared to hand-designed and existing automated baselines, marking a significant step towards self-improving agentic systems.
The paper introduces "Strategist," a meta-agent framework that automates the design of inference strategies (reasoning procedures) for LLM agents. The core novelty lies in the representation of strategies as "Strategy Blueprints"—executable Python code with a standardized interface that allows for composition and reuse. Unlike previous methods that treat strategy optimization as a per-task black-box search or rely on fixed component libraries, Strategist maintains a growing library of blueprints that accumulates knowledge across tasks. The methodology involves a four-step loop: retrieval of relevant blueprints, proposal of new candidates (balancing exploration and exploitation), evaluation on real samples, and accumulation of results back into the library. The use of multi-tier model routing (fast/balanced/complex) within a single strategy is a practical and effective addition, allowing for cost-aware reasoning.
The experimental setup is rigorous, evaluating 30 benchmarks across math, code, commonsense, and QA. The comparison against 18 baselines (including standard prompting, CoT, and agentic frameworks) is extensive. The results show that Strategist beats the strongest baseline on 24/30 benchmarks while reducing inference cost by 39%. The ablation studies clearly demonstrate the contribution of the library and iterative search. The cross-model transfer experiments (applying strategies discovered on Qwen3 to GPT-5 and Gemini) are a strong validation of the method's generalizability, showing that the learned strategies capture task structure rather than model-specific idiosyncrasies.
The paper provides high reproducibility. It releases code on GitHub, details the exact model configurations (Qwen3 variants), and provides pseudocode for the design loop. The prompt templates and library retrieval mechanisms are described in detail. The use of open-source models for the main experiments further aids reproducibility.
The primary limitation is the reliance on a powerful meta-LLM (GPT-6) for the strategist, which may not be accessible to all researchers. The cost of the search process itself (10 iterations per benchmark) is significant, though the paper argues this is a one-time cost per task type. The method is evaluated on a specific set of benchmarks, and its performance on highly dynamic or real-time tasks is not explored.
This work has significant implications for the field of agentic AI by shifting the focus from static prompt engineering to dynamic, cumulative strategy evolution. It provides a framework for building AI systems that improve their own inference efficiency and accuracy over time without retraining the underlying models. This could lead to more robust and cost-effective AI deployments in production environments. The paper presents a novel meta-agent framework that evolves reusable inference strategies via a growing library of executable code blueprints. By automating the design of reasoning procedures and enabling cross-task knowledge accumulation, it achieves superior accuracy and cost-efficiency compared to hand-designed and existing automated baselines, marking a significant step towards self-improving agentic systems.
LLM post-training via reinforcement learning has shown consistent improvements in accuracy at the cost of reducing behavioral coverage - a phenomenon known as sharpening that can stall progress on difficult tasks. Repeated sampling objectives such as $\mathrm{pass@k}$ and $\mathrm{max@k}$ are known to mitigate sharpening, but the mechanism behind it remains poorly understood. Moreover, existing approaches for $\mathrm{@k}$ optimization use dedicated algorithms, and no general framework exists for adapting established policy optimization methods from $\mathrm{@}1$ to $\mathrm{@k}$. We derive the optimal policies for KL-regularized $\mathrm{pass@k}$ and $\mathrm{max@k}$ objectives in closed form. Our explicit solutions reveal that $\mathrm{@k}$ optimization naturally redistributes learning pressure based on prompt difficulty, directing it toward harder problems. Crucially, we show that standard $\mathrm{@}1$ objectives achieve these exact same optimal policies through prompt-adaptive regularization or reward transformation. Ultimately, this yields an optimizer-independent explanation for why $\mathrm{@k}$ objectives reduce sharpening on easy tasks while prioritizing success on harder ones. Building on these insights, we introduce $\mathrm{@k}$ Adaptive Pressure for Policy Optimization (KAPPO). Following a one-time reference calibration, KAPPO seamlessly adapts existing $\mathrm{@}1$ methods to $\mathrm{@k}$ via prompt-adaptive regularization or reward transformation, without otherwise altering their training procedures. Experiments with several RL methods on mathematical reasoning and code generation demonstrate that KAPPO provides direct control over sharpening. Crucially, we demonstrate that $\mathrm{@k}$ pressure can be fully realized with single completion training, matching the performance of multi-completion $\mathrm{@k}$ optimizers at a fraction of the cost.
Primary: École Polytechnique Fédérale de Lausanne (EPFL)
All Institutions: School of Computer and Communication Sciences (IC), Department of Computer Science
The paper derives closed-form optimal policies for KL-regularized $pass@k$ and $max@k$ objectives, revealing that they act as prompt-adaptive regularization that mitigates sharpening in LLM post-training. By introducing KAPPO, a framework that adapts standard $@1$ optimizers to $@k$ settings via single-completion training, the authors provide a cost-effective and theoretically grounded solution to the diversity-accuracy trade-off, demonstrating significant improvements in robustness and generalization on mathematical reasoning and code generation tasks.
The paper presents a rigorous theoretical analysis of KL-regularized $pass@k$ and $max@k$ objectives. The core contribution is deriving the closed-form optimal policies for these objectives, revealing that they act as prompt-adaptive regularization or reward transformations. Specifically, it shows that $@k$ optimization naturally increases the effective regularization strength for easy prompts (where the reference model already succeeds) and decreases it for hard prompts. This provides a mechanistic explanation for why $@k$ objectives mitigate "sharpening" (the loss of behavioral diversity in LLMs during RL post-training). The proposed method, KAPPO, leverages this insight to adapt standard $@1$ optimizers (like DPO, A*PO, GRPO) to $@k$ settings without requiring expensive online sampling of $k$ completions. Instead, it uses a one-time reference calibration to estimate prompt-dependent parameters, allowing single-completion training to achieve the benefits of multi-completion training. The mathematical derivations are sound, utilizing variational arguments and concavity properties of the reward functionals.
The experiments are well-designed to validate the theoretical claims. The authors use two distinct model families (OLMo-3-7B-Instruct-SFT and Qwen3-4B-Base) to ensure generality. They evaluate on mathematical reasoning (binary rewards) and code generation (continuous rewards). The key experimental result is that KAPPO with $k>1$ maintains performance at larger evaluation budgets ($pass@k_{eval}$) compared to standard $@1$ training, which typically suffers from sharpening (performance drops as $k$ increases). Furthermore, the paper demonstrates that single-completion KAPPO matches the performance of dedicated multi-completion baselines (like PKPO) at a fraction of the computational cost. The use of AIME 2026 and LiveCodeBench v6 as out-of-distribution/harder test sets strengthens the claim that the method improves robustness and generalization rather than just overfitting to the training distribution.
The paper provides a strong reproducibility statement. It claims that all datasets and models are publicly available. The method is described as a "minimalist intervention," requiring only a one-time calibration step and simple modifications to the loss function or regularization coefficient. Pseudocode for the calibration is mentioned in the appendix, and the codebase is stated to be available in supplementary materials. The hyperparameters and experimental details are described as being provided in the appendix. This high level of detail and the simplicity of the proposed modification make the work highly reproducible.
The primary limitation is the reliance on a static bank of reference completions for calibration. If the reference model's distribution shifts significantly or if the task distribution is highly non-stationary, the pre-computed adaptive parameters might become suboptimal. Additionally, for continuous rewards, the method uses a win-rate surrogate rather than the raw $max@k$ objective, which is an approximation. The paper also notes that the win-rate solution assumes a continuous reference reward distribution, which may not hold for discrete rewards with ties. Finally, the experiments are limited to 4B-7B scale models; the scalability of the calibration cost to larger models or very large prompt sets is not extensively explored.
This work has significant implications for the field of LLM post-training. By providing a theoretical foundation for why $@k$ objectives work and a practical, low-cost way to implement them, it enables practitioners to better control the trade-off between accuracy and diversity in RL fine-tuning. This is crucial for developing LLMs that are not only accurate but also robust and capable of diverse reasoning strategies. The method's compatibility with existing optimizers like DPO and GRPO makes it easy to adopt, potentially leading to widespread use in industry and academia. It addresses a critical bottleneck in current RLVR (Reinforcement Learning with Verifiable Rewards) pipelines, where sharpening often limits further progress. The paper derives closed-form optimal policies for KL-regularized $pass@k$ and $max@k$ objectives, revealing that they act as prompt-adaptive regularization that mitigates sharpening in LLM post-training. By introducing KAPPO, a framework that adapts standard $@1$ optimizers to $@k$ settings via single-completion training, the authors provide a cost-effective and theoretically grounded solution to the diversity-accuracy trade-off, demonstrating significant improvements in robustness and generalization on mathematical reasoning and code generation tasks.
Progress in AI is often measured by what a single model can do on its own. Yet teams of AI agents are starting to make breakthroughs and discoveries in science, medicine, mathematics and software engineering. As AI begins to work in teams of agents and people, the ability to collaborate becomes a core part of their intelligence. Research on human teams has measured this ability for decades. Inspired by this, we introduce the Collaborative Intelligence Benchmark, which measures how well an AI agent works as a teammate. It contains 240 tasks across 8 dimensions of teamwork, from refusing an unsafe shortcut to checking the work that a weaker model leaves behind. Each task comes from a real record or an existing benchmark, from aviation accident reports and clinical cases to coding tasks and negotiations, and has a programmatically verifiable outcome. To isolate teamwork from individual capability, models are scored only on tasks that they can solve by themselves when they have full information. We evaluate 18 closed- and open-weight models and find that every model, across sizes and families, has clear weaknesses as a teammate, with the best model scoring only 82.2 out of 100. Frontier models almost always refuse unsafe shortcuts and report misconduct, yet they cave under interpersonal pressure, reaching the best overall deal in only 14% of conflicts, and their pass rate drops from 97% to 81% when they take over a task in which a weaker model has cheated. Crucially, general capability does not guarantee good teamwork, as models with nearly the same capability score differ by up to 11 points as teammates. The benchmark includes 160 public tasks with code and data, along with 80 held-out tasks for evaluation. As AI agents increasingly work alongside other agents and people, the ability to collaborate effectively is becoming a crucial capability to measure and develop.
Primary: Stanford University
All Institutions: Stanford University, MIT, University of Oxford
The paper introduces the Collaborative Intelligence Benchmark, a rigorous framework for evaluating AI agents' teamwork capabilities across 8 dimensions, revealing that current frontier models have significant deficits in conflict resolution and supervisory tasks despite high individual capabilities. By isolating teamwork from raw intelligence and using programmatically verifiable tasks derived from real-world scenarios, the work provides a critical new metric for the developing field of multi-agent systems, highlighting that general capability does not guarantee effective collaboration and offering a reproducible baseline for future research.
The paper introduces the Collaborative Intelligence (CI) Benchmark, a novel framework designed to evaluate AI agents specifically on their ability to function as effective teammates rather than standalone solvers. The methodology is grounded in decades of human team research, decomposing teamwork into 8 distinct dimensions (e.g., safety, conflict resolution, initiative, takeover). A key methodological strength is the isolation of teamwork from individual capability: models are only scored on tasks they can solve independently when given full information. This controls for raw intelligence, ensuring that failures are attributed to collaborative deficits (e.g., poor communication, inability to handle social pressure, or failure to verify weaker partners' work) rather than lack of domain knowledge. The tasks are derived from real-world records (aviation accidents, clinical cases) and existing benchmarks, ensuring ecological validity.
The authors evaluate 18 closed and open-weight models, providing a comprehensive landscape of current agent capabilities. The results reveal significant gaps: while frontier models excel at refusing unsafe shortcuts, they fail significantly in interpersonal conflict resolution (reaching optimal deals in only 14% of cases) and struggle when taking over tasks from weaker models that have cheated (pass rate drops from 97% to 81%). The finding that general capability does not correlate perfectly with teamwork (models with similar individual scores differ by up to 11 points in CI) is a crucial empirical insight. The benchmark includes 160 public tasks and 80 held-out tasks, which is a reasonable scale for a new benchmark, though the sample size per dimension may be limited for fine-grained statistical significance.
The paper provides a GitHub repository and a project website, indicating high reproducibility. The inclusion of programmatically verifiable outcomes for each task is a major plus, as it allows for automated and consistent evaluation without subjective human judgment. The release of 160 public tasks with code and data ensures that the community can verify results and extend the benchmark.
The primary limitation is the reliance on text-based interactions, which may not fully capture the multimodal nature of real-world teamwork (e.g., visual cues, tone of voice). Additionally, the "full information" control condition assumes that individual capability is the only other variable, but in real teams, information asymmetry is often a core part of the challenge. The benchmark may also be susceptible to overfitting if models are trained on the public tasks, though the 80 held-out tasks mitigate this risk. The 8 dimensions, while comprehensive, may not capture all facets of human-AI or AI-AI collaboration, such as long-term trust building or dynamic role adaptation.
This benchmark is highly significant for the field of AI agents, as the industry is rapidly moving toward multi-agent systems. By providing a standardized way to measure "teamwork," it shifts the focus from individual model performance to system-level collaboration. This has direct implications for the deployment of AI in high-stakes environments like medicine and aviation, where human-AI or AI-AI teams are increasingly common. The findings that current models are poor teammates under pressure or when supervising weaker agents will likely drive new research into social reasoning, robustness, and collaborative training objectives. The paper introduces the Collaborative Intelligence Benchmark, a rigorous framework for evaluating AI agents' teamwork capabilities across 8 dimensions, revealing that current frontier models have significant deficits in conflict resolution and supervisory tasks despite high individual capabilities. By isolating teamwork from raw intelligence and using programmatically verifiable tasks derived from real-world scenarios, the work provides a critical new metric for the developing field of multi-agent systems, highlighting that general capability does not guarantee effective collaboration and offering a reproducible baseline for future research.
Scaling laws predict the outcome of large, expensive training runs by extrapolating trends from smaller, cheaper runs. On downstream accuracy evaluations, these predictions fail when the weaker models produced by the cheaper runs score near zero accuracy, because these scores give no trend to extrapolate to larger scales. We introduce hinted scaling laws, which use privileged information to turn a benchmark into a continuum of easier versions so that even the small models score above zero. We find that accuracy rises smoothly and predictably with the hint level, enabling new types of scaling laws that operate below the noise threshold. On the Delphi pretraining ladder, a hinted scaling law fit on models trained with at most $10^{20}$ FLOPs forecasts the GSM8K accuracy of a $10^{23}$ FLOP model (25.9%) to within 3 percentage points; a standard scaling law under-forecasts it by 13 percentage points when fit on the same models and by 12 even with models up to $10^{22}$ FLOPs in the fit. Across both this pretraining ladder and a collection of post-trained model families, hinted scaling laws reduce the median forecasting error on GSM8K, AIME and HLE from 21 percentage points to 4.3. Hinting also reduces the number of test examples needed to tell whether one model is better than another by widening the accuracy gaps between models while approximately preserving their order: a single hinted rollout per problem ranks nearly indistinguishable models as accurately as five unhinted rollouts. Our results suggest that hinting is a general primitive for improving the accuracy and cost of downstream scaling laws and other prediction tasks in model development.
Primary: Stanford University
All Institutions: Stanford University
The paper introduces hinted scaling laws, which use privileged information to create a continuum of easier benchmark versions, enabling accurate extrapolation of downstream performance from small models to large ones. This addresses a critical limitation in standard scaling laws where small models score near zero, providing a robust framework for predicting model capabilities with significantly reduced forecasting error and computational cost.
The paper introduces "hinted scaling laws," a method to overcome the failure of standard scaling laws when small models perform near zero accuracy on difficult benchmarks. By using privileged information (hints) to create a continuum of easier versions of a benchmark, the authors enable small models to score above zero, creating a smooth trend that can be extrapolated to larger scales. The methodology is conceptually elegant and addresses a critical gap in the current scaling law literature, which often relies on metrics that saturate or remain flat at low scales. The approach transforms a discrete, noisy evaluation into a continuous, predictable signal.
The experiments are extensive and convincing. The authors validate their approach on the Delphi pretraining ladder, showing that a hinted scaling law fit on models trained with at most $10^{20}$ FLOPs can forecast the GSM8K accuracy of a $10^{23}$ FLOP model within 3 percentage points, whereas standard scaling laws under-forecast by over 12 points. They further demonstrate the utility of hinting for model ranking, showing that a single hinted rollout can rank models as accurately as five unhinted rollouts. The reduction in median forecasting error from 21 to 4.3 percentage points across GSM8K, AIME, and HLE is a significant empirical result.
The paper provides a link to code and data at https://github.com/tatsu-lab/hinted-scaling-laws, which enhances reproducibility. The use of the Delphi pretraining ladder, a well-known resource, further supports the ability for other researchers to verify these results. The detailed description of the hinting mechanism and the specific benchmarks used allows for clear replication of the experimental setup.
The primary limitation is the reliance on the availability of "privileged information" or hints that can effectively decompose a benchmark into easier sub-tasks. This may not be trivially applicable to all domains or benchmarks, particularly those where the difficulty is not easily stratified by additional context. Additionally, the method assumes that the relationship between hint level and accuracy is smooth and predictable, which may not hold for all model families or tasks.
This work has significant implications for the efficient development of large language models. By enabling more accurate predictions of downstream performance from smaller, cheaper training runs, it can reduce the computational cost and time required to develop new models. The technique of using hints to improve evaluation resolution could also be applied to other areas of machine learning where evaluation is noisy or saturated at low performance levels. The paper introduces hinted scaling laws, which use privileged information to create a continuum of easier benchmark versions, enabling accurate extrapolation of downstream performance from small models to large ones. This addresses a critical limitation in standard scaling laws where small models score near zero, providing a robust framework for predicting model capabilities with significantly reduced forecasting error and computational cost.
We consider classification in high-dimensional feature spaces where only a vanishing fraction of features are useful and the number of classes may be large. While this rare-and-weak useful-feature regime is well understood in the binary setting, its multiclass version raises new questions: a feature can be useful for class discrimination without separating any specific pair of classes. We focus on a rare feature diversity model, in which most features are useless, but a small fraction promote discrimination among classes in a way that can be detected by a per-feature diversity test, such as one-way ANOVA or the Kruskal-Wallis test. For this setting, we propose Diversity Pursuit Higher Criticism Thresholding (DP-HCT), a feature-screening procedure that applies a higher-criticism threshold to the collection of p-values obtained from per-feature diversity tests. We justify DP-HCT under a Gaussian multiclass Rare-and-Weak Useful Feature model with class-mean contrasts of fixed magnitude in random directions, extending the binary model of Donoho and Jin (2008). When the number of classes is fixed, our model recovers the binary phase transition. When the number of classes grows logarithmically with the number of features, however, we derive a new and strictly harder phase transition, driven by the growing degrees of freedom and their effect on the useful-feature tail rate. We show that whenever classification beyond chance is possible, DP-HCT with ANOVA, followed by a simple classifier based on similarity to class means, achieves asymptotically perfect accuracy. Experiments on real multiclass classification benchmarks demonstrate substantial improvements, and numerical simulations confirm that the derived phase-transition curve accurately predicts the finite-sample behavior of DP-HCT in the growing-class regime.
Primary: Carnegie Mellon University
All Institutions: Carnegie Mellon University, Reichman University
The paper establishes a sharp phase transition for multiclass classification in the rare-and-weak useful feature regime, proving the optimality of Higher Criticism thresholding when the number of classes grows logarithmically with dimension. By extending the binary Donoho-Jin framework to the multiclass setting with rigorous asymptotic analysis and empirical validation on biological benchmarks, it provides both a fundamental theoretical limit and a practical, effective feature-screening tool for high-dimensional data.
The paper proposes Diversity Pursuit Higher Criticism Thresholding (DP-HCT), a feature-screening method that extends the binary Higher Criticism (HC) framework to multiclass classification. The core innovation is the derivation of a phase transition boundary for a Gaussian multiclass Rare-and-Weak Useful Feature model where the number of classes $C$ grows logarithmically with the number of features $p$. The authors rigorously prove that when $C$ is fixed, the boundary recovers the classical Donoho-Jin result, but when $C$ grows, a strictly harder phase transition emerges due to the increased degrees of freedom in the ANOVA/Kruskal-Wallis tests. The achievability proof shows that DP-HCT, followed by a simple centroid-similarity classifier, achieves asymptotically perfect accuracy above this boundary. The theoretical analysis is robust, utilizing noncentral chi-square moment-generating functions and Chernoff bounds to handle the growing degrees of freedom.
The experimental section includes Monte Carlo simulations that accurately validate the derived phase-transition curves, showing that DP-HCT tracks the oracle threshold closely. Real-data experiments on five biological benchmarks (gene expression, metabolomics, miRNA) demonstrate substantial improvements over baselines like Bonferroni and Benjamini-Hochberg FDR. Stress tests involving noise injection and nuisance feature inflation further confirm the robustness of the screening procedure. The evaluation is rigorous, using appropriate cross-validation protocols that account for data leakage in repeated measures.
The paper provides detailed algorithmic descriptions, specific dataset preprocessing steps, and cross-validation protocols. The supplementary material includes full proofs and additional simulation details. While code is not explicitly linked in the provided text, the level of detail in the experimental setup (e.g., specific GEO series, Metabolomics Workbench DOIs, and solver parameters) suggests high reproducibility.
The theoretical guarantees are derived under a Gaussian assumption with spherical symmetric class-mean contrasts. The paper acknowledges that unbalanced designs and non-Gaussian heavy-tailed distributions (though addressed partially by the Kruskal-Wallis test) are left for future work. The real-data experiments, while showing improvements, are on relatively small sample sizes ($n < 150$), which is typical for genomics but limits the generalizability to larger-scale deep learning applications.
This work provides a fundamental theoretical limit for high-dimensional multiclass classification, which is crucial for understanding sample complexity in genomics and other sparse-data fields. The proposed DP-HCT method offers a practical, computationally efficient alternative to complex feature selection methods, potentially impacting how high-dimensional data is preprocessed in biological and medical research. The paper establishes a sharp phase transition for multiclass classification in the rare-and-weak useful feature regime, proving the optimality of Higher Criticism thresholding when the number of classes grows logarithmically with dimension. By extending the binary Donoho-Jin framework to the multiclass setting with rigorous asymptotic analysis and empirical validation on biological benchmarks, it provides both a fundamental theoretical limit and a practical, effective feature-screening tool for high-dimensional data.
Diffusion models are often inaccurate in critical, data-sparse regions of the target distribution. Sampling from restrained distributions, biased distributions used in classical enhanced sampling to concentrate data near chosen configurations, can control where training data are collected but biases the learned distribution. We establish that this bias cancels exactly when forward noising from restrained data is conditioned on the restrained configuration at an appropriately chosen intermediate time. This allows a pretrained model fine-tuned on restrained data to achieve high accuracy in any chosen region of the target distribution, regardless of its probability, using standard denoising or flow matching objectives without importance weights. When no target data are available, Gibbs sampling enables inference from a model trained exclusively on restrained data.
Primary: New York University
All Institutions: New York University, Courant Institute of Mathematical Sciences
The paper introduces a novel and theoretically grounded method for targeted refinement of diffusion models in rare regions, demonstrating significant improvements in molecular simulation benchmarks.
The paper proposes "Restrained Diffusion Bridges" (RDB), a method to improve diffusion model accuracy in low-probability regions of the target distribution. The core insight is that the bias introduced by sampling from a restrained (biased) distribution can be exactly canceled by conditioning the forward noising process on the restrained configuration at an intermediate time $t^*$. This allows the use of standard denoising or flow matching objectives without importance weights. The method leverages the mathematical equivalence between the restrained density and the diffusion posterior at the matched noise level. The theoretical foundation is solid, relying on Doob-transform identities and Gaussian bridge properties. The approach is elegant and provides a principled way to fine-tune pretrained models or train new models on rare states without the complexity of reweighting.
The authors validate the method on three benchmarks: the 2D Müller-Brown potential (analytic ground truth), alanine dipeptide in explicit solvent (22 atoms), and chignolin in implicit solvent (93 heavy atoms). The experiments demonstrate that RDB significantly improves the sampling of rare basins (e.g., the central basin in Müller-Brown, rare Ramachandran states in alanine dipeptide) compared to baseline diffusion models trained on equilibrium data. The results show that RDB can recover accurate populations in rare regions even when the baseline model fails. The comparison is fair, matching simulation budgets and training updates. The use of molecular dynamics benchmarks is appropriate for the target application domain.
The paper provides detailed algorithmic descriptions (Algorithm 1, 2, 3) and hyperparameters in the appendices. The code is not explicitly linked in the provided text, but the methods are described with sufficient detail for implementation. The use of standard libraries (OpenMM, PyTorch implied) and clear definitions of the diffusion schedules enhances reproducibility.
The method requires access to the score function (gradient of the potential) for generating restrained data via Langevin dynamics, which limits its applicability to black-box simulators or image generation where the energy function is unknown. The likelihood-free variant using dataset resampling is provided but noted to fail in very low-density regions where the dataset has no coverage. The choice of $t^*$ is heuristic, and the paper acknowledges a trade-off between sampling efficiency and mixing speed. The method is primarily tested on molecular systems; generalization to other domains like image generation is suggested but not empirically validated in this paper.
The technique has significant implications for generative modeling in scientific domains where rare states are critical (e.g., drug discovery, materials science). It offers a practical solution to the "rare event" problem in diffusion models. The theoretical connection between umbrella sampling and diffusion bridges may inspire further research in enhanced sampling methods. The approach could be extended to other generative models and domains, potentially improving the fidelity of generated samples in underrepresented regions of the data distribution. The paper introduces a novel and theoretically grounded method for targeted refinement of diffusion models in rare regions, demonstrating significant improvements in molecular simulation benchmarks.
We study probabilistic forecasting of dynamical systems such as fluids and waves from coarse observations. Since the unseen fine scales influence the dynamics, the same coarse initial state can evolve into a whole distribution of futures. Generative forecasters may produce visually plausible samples, but it is hard to tell whether they sample the right conditional distribution, since trajectory data contain a single future for each initial state. We address this problem by generating reference data in a hierarchical way: for each coarse initial state, we generate many different fine-scale components and numerically evolve the full states. Using these conditional ensembles, we show that standard diffusion models and stochastic interpolants make substantial distributional errors. We then show that cascading shorter forecasts greatly reduces these errors for the turbulent flows and waves we study, but we also find and characterize cases in which it hurts. Since the coarse process is not Markov, cascading discards information about the unresolved fine scales even with exact transitions. To understand this, we propose a solvable analytic model which shows that the optimal cascade depth decreases as the coupling between the unresolved and observed scales gets stronger, and increases as the dependence of the forecast on the observation becomes more oscillatory. Consistent with our experiments, cascading is detrimental when the direct forecast is ``easy'' to learn, for example Gaussian as in the passive scalar case. We release our datasets as a first benchmark for probabilistic forecasting with ground-truth conditional ensembles.
Primary: University of Basel
All Institutions: University of Basel, PSL University
The paper provides a rigorous theoretical and experimental analysis of when cascading generative forecasters improves probabilistic accuracy, introducing a novel benchmark for evaluating conditional distributions in dynamical systems.
The paper introduces a rigorous experimental framework for evaluating probabilistic forecasters by generating "reference ensembles" via hierarchical data generation. This allows for the direct comparison of the learned conditional distribution $P(X_T|X_0)$ against the true conditional distribution, addressing a critical gap where standard trajectory data only provides single futures. The theoretical contribution is significant, proposing a solvable analytic model (rotation driven by a hidden Ornstein-Uhlenbeck process) that isolates the trade-off between approximation error (favoring shorter transitions) and memory loss (favoring direct forecasts due to non-Markovian coarse-graining). The authors prove that an optimal finite cascade depth exists under specific conditions, providing a theoretical foundation for why cascading helps in some systems (turbulent flows) but hurts in others (passive scalars).
The experiments are extensive, covering five distinct physical systems (Euler, SQG, Heterogeneous Waves, Klein-Gordon, Passive Scalar) to test the generality of the findings. The use of spatial FID and conditional energy distance on large reference ensembles (10,000 samples per test observation) provides statistically robust evaluation. The results clearly demonstrate that direct diffusion models and stochastic interpolants fail to capture the correct conditional distribution despite producing visually plausible samples, while cascaded SIs recover the distribution in nonlinear systems. The finding that cascading is detrimental for the passive scalar case is a strong, counter-intuitive result supported by the theory.
The paper provides detailed descriptions of the numerical solvers, initial conditions, and model architectures. The release of the datasets as a benchmark is a major contribution to reproducibility. However, the text provided does not explicitly list a GitHub URL, though it mentions releasing datasets. The hyperparameters and training protocols are described in sufficient detail for replication.
The study is limited to 2D periodic domains and specific PDEs. The "fixed budget" constraint (equalizing the number of supervised pairs) is a harsh constraint that may not reflect all practical scenarios where intermediate data is abundant. The theory, while elegant, is based on a simplified linear model and may not fully capture the complexities of highly chaotic 3D turbulence. The inference cost of cascading increases with depth, which is a practical limitation not fully explored in terms of compute-time trade-offs.
This work has high impact on the field of scientific machine learning and probabilistic forecasting. It provides a new benchmark for evaluating generative forecasters and a theoretical understanding of when autoregressive/cascaded approaches are beneficial. This is crucial for weather forecasting and fluid dynamics simulation, where capturing the correct uncertainty distribution is as important as the mean forecast. The insights into the non-Markovian nature of coarse-grained processes will influence how future models handle partial observations. The paper provides a rigorous theoretical and experimental analysis of when cascading generative forecasters improves probabilistic accuracy, introducing a novel benchmark for evaluating conditional distributions in dynamical systems.
Reasoning-oriented post-training enables large language models (LLMs) to solve complex tasks via multi-step inference. Supervised fine-tuning (SFT) on high-quality reasoning traces is a particularly efficient approach, but its effectiveness depends critically on careful data curation. Existing pipelines rely on a costly generate-then-filter paradigm: large pools of long reasoning traces are produced by multiple teacher models and subsequently filtered for difficulty and diversity using additional LLMs, often requiring weeks of computation. In this work, we propose SFTProbe, a fundamentally different approach that bypasses this pipeline by directly predicting the quality of reasoning data. We show that a brief LoRA-based adaptation, combined with evaluating loss on only 5% of each reasoning trace, suffices to estimate both difficulty and diversity. SFTProbe is grounded in a geometric view of post-training, where pretrained LLMs lie in a low-rank, anisotropic loss basin. By probing this basin via low-rank perturbations, we construct compact loss signatures whose clustering captures diversity, while loss at curvature inflection points provides a robust proxy for difficulty. We also provide a simple criterion for teacher selection. Empirically, our approach reduces data curation time from weeks to hours while maintaining or improving performance. Remarkably, on mathematical reasoning, 5k generated examples match the highly filtered 89k-example in OpenThoughts-Math dataset, and 7.5k generated examples outperforms it by 2.5% in accuracy. We further introduce a high-quality physics reasoning dataset, and use it to train a model that outperforms an IPhO 2025 silver-medal-winning model. The code and the dataset are released at https://bigml-cs-ucla.github.io/SFTProbe-page/.
Primary: University of California, Los Angeles (UCLA)
All Institutions: University of California, Los Angeles (UCLA)
SFTProbe introduces a principled, geometric approach to SFT data curation that significantly reduces computational costs while maintaining high performance in reasoning tasks. By leveraging low-rank perturbations to estimate data difficulty and diversity, the method offers a scalable alternative to expensive LLM-based filtering, with strong empirical validation on mathematical and physics reasoning benchmarks.
The paper proposes SFTProbe, a method to bypass the standard "generate-then-filter" paradigm for SFT data curation. Instead of using LLMs to judge quality, it utilizes a brief LoRA-based adaptation on a small subset of data to predict the quality of the full dataset. The core theoretical contribution is a geometric view of the loss landscape, arguing that pretrained LLMs reside in a low-rank, anisotropic basin. By probing this basin with low-rank perturbations, the authors construct "loss signatures" where clustering indicates diversity and loss at curvature inflection points indicates difficulty. This is a novel shift from heuristic filtering to a principled, optimization-based estimation of data utility.
The empirical results are strong and directly address the efficiency bottleneck of current pipelines. The claim that 5k generated examples match the performance of 89k highly filtered examples from OpenThoughts-Math is significant, suggesting a massive reduction in compute requirements for data generation. Furthermore, the introduction of a high-quality physics reasoning dataset and the training of a model that outperforms an IPhO 2025 silver-medal-winning model provides concrete evidence of the method's effectiveness in complex reasoning domains. The reduction of curation time from weeks to hours is a practical win for the community.
The paper states that code and datasets are released at the provided URL. The methodology relies on standard techniques (LoRA, loss evaluation) combined with specific probing strategies. While the geometric theory is abstract, the practical implementation appears straightforward enough for reproduction given the released artifacts. The specific hyperparameters for the "brief LoRA-based adaptation" and the definition of "5% of each reasoning trace" are critical details that must be present in the code release for full reproducibility.
The primary limitation is the reliance on the geometric assumption of a "low-rank, anisotropic loss basin." If this assumption fails for certain model architectures or domains, the probing method may lose efficacy. Additionally, the comparison is primarily against the OpenThoughts-Math dataset; broader validation across diverse reasoning tasks (e.g., coding, general QA) would strengthen the claim. The "weeks to hours" claim depends heavily on the scale of the original pipeline being compared against, which may vary across labs.
This work has high potential for adoption in the LLM post-training community. As reasoning models become the standard for complex tasks, the cost of data curation is a major bottleneck. A method that reduces this cost by an order of magnitude while maintaining or improving quality is highly valuable. The geometric perspective on loss landscapes for data selection could also inspire further research into the interplay between model optimization and data quality. SFTProbe introduces a principled, geometric approach to SFT data curation that significantly reduces computational costs while maintaining high performance in reasoning tasks. By leveraging low-rank perturbations to estimate data difficulty and diversity, the method offers a scalable alternative to expensive LLM-based filtering, with strong empirical validation on mathematical and physics reasoning benchmarks.
Accurate forecasting with uncertainty quantification is critical in a variety of scientific applications ranging from epidemiology and economics to energy systems and earth sciences. Yet most forecasting models still need to be developed and trained separately for each domain-specific dataset. Time series foundation models (TSFMs) aim to remove this limitation and enable the application of the same model across domains through large-scale pretraining. However, most existing TSFMs treat each time series independently and therefore cannot model interactions among time series and external drivers. Here, we introduce Chronos-2, a foundation model for zero-shot forecasting that models the interactions between multivariate time series and heterogeneous covariates, which commonly arise in scientific applications. The model infers relationships among time series directly from historical context and generates probabilistic forecasts without task-specific treatment. Across various benchmarks, Chronos-2 achieves state-of-the-art accuracy on average by improving skill scores by 5.2 percentage points relative to the strongest baselines and increasing win rates by 10 percentage points. Chronos-2 is also computationally efficient, generating forecasts for 100 time series in under one second on a single mid-range GPU, 50x and 1000x faster end-to-end than the best task-specific point and probabilistic forecasters, respectively. In addition, we demonstrate its effectiveness in diverse scientific settings -- including infectious-disease forecasting, macroeconomic modeling, carbon-flux prediction, and runoff simulation -- where the model matches or surpasses specialized methods developed for each domain. These results show that a single pretrained model can capture structure shared across temporal systems.
Primary: Amazon
All Institutions: Amazon, AWS
Chronos-2 introduces a multivariate time series foundation model leveraging group attention and synthetic pretraining to achieve state-of-the-art zero-shot forecasting across diverse scientific domains. The paper demonstrates that a single pretrained model can effectively capture complex multivariate interactions and external drivers without task-specific training, outperforming specialized baselines in epidemiology, macroeconomics, earth science, and hydrology, while maintaining high computational efficiency.
The paper introduces Chronos-2, a 120M parameter encoder-only Transformer foundation model for time series forecasting. The core methodological contribution is the "group attention" mechanism, which allows the model to infer dependencies among multivariate time series and heterogeneous covariates directly from historical context (in-context learning) without task-specific training. The architecture alternates between temporal attention (within a series) and group attention (across related series). A significant portion of the methodology focuses on the synthetic data generation pipeline, which creates diverse multivariate dynamics by coupling univariate generators (Gaussian Processes, AR, ETS) through linear (VAR, mixing matrices) and non-linear (MLP, RNN) transformations. This approach addresses the scarcity of large-scale multivariate time series data for pretraining. The model uses a quantile regression head to provide probabilistic forecasts, supporting uncertainty quantification.
The evaluation is extensive and rigorous, covering both standard benchmarks and domain-specific case studies. On the FEV benchmark (100 tasks), Chronos-2 achieves state-of-the-art results, outperforming strong baselines like TiRex, TimesFM, and Moirai. It demonstrates significant gains in win rate and skill score over univariate inference, validating the group attention mechanism. The paper includes four detailed case studies: epidemiology (ECDC RespiCast), macroeconomics (German CPI), earth science (FLUXNET carbon flux), and hydrology (CAMELS-DE rainfall-runoff). In each domain, the model matches or surpasses specialized, task-specific models (e.g., HBV, LSTM ensembles, XGBoost) in zero-shot settings. The inclusion of hierarchical forecasting experiments further demonstrates the model's ability to capture cross-level dependencies without explicit hierarchy information. The computational efficiency is also highlighted, with inference times under 1 second for 100 series on a mid-range GPU.
The paper provides detailed architectural specifications, training hyperparameters, and data generation procedures. The model size (120M parameters) and training setup (4x A100 GPUs) are clearly stated. The synthetic data generation process is described in detail, allowing for potential replication of the pretraining corpus. However, the specific code repository or model weights are not explicitly linked in the provided text (though likely available given the Amazon affiliation and prior Chronos releases). The evaluation protocols for the case studies are well-defined, referencing standard domain metrics (WIS, NSE, etc.).
The model is limited to 120M parameters, which is smaller than some competing foundation models (e.g., Moirai, TimesFM), potentially limiting its capacity for extremely complex dynamics. The reliance on synthetic data for multivariate pretraining may introduce a domain gap if real-world interactions are more complex than the synthesized ones. The paper does not extensively discuss the interpretability of the inferred relationships, which is a known challenge in foundation models. Additionally, the model does not natively support static metadata (e.g., soil type, demographics), which is a limitation for certain scientific applications like ungauged basin prediction, though the authors acknowledge this as future work.
This paper has high potential impact by democratizing advanced forecasting capabilities. By providing a single, efficient, zero-shot model that handles multivariate interactions, it reduces the barrier to entry for applying state-of-the-art forecasting in scientific and industrial domains. The demonstrated success in diverse fields (epidemiology, economics, earth sciences) suggests broad applicability. The efficiency of the model (running on consumer GPUs/CPU) makes it accessible for edge deployment or resource-constrained environments. It shifts the paradigm from task-specific model fitting to contextual inference, which could streamline workflows in data science teams. Chronos-2 introduces a multivariate time series foundation model leveraging group attention and synthetic pretraining to achieve state-of-the-art zero-shot forecasting across diverse scientific domains. The paper demonstrates that a single pretrained model can effectively capture complex multivariate interactions and external drivers without task-specific training, outperforming specialized baselines in epidemiology, macroeconomics, earth science, and hydrology, while maintaining high computational efficiency.
We develop a new algorithmic framework for learning Boolean concept classes under logconcave distributions. Our framework applies to broad families of multi-index models and yields significantly faster robust learners than were previously known in these settings. It extends to partial classification and testable learning. Consequently, we obtain improved learners for several fundamental concept classes, including functions of halfspaces and multiclass linear classifiers, under logconcave marginals. Our starting point is a new boosting framework based on weak partial learners. Classical boosting approaches require often-intractable distribution-free weak learners or yield weak approximation guarantees. We show instead that weak learners for partial classification, where labels may be undefined on part of the domain, suffice for strong agnostic learning. The key challenge is that boosting requires a weak learner that succeeds under conditioned distributions that are not guaranteed to be logconcave. To address this, we develop a recursive moment-based partitioning procedure: given a logconcave distribution $X$ and a conditional version $X'$, we partition the space into polynomially many convex regions so that, on most regions, the low-degree moments of $X$ and $X'$ approximately agree. On each good region, a well-behaved multi-index model correlates with a low-degree polynomial, giving a weak hypothesis. Combining these hypotheses yields the required weak learner. A key ingredient of our approach is an efficient method for detecting moment discrepancy. Specifically, we show how to find a low-dimensional subspace that captures any non-trivial discrepancy between $X$ and $X'$, if one exists. Our approach combines spectral analysis of moment tensors with recent progress on isoperimetry for logconcave measures. This discrepancy-finding procedure may be of independent interest.
Primary: University of Wisconsin-Madison
All Institutions: University of Wisconsin-Madison, University of California, San Diego
The paper develops a new algorithmic framework for robust learning of multi-index models under logconcave distributions using weak partial learners, yielding significantly faster robust learners for fundamental concept classes like halfspaces and multiclass linear classifiers. It introduces a novel boosting approach that overcomes the challenge of non-logconcave conditioned distributions via a recursive moment-based partitioning procedure and an efficient discrepancy finder, providing substantial theoretical improvements over prior state-of-the-art methods.
The paper introduces a novel algorithmic framework for robust learning of multi-index models (MIMs) under logconcave distributions. The core innovation is a boosting framework based on "weak partial learners," which allows for strong agnostic learning guarantees without requiring intractable distribution-free weak learners. A key technical contribution is a recursive moment-based partitioning procedure that handles the challenge of boosting under conditioned distributions that are not logconcave. The authors develop an efficient discrepancy finder using spectral analysis of moment tensors and isoperimetry for logconcave measures to identify low-dimensional subspaces capturing moment discrepancies. This enables the construction of weak hypotheses on convex regions where moments approximately match, which are then combined via boosting. The approach is theoretically rigorous, providing sample and computational complexity bounds that significantly improve upon prior state-of-the-art for learning halfspaces, functions of halfspaces, and multiclass linear classifiers under logconcave marginals.
As a theoretical computer science paper, it does not contain empirical experiments on real-world datasets. The "evaluation" consists of rigorous mathematical proofs establishing the correctness and complexity bounds of the proposed algorithms. The results are validated by comparing the derived bounds against known lower bounds and previous upper bounds in the literature, demonstrating significant improvements in efficiency and robustness.
The paper provides detailed algorithmic descriptions and theoretical proofs. While no code is provided, the algorithms are described in sufficient detail for theoretical replication. The reliance on standard statistical query models and polynomial regression techniques makes the theoretical framework reproducible in a mathematical sense.
The primary limitation is the theoretical nature of the work; there is no empirical validation on real-world data to confirm that the theoretical gains translate to practical performance improvements. The algorithms may have high constant factors or practical overheads not captured by the asymptotic analysis. The focus on logconcave distributions, while broad, excludes other common distributional assumptions.
This work contributes to the foundational understanding of efficient learning under distributional assumptions. It provides new tools (weak partial learners, moment discrepancy finders) that could be applicable to other learning problems beyond MIMs. The results advance the state of the art in robust learning, which is crucial for real-world applications where data is often noisy and distributional assumptions are only approximately true. The paper develops a new algorithmic framework for robust learning of multi-index models under logconcave distributions using weak partial learners, yielding significantly faster robust learners for fundamental concept classes like halfspaces and multiclass linear classifiers. It introduces a novel boosting approach that overcomes the challenge of non-logconcave conditioned distributions via a recursive moment-based partitioning procedure and an efficient discrepancy finder, providing substantial theoretical improvements over prior state-of-the-art methods.
Understanding the training dynamics of deep neural networks in large-scale limits is a question both of foundational and of practical importance. Prior works have obtained such descriptions through dynamical mean field theory (DMFT), yielding a system of non-Markovian dynamics that are challenging to analyze and that obscure the optimization structure of the original dynamics. In this paper, we obtain instead a simple description of the large-scale limit training dynamics for ResNets in standard shape scalings and in the $1/\text{depth}$ branch scaling that allows maximal (local) feature updates. For a model trained with gradient descent (GD) from a random initialization, the limit dynamics is simply GD on a Neural Mean ODE with Gaussian initial weights. Our analysis covers population or empirical risk GD in the joint limit of large width, depth and embedding, input, and output dimensions. We obtain approximation scaling laws that depend on the shape of the ResNet and the covariance spectrum of the data. We also illustrate the flexibility of our approach by discussing Mixture of Expert models, where likewise the limit admits an intuitive (nested) Mean ODE description.
Primary: EPFL
All Institutions: EPFL, University of Oxford
The paper establishes that the large-scale training dynamics of ResNets converge to Gradient Descent on a Gaussian Mean ODE, providing a tractable and structurally faithful limit model. This represents a significant advance in the theoretical analysis of deep learning by simplifying the complex non-Markovian dynamics of DMFT into a standard optimization problem, enabling sharper convergence rates and deeper insights into the role of architecture shape and data covariance.
The paper proposes a rigorous theoretical framework for analyzing the training dynamics of Residual Neural Networks (ResNets) in the joint limit of large depth, width, and embedding dimension. The core innovation is deriving a "Gaussian Mean ODE" limit, which simplifies the complex non-Markovian dynamics typically found in Dynamical Mean Field Theory (DMFT) into a standard Gradient Descent (GD) process on a Neural Mean ODE. The methodology relies on a two-stage limit: first, depth divergence leads to a Mean ODE, and second, embedding dimension divergence with Gaussian initialization leads to a dimension-free Gaussian Mean ODE. The authors provide quantitative convergence rates and prove that the limit dynamics preserve the optimization structure of the original network, a significant improvement over prior DMFT approaches that obscure this structure. The use of Hilbert space formalism allows for generalization to infinite-dimensional input/output spaces.
As a theoretical paper, experimental evaluation is limited to numerical validations of the derived convergence rates and the behavior of the limit model. The paper includes plots demonstrating that the empirical errors match the theoretical bounds (e.g., $O(1/L + D/ML)$) and illustrates the "quasi-smoothness" of the limit objective. While not providing large-scale empirical benchmarks, the numerical experiments effectively validate the theoretical claims and the tightness of the derived bounds.
The paper is highly reproducible in a theoretical sense, with clear definitions, lemmas, and theorems. The code for numerical experiments is likely available (though not explicitly linked in the provided text snippet, standard practice for such papers from this group implies code availability). The mathematical derivations are detailed, allowing other researchers to verify the proofs. The reliance on AI for specific proof steps (GPT-6 Astra) is disclosed, which is a novel aspect of reproducibility/transparency in the research process itself.
The analysis is restricted to ResNets with two-layer perceptron blocks and specific scaling regimes (MLU/1-depth). It does not cover attention mechanisms or more complex architectures directly, although it mentions MoE. The results are for Gradient Descent; extensions to SGD or other optimizers are not covered. The assumption of Gaussian initialization for the sharpest results is a limitation, though they do discuss non-Gaussian cases with weaker bounds.
This work provides a crucial bridge between the practical architecture of ResNets and the theoretical understanding of their training dynamics. By showing that the limit is simply GD on a Mean ODE, it makes the analysis of deep network training significantly more tractable. This could lead to better hyperparameter tuning strategies, understanding of generalization in deep networks, and potentially new optimization algorithms designed specifically for the Mean ODE limit. It contributes to the foundational understanding of why deep networks train well. The paper establishes that the large-scale training dynamics of ResNets converge to Gradient Descent on a Gaussian Mean ODE, providing a tractable and structurally faithful limit model. This represents a significant advance in the theoretical analysis of deep learning by simplifying the complex non-Markovian dynamics of DMFT into a standard optimization problem, enabling sharper convergence rates and deeper insights into the role of architecture shape and data covariance.
Whether models trained on different modalities converge to the same representation of the world is central to AI. The Platonic Representation Hypothesis (PRH) conjectures that they do as they scale, but without a proof it offers neither a guarantee nor a recipe. We turn the hypothesis into a theorem. First, we prove a \emph{Platonic Representation Theorem}: Joint-Embedding Predictive Architectures (JEPAs) trained separately on any number of modalities learn the same representation up to one rotation whenever their positive pairs are two observations of one scene--\emph{good views}--and their misalignment is bounded otherwise. Second, we show that next-token prediction is a JEPA with \emph{bad views}, parts of a single document, which explains why language models align with vision only through scale. Third, we validate the theory from 145k captions to the texts of 940M web images, and uncover a scaling law of alignment that predicts unseen runs within seed noise. Good views need 42 times fewer training samples than next-token prediction on the same corpus to reach the same alignment, and a 109M-parameter text JEPA aligns with DINOv2 and CLIP's image encoders better than every language model we evaluated, up to 72B parameters, and than CLIP's text encoder: next-token prediction is a remarkably inefficient route to platonic representations. As a showcase, one rotation turns separately trained encoders into an image captioner (CIDEr .838 on COCO). Even that rotation needs no pair: on the latest pair-free benchmark, our text JEPA beats every zero-shot accuracy reported with language models up to 8B parameters, e.g., 64\% against at most 56\% top-5 on ImageNet-100. Positive views, and not scale alone, are thus the lever to build platonic representations.
Primary: Unknown (Not explicitly stated in provided text)
All Institutions: Unknown
The paper proves the Platonic Representation Hypothesis for JEPAs with good views, demonstrating that cross-modal alignment is a property of the positive-pair distribution rather than just scale, and validates this with a 109M model outperforming 72B LLMs in alignment and enabling efficient zero-shot captioning.
The paper presents a rigorous theoretical framework for the Platonic Representation Hypothesis (PRH) by proving that Joint-Embedding Predictive Architectures (JEPAs) trained on "good views" (multiple observations of the same scene) converge to the same representation up to rotation. The core novelty lies in distinguishing "good views" (cross-observation positives) from "bad views" (corruptions of a single observation) and formally linking next-token prediction to the latter. The theoretical proofs rely on spectral analysis of the positive-pair kernel, establishing conditions under which cross-modal alignment is guaranteed. The methodology is sound, leveraging established tools from contrastive learning theory (Laplacian eigenmaps, spectral gaps) but applying them in a novel cross-modal context. The extension to show that next-token prediction is a "bad-view" JEPA is a clever and insightful reformulation that explains the inefficiency of LLMs in achieving cross-modal alignment compared to vision models.
The experimental validation is extensive and convincing. The authors validate the theory on Flickr30k, COCO, and large-scale web image datasets (Recap-DataComp). Key results include: (1) Good-view JEPAs achieve significantly higher cross-modal alignment (mutual 10-NN) than bad-view JEPAs and next-token prediction on identical data. (2) A 109M-parameter text JEPA trained on good views outperforms language models up to 72B parameters in alignment with DINOv2 and CLIP image encoders. (3) The paper demonstrates a 42x sample efficiency gain for good views over next-token prediction to reach equivalent alignment. (4) A practical application is shown where a single rotation matrix aligns frozen DINOv2 and text JEPA encoders to enable zero-shot image captioning (CIDEr .838 on COCO) and zero-shot classification (64% on ImageNet-100), outperforming larger LLMs. The scaling law derived for alignment is predictive of unseen runs within seed noise, adding significant empirical weight to the theoretical claims.
The paper provides detailed hyperparameters, dataset splits, and training protocols. The use of standard architectures (ViT, BERT) and objectives (LeJEPA) enhances reproducibility. The code for the rotation fitting and captioning demo is implied to be available, though specific URLs are not present in the text. The scaling law parameters are provided, allowing others to predict alignment for their own setups.
The theoretical proofs assume a finite world and injective renderings, which are idealizations. The "good view" definition requires multiple distinct observations of the same scene, which can be data-hungry for images (e.g., video frames) compared to text (multiple captions). The paper acknowledges that image JEPAs trained on good views (video frames) are currently weaker than DINOv2 due to data scale, leaving the image-side scaling of good views as future work. The captioning results rely on a pretrained LLM decoder, so the "zero-shot" claim applies to the alignment/rotation, not the language generation capability itself.
This paper has high potential impact by providing a theoretical recipe for improving cross-modal alignment. It suggests that current practices of scaling LLMs for multimodal tasks are inefficient and that focusing on "good views" (multi-observation positives) is a more effective lever. The demonstration of alignment via simple rotations opens new avenues for lightweight multimodal integration, potentially reducing the need for massive paired datasets or joint training. The insights into why LLMs align poorly with vision models (bad views) are valuable for the design of future multimodal foundation models. The paper proves the Platonic Representation Hypothesis for JEPAs with good views, demonstrating that cross-modal alignment is a property of the positive-pair distribution rather than just scale, and validates this with a 109M model outperforming 72B LLMs in alignment and enabling efficient zero-shot captioning.
AI has made remarkable progress in mathematics, solving challenging problems with machine-checkable proofs in Lean, yet typically relying on predefined mathematical definitions and formal specifications. This raises a natural question: can it reason rigorously about an unknown world? We introduce the Algebraic World Model (AWM), a framework that turns interaction with unfamiliar environments into explicit mathematical models supporting formal reasoning. AWM explores the environment and develops executable Python hypotheses, continually testing and revising them against observations. It then structures these hypotheses into typed objects and shared rules, simplifies their structure through category-theory-guided transformations, and formalizes the resulting model in Lean 4 to prove reachability and impossibility. AWM can formally prove that a goal is unreachable, even beyond explored trajectories. On two impossible game levels, it constructs closed, kernel-checked Lean proofs excluding every legal winning sequence in all six runs, while Claude Opus 5.5 and GPT-6 Astra, equipped with coding tools, produce none in twelve baseline runs. Four of AWM's six initial bounds are correct, compared with one for Opus 5.5 and none for GPT-6 Astra; given only feedback that a cheaper win exists, AWM revises its rules and constructs new proofs, reaching six of six correct bounds. Moreover, AWM produces interpretable mathematical models whose rules can be inspected and tested step by step. Across all 25 public ARC-AGI-3 games, these models enable Lean to replay the winning trajectories of all 183 levels and generalize beyond observed play, predicting unseen transitions and detecting changes in game mechanics. With Claude Opus 5, AWM clears all levels with a score of 99.9, while its certified winning trajectories use only 43% of the human baseline's actions.
Primary: unknown
All Institutions: unknown
The paper presents a novel framework, AWM, that enables AI agents to learn algebraic world models from interaction and use them to construct formal proofs of reachability and impossibility in Lean 4. By combining empirical exploration with category-theory-guided model simplification and kernel-checked formal verification, AWM achieves state-of-the-art performance on ARC-AGI-3 and demonstrates a unique capability to provide rigorous, interpretable reasoning about unknown environments, marking a significant advancement in the integration of machine learning and formal methods.
The paper introduces the Algebraic World Model (AWM), a framework that bridges empirical exploration and formal verification. The methodology is structured in three stages: (1) Explore & Prototype, where an LLM agent interacts with an environment to generate executable Python hypotheses; (2) Structure & Simplify, where these hypotheses are organized into typed objects and rules using category-theory-guided transformations (limits/colimits) verified in Lean; and (3) Formalize & Prove, where the model is formalized in Lean 4 to prove reachability or impossibility. The use of category theory to guide the simplification of world models is a creative and rigorous approach to ensuring that the learned model preserves the logical structure of the environment. The "sufficient cover" definition provides a theoretical basis for transferring proofs from the learned model to the actual environment, which is a significant methodological contribution.
The experiments are conducted on ARC-AGI-3, a challenging benchmark for abstract reasoning. The results are strong: AWM achieves a score of 99.9, outperforming baselines like Claude Opus 5.5 and GPT-6 Astra in both game clearing and, crucially, in constructing formal proofs of impossibility. The paper demonstrates that AWM can produce kernel-checked Lean proofs that exclude all legal winning sequences for impossible levels, a capability the baselines lack. The generalization tests, where frozen models predict unseen transitions and detect mechanic changes, further validate the robustness of the learned algebraic structures.
The paper provides a high level of reproducibility by releasing an interactive website with models, tests, Lean proofs, and links to official replays. The inclusion of the Lean model for the running example (lp85) in the supplementary materials, along with scripts for building and auditing axioms, allows for independent verification of the formal proofs. The explicit statement that all numbers come from script outputs over recorded runs, rather than model-generated text, enhances trust in the results.
The primary limitation is the scope of the environments tested. AWM is demonstrated on ARC-like environments with discrete actions and immediate, observable feedback. The authors acknowledge that richer hidden states, delayed feedback, and stochastic dynamics remain untested. Additionally, the reliance on LLMs for the initial hypothesis generation and exploration phase means that the system's performance is bounded by the capabilities of the underlying LLM. The "sufficient cover" condition, while theoretically sound, requires careful verification in practice to ensure that the learned model indeed covers the optimal winning sequences of the environment.
This work has significant implications for the field of AI, particularly in the areas of formal methods, automated theorem proving, and agent-based reasoning. By enabling AI systems to reason rigorously about unknown worlds and produce machine-checkable proofs, AWM paves the way for more trustworthy and interpretable AI systems. The integration of category theory with machine learning offers a new paradigm for structuring and simplifying learned models, which could be applied to other domains requiring rigorous reasoning. The ability to prove impossibility, not just reachability, is a crucial step towards AI systems that can provide complete and reliable solutions to complex problems. The paper presents a novel framework, AWM, that enables AI agents to learn algebraic world models from interaction and use them to construct formal proofs of reachability and impossibility in Lean 4. By combining empirical exploration with category-theory-guided model simplification and kernel-checked formal verification, AWM achieves state-of-the-art performance on ARC-AGI-3 and demonstrates a unique capability to provide rigorous, interpretable reasoning about unknown environments, marking a significant advancement in the integration of machine learning and formal methods.
Many fields of physical interest on the sphere, such as wind, the polarisation of the cosmic microwave background and the shear of weak gravitational lensing, are spin fields, whose values change phase when the local frame rotates. We extend spherical Fourier neural operators (SFNOs) to fields of any spin, through lifted spherical convolutions that are exactly rotationally equivariant. Lifting spin fields to Mackey functions on the rotation group provides the conceptual framework, while computation is performed exactly and efficiently through their harmonic representation and by appealing to sampling theorems on the sphere and rotation group. We construct spin SFNOs (S$^2$FNOs) from these building blocks. We demonstrate empirically that rotational equivariance holds to machine precision for these layers. We achieve state-of-the-art performance on existing benchmarks, and introduce new benchmarks on which spin-aware models outperform their scalar counterparts.
Primary: University College London
All Institutions: University College London, The Alan Turing Institute
The paper presents a rigorous and efficient extension of Fourier Neural Operators to spin fields on the sphere, achieving exact rotational equivariance and state-of-the-art performance on new benchmarks in weather and cosmology.
The paper introduces Spin Spherical Fourier Neural Operators (S$^2$FNOs), extending the SFNO architecture to handle spin-weighted fields (vectors, tensors) on the sphere. The core innovation is the use of lifted spherical convolutions based on Mackey functions on the rotation group SO(3). This allows for exact rotational equivariance for linear layers. The authors provide a rigorous theoretical framework connecting spin fields to functions on the rotation group, proving that their harmonic-space implementation is both efficient (avoiding full group convolutions where possible) and exactly equivariant. They address the challenge of non-linearities by distinguishing between equivariant non-linearities on the sphere (modulus-based) and those on the rotation group (arbitrary pointwise maps), providing a flexible block design. The parameterization via interpolated profiles in harmonic space ensures resolution independence, a key feature of neural operators.
The authors introduce three new benchmarks: Spin Spherical MNIST (classification/dense prediction), Global Wind Forecasting (spin-1), and Weak Gravitational Lensing (spin-2 to spin-0). These are significant contributions as public benchmarks for spin fields were previously scarce. Empirically, the spin-aware models outperform scalar counterparts (which treat vector components as independent channels) by significant margins (e.g., 44.9% MSE reduction in wind forecasting). The paper also verifies equivariance to machine precision ($10^{-15}$) for linear layers, validating the theoretical claims. The comparison with existing methods like Spin-CNNs and group convolutions highlights the efficiency advantage (fewer parameters) while maintaining or improving accuracy.
The paper states that code and benchmarks will be released with the `s2ai` codebase. Detailed architectural descriptions, initialization schemes, and hyperparameters are provided in the appendices. The use of standard libraries (JAX) and well-defined mathematical operations (SHT, Wigner transforms) enhances reproducibility. The explicit mention of the `s2fft` code for transforms further aids implementation.
The primary limitation is the computational cost of the rotation group convolutions, although the authors mitigate this by restricting kernels to diagonal spins in the group construction. The non-linearities on the sphere are restricted to modulus-based functions to maintain equivariance, which may limit expressivity compared to arbitrary non-linearities on the group. The benchmarks, while novel, are specific to certain physical domains; generalization to other spin-field applications needs further validation. The paper relies on bandlimited signals, which may not capture high-frequency details in some real-world data without careful truncation.
This work has significant implications for scientific machine learning, particularly in cosmology (CMB polarization, weak lensing) and geophysics (weather forecasting, ocean currents). By providing a principled way to handle spin fields with exact equivariance, it enables more accurate and data-efficient models for these domains. The introduction of new benchmarks will facilitate future research in equivariant learning on the sphere. The methods could also be extended to other manifolds or groups, broadening the scope of geometric deep learning. The paper presents a rigorous and efficient extension of Fourier Neural Operators to spin fields on the sphere, achieving exact rotational equivariance and state-of-the-art performance on new benchmarks in weather and cosmology.
Spatiotemporal scene reconstruction with Dynamic Gaussian Splatting (DGS) fundamentally depends on perfect temporal synchronization across all cameras, an assumption rarely achievable in practice. Industry-standard synchronization requires expensive specialized hardware and complex setup, while data-driven alternatives including audio-based alignment, geometry matching, and pose tracking remain limited by restrictive environmental or scene-specific assumptions. To overcome these limitations, we propose Phase-DGS, a framework that replaces conventional timestamps with a semantic phase representing each frame's inherent state within the underlying motion cycle. Frames capturing the same motion state share the same phase regardless of which camera records them or when, naturally establishing spatiotemporal correspondence across unsynchronized cameras. We validate Phase-DGS under temporal offsets, frame drops, and camera freezes, achieving up to +8.5 dB PSNR improvement over baselines while enabling seamless integration with existing DGS backbones including RealTime-4DGS and FreeTimeGS.
Primary: Electronics and Telecommunications Research Institute (ETRI)
All Institutions: Electronics and Telecommunications Research Institute (ETRI), Seoul National University
The paper presents a practical and effective solution to the synchronization problem in dynamic scene reconstruction by introducing a semantic phase representation. While the novelty is moderate and the evaluation relies on simulated errors, the technical contribution is significant for enabling robust 4D Gaussian Splatting in real-world, imperfect capture environments.
The paper proposes Phase-DGS, a framework that replaces absolute timestamps in Dynamic Gaussian Splatting (DGS) with a learned "semantic phase" derived from image content. The core innovation is the use of a frozen DINOv2 ViT encoder followed by a lightweight MLP to project visual features into a 1D phase space, coupled with a learnable "feature bank" to enforce cross-camera consistency. The method is modular and plug-and-play, integrating with existing DGS backbones like FreeTimeGS and RealTime-4DGS. The approach is clever in that it bypasses the need for explicit temporal alignment or hardware synchronization by relying on the semantic similarity of motion states. However, the reliance on a single scalar phase to represent the entire temporal state of a scene is a strong assumption that limits its applicability to complex, non-periodic, or multi-object independent motions.
The experiments are conducted on two standard datasets (Nvidia Dynamic Scene Dataset and SelfCap) under simulated unsynchronized conditions (Shifted, Dropped). The paper reports significant PSNR improvements (up to +8.5 dB) over baselines like VisualSync and Sync-NeRF under severe asynchrony. The ablation studies effectively demonstrate the necessity of the monotonic loss and the learnable feature bank. However, the evaluation relies heavily on simulated errors rather than real-world unsynchronized captures, which may not fully capture the complexity of actual hardware drift or network-induced packet loss. The comparison with Sync-NeRF is somewhat unfair due to configuration mismatches noted in the text.
The paper provides detailed implementation specifics, including hyperparameters, optimizer settings, and compute resources (RTX 5090). The code and project page are available. The use of standard pretrained models (DINOv2) and open-source datasets enhances reproducibility. The manual ROI cropping step, while acknowledged, introduces a minor reproducibility hurdle, though the authors state they will release coordinates.
The method struggles with highly periodic motions (e.g., the Balloon scene) where visual patterns repeat, leading to ambiguous phase assignments. It assumes a single global phase per frame, which may not suffice for scenes with multiple independent moving objects at different speeds. The evaluation is limited to simulated asynchrony; performance on truly wild, unsynchronized real-world data is not tested. The method requires a test-time search window for phase refinement, adding a slight inference overhead.
This work addresses a practical bottleneck in 4D reconstruction: the cost and complexity of camera synchronization. By enabling robust reconstruction from unsynchronized video, it lowers the barrier to entry for dynamic scene capture, potentially benefiting applications in autonomous driving, sports analytics, and immersive media where perfect hardware sync is infeasible. The plug-and-play nature ensures it can be adopted by the broader 4DGS community with minimal effort. The paper presents a practical and effective solution to the synchronization problem in dynamic scene reconstruction by introducing a semantic phase representation. While the novelty is moderate and the evaluation relies on simulated errors, the technical contribution is significant for enabling robust 4D Gaussian Splatting in real-world, imperfect capture environments.
Computer vision models learn hand action only from its visual consequences, without access to the underlying embodied motor signals. We address this absence with MotorSense, the first large-scale paired egocentric video and surface electromyography (EMG) dataset. MotorSense pairs bimanual EMG bands with egocentric video and yields synchronized video, bimanual EMG and IMU, and audio across 100 hours of unscripted manipulation. EMG offers a powerful scalable alternative to manual annotation: it captures finger movement, hand configuration, and the forces exerted between hand and environment as a dense signal free of occlusion and labeler bias, whereas manual annotation is costly, subjective, and unreliable under occlusion. We use MotorSense to learn sensorimotor alignment, producing an organization of action that is shared across the visual and motoric spaces, through cross-modal pre-training between video and EMG. The resultant vision models are then transferred to downstream tasks that benefit from sensorimotor alignment: in this case, action recognition, 3D hand reconstruction, and 3D contact reconstruction. We further demonstrate a generative application in which EMG motor signals are used to condition a video diffusion model, yielding videos of fine-grained hand manipulation. Across four action-centric computer vision tasks and eight popular video datasets, we demonstrate that sensorimotor alignment provides a stronger foundation for visual representation learning than conventional self-supervised learning methods over video, outperforming every baseline across all tasks and datasets evaluated.
Primary: Massachusetts Institute of Technology
All Institutions: Massachusetts Institute of Technology, Carnegie Mellon University, University of Maryland
The paper introduces MotorSense, a large-scale paired egocentric video and EMG dataset, and demonstrates that cross-modal sensorimotor alignment significantly enhances action-centric computer vision tasks. By leveraging EMG as a privileged supervisory signal in a CLIP-style pre-training framework, the authors achieve consistent state-of-the-art results in action recognition, 3D hand reconstruction, and contact prediction, while also enabling fine-grained control of video diffusion models through learned motor codes.
The paper introduces a novel cross-modal pre-training framework that aligns egocentric video with surface electromyography (EMG) signals. The core methodological contribution is the use of EMG as a "privileged supervisory signal" in a CLIP-style contrastive objective. By jointly training a video encoder and an EMG encoder, the authors create a shared embedding space where visual representations are shaped by the underlying motor intent, which is then discarded at inference time. This approach is technically sound, leveraging the dense, occlusion-free nature of EMG to provide a stronger inductive bias for action-centric vision tasks than standard self-supervised video learning. The inclusion of a VQ-VAE to learn discrete "motor codes" for conditioning video diffusion models is a creative extension, though the reliance on the Video2LoRA framework limits the novelty of the generative component itself.
The experimental evaluation is extensive and rigorous. The authors demonstrate consistent improvements over strong baselines (SimCLR, MoCo, BYOL, and ImageNet/MAE initialization) across four distinct task families: action recognition (EPIC-Kitchens, SSv2, Ego-Exo4D), 3D hand reconstruction (ARCTIC, H2O, HOT3D), 3D contact reconstruction, and video generation. The gains are particularly notable in the frozen-backbone setting for action recognition, suggesting the pre-training captures robust, transferable features. The comparison against RGB-only self-supervised methods trained on the same data effectively isolates the contribution of the EMG signal. The qualitative results for video diffusion show clear improvements in fine-grained hand-object interaction fidelity compared to language or video-history conditioning.
The paper provides high reproducibility. It details the specific hardware (MindRove Armband, Meta Aria glasses), data collection protocols, preprocessing steps (filtering, normalization), and model architectures. The release of the 100-hour dataset is a significant asset. However, the specific implementation of the "Video2LoRA" and "HyperNetwork" components relies on external citations, and the code for the VQ-VAE motor code discovery is described but not fully detailed in the main text, potentially requiring access to the supplementary materials or code repository (not explicitly linked in the provided text, though a project page is mentioned) for full replication.
The primary limitation is the scale of the dataset (100 hours) compared to massive video datasets like Kinetics or Ego4D. The EMG hardware used is consumer-grade (8 channels), which may limit the granularity of muscle activity captured compared to research-grade high-density EMG. The video diffusion experiments are limited to a specific backbone (CogVideoX-Fun) and a lightweight LoRA approach, which may not generalize to all state-of-the-art video generation models. Additionally, the dataset is focused on kitchen/household manipulation, which may limit generalizability to other domains.
This work has significant potential impact on embodied AI, robotics, and VR/AR. By providing a dataset and method that links visual perception with motor intent, it offers a pathway to more intuitive human-robot interaction and better understanding of human actions from video. The motor-code conditioning for video generation could enable new tools for content creation and simulation. The dataset itself becomes a valuable resource for the community, addressing a gap in paired video-EMG data. The paper introduces MotorSense, a large-scale paired egocentric video and EMG dataset, and demonstrates that cross-modal sensorimotor alignment significantly enhances action-centric computer vision tasks. By leveraging EMG as a privileged supervisory signal in a CLIP-style pre-training framework, the authors achieve consistent state-of-the-art results in action recognition, 3D hand reconstruction, and contact prediction, while also enabling fine-grained control of video diffusion models through learned motor codes.
Pixel-space generative Transformers typically use one token per image patch, so capturing finer detail requires more tokens and makes global attention increasingly expensive. We observe that finer detail does not necessarily require more global communication: low-frequency structure benefits from global exchange, while high-frequency detail can be modeled locally. Based on this insight, we introduce SpecSlot, a pixel-space rectified-flow Transformer that decouples output detail from the length of the global attention sequence. SpecSlot expands each spatial anchor into multiple slots, each assigned a disjoint band of DCT coefficients; this separates anchor density from spectral subdivision. Only the lowest-frequency slot from each anchor participates in global attention, while all slots interact locally. This allows the model to increase spectral detail without enlarging the global sequence. A spatial head further adds a local correction to the spectral prediction, while the generative state, target, and loss remain in pixel space. On ImageNet-256, SpecSlot achieves a comparable final FID to a FLOPs-matched JiT control while converging faster. On ImageNet-512, it improves FID by 2.84 over the control with 3.9% fewer FLOPs, and comes within only 0.69 FID of JiT-B/16 at 38.2% lower cost. On text-to-image generation, SpecSlot surpasses JiT-B/8 on DPG-Bench by 1.06 at 40.6% lower forward cost.
Primary: Peking University
All Institutions: Peking University, Nanyang Technological University, SenseTime
SpecSlot introduces a role-coupled tokenization scheme for pixel-space generative Transformers that decouples spectral detail from global attention length, achieving significant efficiency gains and competitive performance on ImageNet and text-to-image tasks. The paper presents a well-motivated architectural innovation that addresses a key bottleneck in pixel-space generation, with strong empirical results demonstrating its effectiveness and efficiency compared to state-of-the-art baselines like JiT.
The paper proposes SpecSlot, a pixel-space rectified-flow Transformer that decouples output detail from global attention sequence length. The core innovation is the "Role-Coupled Tokenization" where each spatial anchor is expanded into multiple slots, each assigned a disjoint band of DCT coefficients. Only the lowest-frequency slot participates in global attention, while high-frequency slots interact locally. This is a clever architectural modification that leverages the spectral properties of natural images (low-frequency structure requires global context, high-frequency detail is local). The method remains in pixel space, avoiding the artifacts often associated with VAE-based latent diffusion, while mitigating the quadratic cost of attention in pixel-space models. The inclusion of a spatial head for local correction is a sensible addition to handle residual errors.
The experiments are conducted on ImageNet-256 and ImageNet-512, as well as text-to-image generation benchmarks (DPG-Bench). The results show that SpecSlot achieves comparable FID to FLOPs-matched JiT controls on ImageNet-256 with faster convergence. On ImageNet-512, it improves FID by 2.84 over the control with fewer FLOPs and comes within 0.69 FID of a larger JiT model at significantly lower cost. The text-to-image results show improvements over JiT-B/8. The comparisons are against strong baselines (JiT), and the efficiency gains are substantial.
The paper provides links to the project page, GitHub repository, and Hugging Face model weights. The methodology is described in sufficient detail in the provided text (though the full technical details are in the sections referenced but not fully expanded in the prompt's text snippet, the abstract and summary provide the core mechanism). The availability of code and weights significantly enhances reproducibility.
The method relies on DCT coefficient bands, which may not be optimal for all types of content or modalities. The local interaction of high-frequency slots might limit the model's ability to capture long-range high-frequency correlations, although this is argued to be less critical. The computational overhead of managing multiple slots per anchor needs careful engineering to ensure the efficiency gains are realized in practice.
This work offers a viable path for pixel-space generative models to scale to higher resolutions without the prohibitive cost of full global attention. It could influence the design of future pixel-space Transformers and potentially other modalities where spectral decomposition is applicable. The efficiency gains make high-quality pixel-space generation more accessible. SpecSlot introduces a role-coupled tokenization scheme for pixel-space generative Transformers that decouples spectral detail from global attention length, achieving significant efficiency gains and competitive performance on ImageNet and text-to-image tasks. The paper presents a well-motivated architectural innovation that addresses a key bottleneck in pixel-space generation, with strong empirical results demonstrating its effectiveness and efficiency compared to state-of-the-art baselines like JiT.
We present Youtu-Parsing-Omni, an omni-modal parser that converts documents, images, audio, and video into structured records grounded in space and time, providing a perception layer for agents that read, locate, and cite evidence across modalities. Its design rests on one encoder for every modality and one schema for every task. Youtu-Omni-Encoder, initialized from a pre-trained language model to strengthen text reading, encodes all modalities with shared weights in a unified space-time coordinate system, and a few fusion layers let video frames attend to concurrent audio, bringing cross-modal fusion and temporal alignment into perception itself. OmniSchema unifies the outputs of all ten parsing tasks, allowing a single self-evolving post-training procedure, OmniSchema-Aware On-Policy Distillation (OSAD), to provide a shared token-role-based objective across tasks. In each round, the model is trained on its own rollouts against token-level targets from a reference-conditioned copy of itself, which is then refreshed with the trained weights, so that the parser and its supervision co-evolve. With a single 5 B-parameter checkpoint, Youtu-Parsing-Omni achieves state-of-the-art performance on OmniDocBench v1.6 (96.96 Overall) and leads all open-weight models on OmniParsingBench (75.08 Avg.), with its advantage concentrated on audio and video. It also achieves the best overall chart-parsing score (95.02) and the highest exact chemical-fingerprint agreement among compared systems, and matches a dedicated optical-music model in character-level transcription of typeset sheet music while staying far ahead of general-purpose systems. Code and model weights are publicly available.
Primary: Tencent Youtu Lab
All Institutions: Tencent Youtu Lab
Youtu-Parsing-Omni introduces a unified encoder and schema for omni-modal parsing, achieving state-of-the-art results on multiple benchmarks. The paper presents a novel architecture that integrates cross-modal fusion into the perception layer and a self-evolving training procedure, demonstrating strong performance across documents, images, audio, and video.
The paper proposes Youtu-Parsing-Omni, a unified architecture for parsing documents, images, audio, and video into structured records. The core methodological contribution is the "Youtu-Omni-Encoder," which initializes from a pre-trained language model and uses shared weights to encode all modalities in a unified space-time coordinate system. A key architectural innovation is the inclusion of fusion layers that allow video frames to attend to concurrent audio, integrating cross-modal fusion directly into the perception layer. The training methodology introduces "OmniSchema-Aware On-Policy Distillation" (OSAD), a self-evolving post-training procedure where the model is trained on its own rollouts against token-level targets from a reference-conditioned copy of itself. This allows the parser and its supervision to co-evolve, unifying the outputs of ten different parsing tasks under a single schema.
The paper reports state-of-the-art performance on OmniDocBench v1.6 (96.96 Overall) and leads all open-weight models on OmniParsingBench (75.08 Avg.). The results highlight a significant advantage in audio and video parsing compared to general-purpose systems. It also achieves the best overall chart-parsing score (95.02) and high exact chemical-fingerprint agreement. The model matches dedicated optical-music models in character-level transcription of typeset sheet music. The evaluation covers a wide range of modalities and tasks, demonstrating the versatility of the unified approach.
The paper states that code and model weights are publicly available, which is a strong positive for reproducibility. However, the provided text does not include specific URLs or detailed hyperparameter settings, making it difficult to verify the exact implementation details without accessing the external repository. The description of the OSAD training procedure is high-level, and the specific mechanics of the "reference-conditioned copy" and "token-role-based objective" would require the code to fully understand and replicate.
The paper relies heavily on the performance of the pre-trained language model initialization, which may limit its applicability to domains where text is not the primary signal. The "self-evolving" nature of the OSAD training could be computationally expensive and potentially unstable, though this is not explicitly discussed. The evaluation is primarily on benchmark datasets, and real-world robustness to noisy or unstructured data is not thoroughly tested. The lack of specific URLs in the text prevents immediate verification of the claimed open-source availability.
This work has significant potential to impact the field of multimodal AI by providing a unified perception layer for agents that need to read, locate, and cite evidence across multiple modalities. The ability to parse documents, images, audio, and video into structured records using a single encoder and schema could simplify the development of complex AI agents. The focus on spatial and temporal grounding is particularly relevant for applications in robotics, autonomous systems, and advanced document analysis. Youtu-Parsing-Omni introduces a unified encoder and schema for omni-modal parsing, achieving state-of-the-art results on multiple benchmarks. The paper presents a novel architecture that integrates cross-modal fusion into the perception layer and a self-evolving training procedure, demonstrating strong performance across documents, images, audio, and video.
Effective multi-agent collaboration requires finalizing solutions from reasoning that is distributed across the individual attempts of many agents. However, useful intermediate reasoning can remain unexploited when collaboration is limited to voting on conclusions reached among individual agents, or aggregating long, noisy collections of reasoning trajectories in context. In this work, we propose TTT-Consolidate, a framework that uses multi-agent reasoning trajectories as adaptation data to improve a model's subsequent reasoning. TTT-Consolidate jointly consolidates trajectories into parametric representations through test-time training, then generates a new solution by conditioning on the consolidated information. We meta-learn how to consolidate using bi-level optimization at training time, where an inner loop encodes reasoning trajectories into a lightweight LoRA adapter, and an outer loop learns meta-parameters for the inner loop such that the consolidation is helpful for the model to synthesize a final solution. As no large-scale corpus of diverse reasoning trajectories exists for training aggregation methods, we construct the OpenParallelThinking corpus composed of 12,810 math problems paired with a mixture of trajectories from six LLMs. Evaluated on challenging math and STEM benchmarks, TTT-Consolidate outperforms the best aggregation baselines by up to 20 percentage points, even under matched compute and runtime settings, with the gap widening at larger model scales. TTT-Consolidate also recovers from unsuccessful individual trajectories more robustly, an ability that improves as the number of available (even incorrect) trajectories increases. Finally, we demonstrate that TTT-Consolidate benefits from both SFT and RL-based outer loop objectives. We will release TTT-Consolidate and OpenParallelThinking for use in future work on reasoning aggregation.
Primary: ETH Zurich
All Institutions: ETH Zurich, University of Zurich, Google, Sony Group Corporation
The paper introduces a novel meta-learning framework for consolidating multi-agent reasoning trajectories using test-time training, significantly improving solution accuracy and robustness. By shifting from simple aggregation to parametric integration via LoRA adapters, TTT-Consolidate demonstrates substantial performance gains over existing baselines, offering a promising direction for enhancing LLM reasoning capabilities in collaborative settings.
The paper proposes TTT-Consolidate, a framework that leverages test-time training (TTT) to consolidate multi-agent reasoning trajectories into parametric representations via LoRA adapters. The core innovation lies in the bi-level optimization strategy: an inner loop fine-tunes a lightweight adapter on the collected trajectories, while an outer loop meta-learns the parameters governing this consolidation process to maximize the quality of the final synthesized solution. This approach shifts the paradigm from simple aggregation (voting or context stuffing) to parametric integration of distributed reasoning. The methodology is sound, leveraging established techniques (LoRA, meta-learning) in a novel configuration specifically tailored for reasoning aggregation. The construction of the OpenParallelThinking corpus is a significant methodological contribution, providing a necessary resource for training such aggregation methods.
The experiments are rigorous, evaluating the method on challenging math and STEM benchmarks. The claim of outperforming baselines by up to 20 percentage points is substantial, particularly under matched compute settings. The analysis of robustness to incorrect trajectories is a strong point, demonstrating that the method can extract value from noisy data, which is a common real-world scenario. The scaling behavior, where the gap widens with model size, suggests the method's potential for larger, more capable models. However, the specific benchmarks used and the exact baseline implementations should be scrutinized to ensure fair comparison, especially regarding the "matched compute" claim.
The authors state they will release TTT-Consolidate and the OpenParallelThinking corpus, which is highly positive for reproducibility. The use of standard techniques like LoRA and meta-learning makes the implementation details likely to be transparent. However, without access to the code at the time of review, full reproducibility cannot be guaranteed, but the commitment to release is a strong indicator of good practice.
The method relies on the availability of multiple reasoning trajectories, which may not always be feasible in latency-sensitive applications. The bi-level optimization process could be computationally expensive during the meta-learning phase, although the test-time inference cost is mitigated by the lightweight LoRA adapter. The generalization of the method to non-mathematical reasoning tasks is not extensively explored.
This work has significant implications for multi-agent systems and LLM reasoning. By providing a way to effectively consolidate distributed reasoning, it enables more robust and accurate problem-solving in complex domains. The release of the OpenParallelThinking corpus will benefit the broader community by providing a standardized dataset for evaluating reasoning aggregation methods. The approach could be extended to other domains where multiple expert opinions or reasoning paths need to be synthesized. The paper introduces a novel meta-learning framework for consolidating multi-agent reasoning trajectories using test-time training, significantly improving solution accuracy and robustness. By shifting from simple aggregation to parametric integration via LoRA adapters, TTT-Consolidate demonstrates substantial performance gains over existing baselines, offering a promising direction for enhancing LLM reasoning capabilities in collaborative settings.
Generative spoken language models (SLMs) offer a route to learning linguistic structure purely from speech, yet they lag far behind text-based models in both capability and data scaling efficiency. We identify two bottlenecks underlying this gap: the limited linguistic quality of existing speech units and the reliance on a single stream of discrete units to represent speech. Multiple codebooks are widely used to encode acoustic features but rarely to represent speech semantics. We present GSLM2, which learns and models complementary properties of speech at multiple levels of abstraction. Its encoder, SpidR 2.0, uses self-supervised learning with differentiable prediction heads to induce high-quality discrete units at multiple levels. A multi-stream SLM then jointly models these units using a shared temporal backbone, stream-specific predictors, and an autoregressive head that captures cross-stream dependencies. Across training data sizes and model scales, up to 6M hours and 6.5B parameters, GSLM2 consistently improves linguistic competence while scaling more efficiently with data. As a result, it substantially narrows the gap between spoken and text-based LMs; across read and conversational speech domains, we shrink the gap by about two orders of magnitude on sWuggy, sBLIMP, and tSC, while also outperforming competitive speech-text systems on all three tasks. We open-source the training code and model checkpoints at https://github.com/facebookresearch/gslm2.
Primary: Meta Superintelligence Labs
All Institutions: Meta Superintelligence Labs
GSLM2 introduces a multi-stream spoken language modeling framework that learns complementary discrete units at multiple levels of abstraction via differentiable self-supervised pretraining, significantly narrowing the data-efficiency gap between speech and text language models. The paper presents a rigorous and scalable approach to textless language modeling, demonstrating that leveraging multi-level representations from self-supervised encoders allows SLMs to achieve linguistic competence comparable to text LMs with far less data, offering a promising path for speech-native AI applications.
The paper introduces GSLM2, a framework that addresses the data-efficiency gap between spoken language models (SLMs) and text-based LMs. The core methodological contribution is twofold: (1) SpidR 2.0, an encoder that uses self-supervised learning with differentiable prediction heads to induce high-quality discrete units at multiple levels of abstraction (layers) rather than relying on post-hoc k-means on a single layer; and (2) a multi-stream SLM architecture that jointly models these complementary unit streams using a shared temporal backbone, stream-specific predictors, and an autoregressive generative head. The use of entropy regularization to stabilize the code distributions across layers is a significant technical improvement over prior work like SpidR. The approach effectively decouples the representation of fine-grained phonetic details from higher-level semantic regularities, allowing the model to leverage both for efficient language modeling.
The experiments are extensive and rigorous. The authors scale models up to 6.5B parameters and 6M hours of data, which is a substantial computational effort. They demonstrate that GSLM2 shrinks the gap to text-based LMs by about two orders of magnitude on key benchmarks (sWuggy, sBLIMP, tSC). The comparison against previous SLMs (including Moshi) shows clear superiority in parameter and data efficiency. The generative evaluation, while relying on ASR and LLM judges (which introduces noise), still shows consistent improvements in coherence and perplexity. The ablation studies on layer-wise representations provide strong evidence for the multi-level hypothesis.
The paper is highly reproducible. The authors explicitly state they open-source the training code and model checkpoints. The reproducibility statement is detailed, covering datasets, preprocessing, model configurations, and evaluation protocols. The use of standard datasets (Libri-Light, LibriSpeech) and public benchmarks further enhances reproducibility.
The paper acknowledges that it does not explicitly preserve acoustic properties like prosody, which is crucial for speech resynthesis. Multi-stream modeling increases computational cost. The evaluation of generation quality relies on indirect metrics (ASR-PPL, LLMJ) which may conflate language modeling errors with TTS/ASR errors. The analysis of how individual streams contribute to the final performance is limited to empirical probing.
This work has significant implications for the development of speech-first AI systems. By demonstrating that linguistic structure can be learned efficiently from speech alone using multi-level representations, it challenges the prevailing reliance on text supervision for speech models. This could lead to more robust, language-agnostic speech models that do not require written language resources. The open-sourcing of the code and models will likely accelerate research in this area. GSLM2 introduces a multi-stream spoken language modeling framework that learns complementary discrete units at multiple levels of abstraction via differentiable self-supervised pretraining, significantly narrowing the data-efficiency gap between speech and text language models. The paper presents a rigorous and scalable approach to textless language modeling, demonstrating that leveraging multi-level representations from self-supervised encoders allows SLMs to achieve linguistic competence comparable to text LMs with far less data, offering a promising path for speech-native AI applications.
Predicting how the world evolves in response to robot actions is central to policy learning, evaluation, and planning. However, many action-conditioned world models predict future observations in 2D image or video space without explicitly modeling the underlying 3D scene, making spatial consistency across viewpoints and over long-horizon interactions challenging. We introduce 3D World Simulator, an action-conditioned multimodal world modeling framework that jointly predicts appearance, geometry, and tactile observations from a unified 3D representation. Our approach combines a hybrid particle-voxel autoencoder that compresses point clouds into compact 3D voxel latents with an action-conditioned latent dynamics model that predicts their evolution. The shared 3D representation supports geometrically consistent rendering from arbitrary viewpoints and flexible integration of spatially aligned tactile signals. Our experiments demonstrate interactive, long-horizon, multimodal prediction at 50 Hz. We then show how our model enables scalable data generation for policy training and evaluation using tactile and multi-view visual observations, including wrist-mounted and novel camera views. Furthermore, we scale our method to large datasets containing diverse objects and demonstrate model-based planning with unseen objects and robot configurations. Together, these results highlight the potential of hybrid particle-voxel dynamics as a foundation for multimodal 3D world simulation.
Primary: University of Michigan
All Institutions: Columbia University, UC Berkeley, University of Michigan
The paper introduces a unified 3D world model that jointly predicts appearance, geometry, and tactile signals, demonstrating superior spatial consistency and enabling scalable data generation for robotic policy learning and evaluation.
The paper proposes a "3D World Simulator" that unifies appearance, geometry, and tactile signals within a single 3D latent space. The core technical contribution is a hybrid particle-voxel autoencoder that compresses point clouds into sparse voxel latents and decodes them into 3D Gaussians and tactile particles. This is coupled with an action-conditioned 3D U-Net dynamics model that predicts the evolution of these latents. The approach is sound, leveraging established components (sparse convolutions, 3D Gaussians, U-Nets) in a novel configuration to solve the specific problem of multimodal 3D consistency. The integration of tactile signals as 3D particles within the same space as visual geometry is a clever and effective design choice that ensures spatial alignment without complex cross-modal attention mechanisms.
The experiments are extensive and well-structured. The authors evaluate the model on six diverse manipulation tasks (rigid, articulated, deformable) using an ALOHA bimanual robot, as well as on the large-scale Deform360 dataset. They provide quantitative comparisons against strong 2D video baselines (Cosmos, Robot4DGen) and 3D particle baselines (PointWorld, ParticleFormer). The results show clear improvements in geometric consistency (Chamfer Distance) and multi-view rendering fidelity. The downstream applications—policy training, policy evaluation correlation, and zero-shot planning—are convincingly demonstrated, particularly the strong correlation between simulated and real-world policy success rates, which is a critical metric for world models.
The paper provides detailed architectural descriptions, loss functions, and training schedules in the appendix. It specifies hardware requirements (RTX 5090) and training times. However, the code and pre-trained models are not linked in the provided text, and the reliance on specific proprietary or recent datasets (Deform360, FlexiTac) may limit immediate reproducibility for the broader community without access to those specific resources.
The model is limited to a specific workspace and requires composition with static 3D assets for larger scenes, which is a significant constraint for general-purpose world simulation. The "zero-shot" planning on unseen objects relies on a pre-trained model from a large dataset, so true generalization to entirely new physics or object categories is not fully tested. The computational cost of 3D U-Net dynamics, while manageable, is higher than 2D video diffusion models, potentially limiting real-time applications on edge devices.
This work has high potential impact in the robotics community by providing a robust framework for generating synthetic data that preserves 3D and tactile consistency. It addresses a key bottleneck in policy learning: the lack of diverse, physically consistent training data. The ability to evaluate policies in a simulator that correlates strongly with real-world performance could accelerate the development of robotic manipulation systems. The paper introduces a unified 3D world model that jointly predicts appearance, geometry, and tactile signals, demonstrating superior spatial consistency and enabling scalable data generation for robotic policy learning and evaluation.
AIRoA-0 (A-0) is a bimanual manipulation system learned from 16,151 hours of handheld demonstrations, with no robot data collected at any stage. The demonstrations are recorded with YUBI, our open-source handheld gripper, which lets people record manipulation data anywhere with no robot present. From a single instruction, A-0 sorts more than a dozen small parts into their numbered cells and changes the mop pads and brush of a robot vacuum; among other tasks, it unzips a pencil case and folds a towel. It performs these tasks using only a camera on each wrist, with no head or scene camera, at a site where none of its training data was recorded. Across the ten tasks of our evaluation, it averages a success rate of 78%, against 27% for the strongest of four other vision-language-action models post-trained on the same task demonstrations. We collect no teleoperation data and use no reinforcement learning or on-robot corrections, and the robot appears only at test time. The system learns which regions of each wrist view matter for the task, predicts the stage the task has reached, and decodes actions as a continuous stream. A-0 is embodiment-agnostic: it runs on robots that mount the YUBI gripper, such as a bimanual Franka, TX-G2, OpenArm and UR. We are releasing the dataset, to our knowledge the largest open dataset of handheld-gripper demonstrations with task and subtask instructions, together with the pre-trained weights and the post-training recipe. Videos of A-0 on every task are on the project page, https://a-0.airoa.io. Anyone can build the gripper, record a task of their own and post-train the model on it in under four hours on a single node of eight H200 GPUs.
Primary: Preferred Networks (implied by NEDO funding and YUBI context, though not explicitly named in text)
All Institutions: Preferred Networks (inferred), NEDO (Funding Agency)
[One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents AIRoA-0, a bimanual manipulation system trained exclusively on 16,151 hours of handheld gripper data, achieving state-of-the-art performance on long-horizon tasks with high generalization to unseen sites and robot embodiments, thereby validating the scalability and effectiveness of human-centric data collection for robotic learning.
The paper introduces AIRoA-0 (A-0), a bimanual manipulation system that leverages a large-scale dataset of handheld gripper demonstrations (YUBI) to train a vision-language-action model without any robot-specific data. The methodology is robust, utilizing a "See Less, Specify More" (S2) objective with a visual evidence budget gate to focus attention on relevant regions of the wrist camera views. Key innovations include the use of anchored actions relative to the gripper pose to ensure embodiment agnosticism, a stage prediction head to handle long-horizon task ambiguity, and a non-uniform streaming decoding scheme for continuous, low-latency action generation. The integration of proprioceptive memory and the specific handling of language at both task and subtask levels are well-designed components that address the challenges of learning from human-centric data.
The experimental evaluation is strong, demonstrating a 78% success rate on ten complex bimanual tasks, significantly outperforming four other state-of-the-art VLA models (27% average). The evaluation is conducted at a site where no training data was recorded, highlighting the system's generalization capabilities. The paper also demonstrates embodiment agnosticism by running the same weights on four different robot platforms (Franka, TX-G2, OpenArm, UR). The inclusion of scaling laws for pre-training data and the robustness to scene disturbances (e.g., moving bins) further strengthens the empirical claims.
The paper claims high reproducibility by releasing the full dataset (16,151 hours), pre-trained weights, post-training recipe, and the open-source YUBI gripper design. The claim that post-training can be done in under four hours on a single node of eight H200 GPUs is a significant contribution to accessibility. However, the specific hardware requirements for data collection and the exact implementation details of the "visual evidence budget" gate are described at a high level, which may pose challenges for full replication without the provided code.
The system relies heavily on the specific YUBI gripper for data collection and deployment, which may limit immediate adoption by labs without access to this hardware. The evaluation, while strong, is limited to ten specific tasks, and the generalization to completely novel task categories or objects outside the training distribution is not extensively explored. The reliance on wrist cameras only, while a strength for embodiment agnosticism, limits the field of view and may struggle with tasks requiring global scene understanding.
This work has significant broader impact by demonstrating that large-scale, high-quality manipulation data can be collected without robots, lowering the barrier to entry for training advanced manipulation policies. The release of the dataset and tools could accelerate research in robotic manipulation and embodied AI. The embodiment-agnostic nature of the system suggests potential for widespread deployment across different robot platforms, promoting standardization in the field. [One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents AIRoA-0, a bimanual manipulation system trained exclusively on 16,151 hours of handheld gripper data, achieving state-of-the-art performance on long-horizon tasks with high generalization to unseen sites and robot embodiments, thereby validating the scalability and effectiveness of human-centric data collection for robotic learning.
Dexterous in-hand manipulation is commonly formulated around explicit object pose estimation, yet reliable pose tracking is challenging under severe hand-object occlusions and often requires object-specific supervision. This dependence creates a bottleneck between perception and control that can limit robustness and scalability. We present an end-to-end visuomotor framework for dexterous in-hand reorientation that operates on RGB observations and proprioception without intermediate pose estimation. Our approach jointly learns visual representations and control policies through behavior distillation from privileged state teachers and auxiliary reconstruction objectives. Furthermore, we introduce image-goal-conditioned reorientation, allowing desired object configurations to be specified directly through images rather than canonical pose representations. We evaluate our approach across a diverse set of object geometries and instances in simulation, where it consistently outperforms pose-based methods and policies built on pretrained visual encoders. Finally, we demonstrate direct sim-to-real transfer of a single image-goal-conditioned policy across five real-world objects, showing that accurate dexterous reorientation can be achieved without explicit pose estimation. For videos and code release, please refer to the project website: https://rffr.leggedrobotics.com/works/goaldex/
Primary: ETH Zurich
All Institutions: ETH Zurich, Swiss National Science Foundation, NVIDIA, Meta
GoalDex introduces an end-to-end visuomotor framework for dexterous in-hand manipulation that eliminates the need for explicit object pose estimation by jointly learning visual representations and control policies through behavior distillation and image-goal conditioning. The paper demonstrates robust sim-to-real transfer across diverse objects, offering a scalable and robust alternative to traditional pose-based pipelines in dexterous robotics.
The paper proposes GoalDex, an end-to-end visuomotor framework for dexterous in-hand manipulation that bypasses explicit object pose estimation. The core methodological contribution is the use of behavior distillation from privileged state teachers combined with auxiliary reconstruction objectives to learn robust visual representations directly from RGB and proprioception. A significant innovation is the introduction of image-goal conditioning, where desired object configurations are specified via images rather than canonical pose vectors. This allows for more flexible goal specification and leverages the visual modality for both state and goal representation. The integration of a Vision-Language Model (VLM) to select visual goals from natural language instructions further extends the utility of the framework, bridging high-level semantic commands with low-level dexterous control.
The evaluation is conducted primarily in simulation across a diverse set of object geometries, demonstrating consistent outperformance over pose-based methods and policies using pretrained visual encoders. A critical strength of the experimental design is the direct sim-to-real transfer of a single image-goal-conditioned policy to five distinct real-world objects. The zero-shot transfer capability without explicit pose estimation in the real world is a strong empirical result, validating the robustness of the learned representations. The comparison against baseline methods that rely on explicit pose tracking highlights the practical advantages of the proposed approach in scenarios where pose estimation is unreliable due to occlusions.
The paper mentions code release via the project website. The methodology relies on standard deep learning techniques (distillation, reconstruction) applied to a specific robotics setup. While the specific hyperparameters and simulation environment details are likely in the appendix (not fully provided in the excerpt), the use of standard simulation-to-real transfer protocols and clear description of the training objectives (privileged teachers, reconstruction) provides a reasonable path for reproduction. The reliance on specific hardware (dexterous hand) may limit immediate reproducibility for labs without such setups, but the simulation code should be accessible.
The primary limitation is the reliance on simulation for training, which requires careful domain randomization to achieve zero-shot transfer. The method is tested on a specific set of objects and a specific dexterous hand; generalization to significantly different hand morphologies or object categories beyond the tested set is not fully established. The use of a VLM for goal selection adds computational overhead and potential latency, which may not be suitable for all real-time applications. Additionally, the "privileged state teacher" setup requires access to ground truth state information during training, which is a standard assumption in sim-to-real but still a dependency.
This work has significant implications for the field of dexterous robotics by demonstrating that explicit pose estimation is not a strict prerequisite for high-performance manipulation. By shifting the burden from geometric perception to end-to-end visuomotor learning, it opens the door to more robust systems in unstructured environments. The image-goal conditioning paradigm could be adopted in other manipulation tasks, providing a more intuitive interface for specifying goals. The successful sim-to-real transfer reinforces the viability of simulation-based learning for complex dexterous tasks, potentially reducing the cost and time associated with real-world data collection. GoalDex introduces an end-to-end visuomotor framework for dexterous in-hand manipulation that eliminates the need for explicit object pose estimation by jointly learning visual representations and control policies through behavior distillation and image-goal conditioning. The paper demonstrates robust sim-to-real transfer across diverse objects, offering a scalable and robust alternative to traditional pose-based pipelines in dexterous robotics.
Most agentic robotic systems use vision-language models (VLMs) either to orchestrate predefined skills or to directly generate actions, constraints, and control programs. The former is limited by the available skill library, while the latter requires VLMs to reason about precise geometry and contact. In this paper, we propose WorldSketch, a new paradigm for agentic robotics in which a VLM constructs a world model rather than generating robot actions. The VLM acts as an agent that actively queries perception tools to build and visually validate a minimal kinematic world model, capturing only task-relevant objects, their interaction geometry, and possible motions. A geometric planner then jointly optimizes contact and whole-body motion on the resulting model. Moreover, the world model is self-improving: execution feedback verifies outcomes, updates uncertain hypotheses, and enables the agent to revise the model when its predictions are contradicted. We demonstrate the generality of this paradigm across RoboCasa, LIBERO-PRO, and R2R-CE navigation, achieving zero-shot success rates of 91%, 97%, and 76%, respectively. On RoboCasa, WorldSketch outperforms the next-best zero-shot agent baseline by 58 percentage points while using 10 times fewer VLM input tokens; on LIBERO-PRO, it exceeds CaP-X by 82 percentage points. In navigation, it matches the success rate of step-by-step VLM agents while using 6 times fewer input tokens.
Primary: Unknown
All Institutions: Unknown
The paper introduces a novel "WorldSketch" paradigm where VLMs construct minimal kinematic world models for robotic planning, achieving state-of-the-art zero-shot success rates in manipulation and navigation tasks while significantly reducing token usage compared to code-generation agents. By decoupling semantic scene understanding from geometric motion planning, the method demonstrates that explicit, verifiable world models built by VLMs can outperform end-to-end action generation, offering a scalable and interpretable approach to general-purpose robotics.
The paper proposes "WorldSketch," a paradigm where a Vision-Language Model (VLM) constructs a minimal, kinematic world model (sketch) rather than generating actions directly. The VLM acts as an agent querying perception tools (depth lifting, point marking) to build a model of task-relevant entities (joints, links, parts). A geometric planner then optimizes contact and whole-body motion on this explicit model. The method includes a self-improving loop where execution feedback verifies outcomes and updates joint hypotheses via Bayes' rule. The methodology is sound, leveraging the VLM's semantic understanding for scene abstraction and the planner's precision for geometry. The separation of concerns (VLM for "what matters," Planner for "how to move") is a strong architectural choice that addresses the limitations of end-to-end VLA models and code-as-policy agents.
The evaluation is extensive, covering RoboCasa (kitchen manipulation), LIBERO-PRO (tabletop), and R2R-CE (navigation). The results are impressive, with 91% success on RoboCasa zero-shot, significantly outperforming baselines like CaP-X (33%) and Code as Policies (9%). The ablation studies are rigorous, isolating the impact of grounding methods (points vs. masks), overlay checks, and hypothesis testing. The comparison against trained VLAs (GR00T, pi0.5) demonstrates the superiority of the zero-shot agentic approach in generalization. The token efficiency analysis (10x fewer tokens than CaP-X) is a valuable practical insight.
The paper provides detailed descriptions of the sketch language, tool interfaces, and planner components. However, the system relies on proprietary VLMs (GPT-6.1-Sol, GPT-6-Astra) and specific simulation environments (RoboCasa365, MuJoCo). The exact prompts and backend logic are described but not fully open-sourced in the text provided. The reliance on specific VLM versions may limit immediate reproducibility for those without access to these models, but the architectural principles are clearly defined.
The system is evaluated primarily in simulation. Real-world sensor noise (depth errors >1cm) degrades performance. The current sketches cover only rigid parts with single joints, excluding deformable objects and complex multi-joint chains. The VLM latency (15s median) adds significant time to the loop. Contact-rich execution on small/thin parts remains a challenge, with failures often due to gripper limitations rather than planning errors.
This work offers a compelling alternative to the dominant VLA trend, suggesting that explicit world modeling by VLMs can achieve superior zero-shot generalization with lower computational cost. It bridges the gap between high-level semantic reasoning and low-level geometric planning. The paradigm of "sketching" the world is intuitive and could be extended to other robotic domains, potentially influencing how future agentic systems are designed to interact with physical environments. The paper introduces a novel "WorldSketch" paradigm where VLMs construct minimal kinematic world models for robotic planning, achieving state-of-the-art zero-shot success rates in manipulation and navigation tasks while significantly reducing token usage compared to code-generation agents. By decoupling semantic scene understanding from geometric motion planning, the method demonstrates that explicit, verifiable world models built by VLMs can outperform end-to-end action generation, offering a scalable and interpretable approach to general-purpose robotics.
A robot's next action can depend on recent motion or earlier visual evidence absent from its current observation. A recurrent visual-memory architecture can retain this history in a visual state, but its usefulness depends on what the state represents and how it is updated with new observations. We introduce Prediction-Regulated Memory (PREM), a framework that uses action-conditioned future prediction to guide both memory learning and updating in vision-language-action policies. During training, predicting future visual features provides temporal supervision for memory learning. During execution, PREM compares current visual features with the prediction made at the preceding decision and uses the spatially normalized prediction discrepancy to regulate updates to the visual state at each spatial location. The updated visual state supplies temporal context for action generation alongside current observations. Built on $π_{0.5}$, PREM improves average success rate from 9.63% to 20.74% on DOMINO dynamic manipulation tasks and from 14.4% to 83.8% on the RMBench M(1) memory-dependent setting. Further evaluations on LIBERO and LIBERO-Pro show improvements in general manipulation and robustness to spatial perturbations. PREM incurs a mean inference overhead of 7.7 ms on a single NVIDIA H800 GPU.
Primary: Shanghai Jiao Tong University
All Institutions: Shanghai Jiao Tong University
[One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper introduces a novel memory regulation mechanism for VLA policies that leverages action-conditioned future prediction to improve temporal context retention, demonstrating significant performance gains on memory-dependent and dynamic manipulation benchmarks with minimal computational overhead.
The paper proposes PREM (Prediction-Regulated Memory), a framework that integrates action-conditioned future prediction into the memory mechanism of Vision-Language-Action (VLA) policies. The core innovation is twofold: during training, future visual feature prediction serves as a temporal supervision signal for the memory state; during inference, the discrepancy between the predicted future visual state and the actual current observation is used to spatially regulate the update of the recurrent memory. This approach addresses the common issue in robotic manipulation where the current observation lacks sufficient context about recent dynamics or hidden states. By leveraging prediction errors to modulate memory updates, the method aims to retain relevant historical information while discarding noise. The method is built upon the $\pi_{0.5}$ architecture, a strong baseline in the field.
The experimental evaluation is robust, covering multiple benchmarks: DOMINO (dynamic manipulation), RMBench (memory-dependent tasks), LIBERO, and LIBERO-Pro. The reported improvements are substantial, particularly on RMBench M(1), where success rate jumps from 14.4% to 83.8%, and on DOMINO, from 9.63% to 20.74%. These results suggest the method effectively handles tasks requiring temporal context. The inclusion of LIBERO-Pro evaluations for robustness to spatial perturbations adds depth to the generalization claims. The inference overhead of 7.7 ms on an H800 GPU is reported as negligible, which is a practical advantage for real-time control.
The paper includes a detailed reproducibility statement, promising code and configuration files in the supplementary material. It specifies architectural dimensions, hyperparameters, and training schedules in the appendix. The use of standard benchmarks (LIBERO, DOMINO, RMBench) facilitates independent verification. The AI use statement is transparent, clarifying that AI was used only for editing and formatting, not for data generation or hypothesis formulation.
The primary limitation is the reliance on the $\pi_{0.5}$ backbone; it is unclear how well the PREM module generalizes to other VLA architectures or non-recurrent memory structures. The evaluation is limited to simulation and specific real-world setups (implied by the benchmarks), and long-horizon real-world deployment challenges (e.g., lighting changes, object deformations) are not extensively explored. The "prediction" mechanism assumes a certain degree of predictability in the environment, which may not hold in highly stochastic or adversarial settings.
This work contributes to the broader goal of creating robots that can learn from experience and maintain context over time. The technique of using prediction errors to regulate memory updates is a concept that could be adapted to other sequential decision-making tasks beyond robotics, such as autonomous driving or interactive agents. It addresses a fundamental challenge in embodied AI: how to efficiently compress and update historical information for future action planning. [One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper introduces a novel memory regulation mechanism for VLA policies that leverages action-conditioned future prediction to improve temporal context retention, demonstrating significant performance gains on memory-dependent and dynamic manipulation benchmarks with minimal computational overhead.
Recent work shows that a frozen robot policy performs far better inside an agent harness, where control code decides when to query the policy, consult vision tools, or command the arm directly. Yet today these harnesses are built by hand and redesigned for each new robot or benchmark. We show that harness design can be automated, offering a strong alternative to finetuning on a novel setup. In RoboArchitect, a coding model observes a robot's rollouts, diagnoses failures, and iteratively revises the harness around the frozen policy. The harness evolves on a set of episodes for its optimization and is evaluated on held-out scenes and tasks. Without updating any model weights, RoboArchitect raises average success from 9.7% to 43.3% on unseen scenes (4.5x) and from 12.7% to 40.2% on unseen tasks (3.2x), across multiple benchmarks and policies. The gains are largest where the benchmark is far from the policy's training data: wrapping a general-purpose policy, the evolved harness yields a specialist that often outperforms even the finetuned version. Intuitive control strategies are automatically designed, such as lifting a jammed arm clear or reaching in horizontally when an obstacle blocks the way from above. The harness also evolves its own persistent memory, carrying learned experience, such as effective grasp depths, into later decisions. These results suggest that substantial headroom in robotic manipulation lies in the agent harness, and its design can be effectively automated and optimized without weight updates.
Primary: Stanford University
All Institutions: Stanford University
[One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper introduces RoboArchitect, a method that automates the design of agent harnesses around frozen robot policies using iterative coding LLM optimization, demonstrating that substantial performance gains in robotic manipulation can be achieved without weight updates by evolving control logic and persistent memory, effectively turning general policies into specialists for specific setups.
The paper proposes "RoboArchitect," a framework that automates the design of an "agent harness" surrounding a frozen Vision-Language-Action (VLA) policy. The core methodology involves an iterative optimization loop where a coding LLM (Claude Opus 5) observes robot rollouts, diagnoses failure modes, and rewrites the control code (harness) to mediate between the environment and the frozen policy. The harness is not a fixed architecture but evolves to include specific control routines (e.g., jam recovery, force rescaling) and persistent memory structures. The approach is distinct from standard fine-tuning as it updates no model weights, relying instead on programmatic logic and external tool calls (VLMs, detectors). The novelty lies in treating the agent's control logic as an evolvable software artifact rather than a static wrapper, allowing the system to dynamically decide when to trust the policy vs. direct control.
The experiments are extensive, covering multiple benchmarks (LIBERO-Pro, RoboCerebra, VLABench, RoboCasa, SimplerEnv) and multiple policies (OpenVLA, Pi-0, MolmoAct-2, etc.). The evaluation protocols are rigorous, distinguishing between "unseen scenes" (same tasks, new layouts) and "unseen tasks" (new tasks, shared skills). The results show significant improvements (e.g., 9.7% to 43.3% on unseen scenes), and notably, the evolved harness often outperforms the fine-tuned version of the same policy in out-of-domain settings. The ablation on iterative refinement vs. one-shot design confirms the value of the self-improvement loop. The emergence of persistent memory (e.g., storing grasp depths) is a strong qualitative and quantitative finding.
The paper provides a GitHub link and detailed implementation sections. It specifies the exact coding model (Claude Opus 5), auxiliary tools (Qwen3, Molmo2, Grounding-DINO, SAM 3), and iteration limits. The cost analysis (<$150, <6 hours) is a strong point for practical reproducibility. However, the heavy reliance on a specific, proprietary coding model (Claude) may limit reproducibility for labs without access to that specific model version or API.
The primary limitation is that all evaluations are in simulation. The paper acknowledges this and suggests the interface should transfer to real robots, but no real-world validation is provided. Additionally, the method is capped at 5 iterations, and the saturation point is not fully explored. The complexity of the harness code could become a maintenance burden, though the paper argues it is interpretable. The reliance on a powerful coding LLM makes the method expensive in terms of API costs, though the paper argues this is cheaper than collecting new demonstrations.
This work has high potential impact by offering an alternative to the data-hungry fine-tuning paradigm in robotics. If the "harness optimization" approach scales to real-world robots, it could drastically reduce the cost of adapting robot policies to new environments. It bridges the gap between code-as-policy and learned VLA policies, suggesting a hybrid future where LLMs write the control logic around frozen neural networks. The concept of "evolving" the agent's software interface is a significant shift in how robotic agents are deployed. [One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper introduces RoboArchitect, a method that automates the design of agent harnesses around frozen robot policies using iterative coding LLM optimization, demonstrating that substantial performance gains in robotic manipulation can be achieved without weight updates by evolving control logic and persistent memory, effectively turning general policies into specialists for specific setups.
Realistic, fast robotics simulations could enable large-scale training of embodied AI models that transfer to the real world. However, despite decades of progress, available simulations cover only a narrow range of real-world tasks and physical phenomena. The core bottleneck is that creating better simulations requires significant expert effort and therefore does not scale. To address this, we introduce Gym-Anything for Robotics, a framework in which coding agents create realistic robotics simulations. Our core insight is that simulation creation is a coding agent task, so improvements in coding agents translate directly into richer and more faithful simulations. We select simulations based on U.S. GDP data to cover economically valuable physical work, define what makes a good simulation and a good task, and develop an adversarial creation-audit loop that checks each simulation and task against these definitions. Using this pipeline, we build RoboWorld, a set of 25 simulations with 25 test and 175 training RL environments of varying difficulty. Even the best frontier models fail on every RoboWorld test task, achieving an average reward of only 0.06. Building on RoboWorld, we introduce several new ways for frontier models to improve, including in-context learning across repeated attempts, a privileged teacher that reads the simulator state and teaches a student in context, self-assisted learning from the model's own privileged experience, and code policies that generate demonstrations across varied scenes to train vision-language-action models (VLAs) such as $π_{0.5}$. Each of these substantially improves performance. We release our pipeline, RL environments, and benchmark to facilitate future research. Code and data are available at https://gym-anything.github.io/robotics
Primary: Carnegie Mellon University
All Institutions: Carnegie Mellon University
The paper presents a novel framework for scaling robotics simulation creation using coding agents, resulting in a high-fidelity benchmark (RoboWorld) that exposes the limitations of current frontier models. While the lack of sim-to-real transfer is a significant gap, the technical depth of the simulations and the innovative use of LLMs for simulation generation represent a substantial contribution to the field.
The paper introduces "Gym-Anything for Robotics," a framework leveraging coding agents to automatically generate realistic robotics simulations. The core methodological innovation is the "adversarial creation-audit loop," where an agent creates a simulation and a separate verifier checks it against rigorous physical and task definitions. The paper demonstrates this by building "RoboWorld," a suite of 25 simulations covering economically valuable physical tasks (e.g., welding, sewing, fluid mixing, forklift operation). The simulations are notably detailed, incorporating specific physical laws (e.g., Hertz/Hunt-Crossley contact, Drucker-Prager powder mechanics, Bingham thin-film lubrication) and real-world object parameters (datasheet dimensions, material properties). The learning component explores several novel training strategies, including in-context learning across attempts, privileged teacher-student setups, and code policies for generating demonstrations for Vision-Language-Action (VLA) models.
The experimental evaluation is extensive in scope, covering 25 distinct simulation domains. The paper reports that even frontier models fail on the test tasks, achieving an average reward of only 0.06, which highlights the difficulty and realism of the benchmark. The authors demonstrate that their proposed learning methods (in-context learning, privileged teaching, code policies) substantially improve performance over baselines. The comparison table with prior simulations is detailed, showing that RoboWorld simulations include significantly more physics checks and adversarial evaluator cases than prior work. However, the evaluation is limited to simulation performance; there is no sim-to-real transfer evaluation, which is a critical gap for a robotics paper claiming to enable real-world training.
The authors state that the pipeline, RL environments, and benchmark are released. The project URL is provided. The level of detail in the simulation descriptions (e.g., specific solver parameters, material properties, robot configurations) suggests high reproducibility for the simulation setup. However, the reliance on coding agents for generation introduces variability, and the exact prompts or agent configurations used to generate the 25 simulations are not fully detailed in the provided text, which may limit exact reproduction of the generation process.
The primary limitation is the lack of sim-to-real validation. The paper argues that realistic simulations enable transfer, but does not demonstrate this. The benchmark is extremely difficult (average reward 0.06), which may be a barrier for adoption if the learning curves are not well-characterized. The reliance on coding agents for simulation creation may introduce subtle errors or biases that are not fully captured by the adversarial audit loop. The paper is also very long and complex, which may hinder accessibility.
This work has high potential impact by addressing a fundamental bottleneck in robotics research: the scalability of simulation creation. If the framework can be extended to generate thousands of high-fidelity simulations, it could enable large-scale pre-training of embodied AI models. The focus on economically valuable tasks (GDP-based selection) aligns with practical industrial applications. The release of the benchmark and pipeline will likely stimulate research in simulation-based RL and embodied AI. The paper presents a novel framework for scaling robotics simulation creation using coding agents, resulting in a high-fidelity benchmark (RoboWorld) that exposes the limitations of current frontier models. While the lack of sim-to-real transfer is a significant gap, the technical depth of the simulations and the innovative use of LLMs for simulation generation represent a substantial contribution to the field.