Last 7 Days (October 07 – October 13, 2026)
Recent incidents have highlighted the challenge of monitoring LLM agents and the danger of models deceiving people. We show that white-box deception detection via probes can be scaled up to frontier monitoring settings by collecting the largest deception dataset to date for training probes and introducing a novel probe architecture which can aggregate information across many layers and tokens. Our probes achieve 98.8% AUC in SHADE-Arena, surpassing an Opus 5.5 text-monitoring baseline, and show improved efficacy as the underlying model is scaled up. To push our probes to their limit, we test them on several cases where deception cannot be determined from the context alone. In these cases, which we refer to as introspective deception, the ground truth can only be determined through careful elicitation or thorough knowledge of a model's training data. In one such evaluation, we show that probes can distinguish transcripts containing a model's true hidden goal from other goals with an AUC of up to 99.7%. Our probes also readily detect deception on prominent open-weight models which lie about politically sensitive topics, and about their beliefs when put under pressure. We release our training dataset, dubbed FIBS, to help drive frontier deployment of effective probes, and encourage the community to expand upon it with further examples of deception and sabotage.
Primary: FAR.AI
All Institutions: FAR.AI, Coefficient Giving
The paper presents a highly effective white-box probe architecture and dataset for detecting LLM deception, outperforming black-box monitors in agentic and introspective settings. It provides a rigorous, scalable solution to the critical problem of monitoring frontier AI agents for misaligned behavior, with significant implications for deployment safety.
The paper introduces a rigorous framework for white-box deception detection, distinguishing between "contextual falsehoods" (detectable from context) and "introspective falsehoods" (requiring knowledge of model beliefs). The core methodological contribution is the "FIBS" dataset (163k transcripts) and a novel probe architecture that applies factored causal attention over the layer-token grid, allowing for aggregation of information across many layers and tokens. This is a significant step up from standard linear probes or single-layer MLPs. The use of "contrast scores" (prefilling "Yes" vs "No" and measuring the difference) is a clever practical technique to mitigate context-specific biases in probe outputs.
The experimental setup is robust and comprehensive. The authors evaluate on SHADE-Arena, a standard benchmark for agentic sabotage, showing that their probes outperform a strong black-box baseline (Opus 5.5) with much lower computational cost. Crucially, they test generalization to "introspective" settings, including model organisms with hidden goals and politically sensitive topics where black-box monitors are structurally blind. The results showing high AUC (up to 99.7%) in distinguishing true hidden goals from false denials are particularly strong, addressing a key gap in prior work (e.g., Cooney et al. 2026).
The paper commits to releasing the FIBS dataset and trained probes. Detailed hyperparameters and implementation details are provided in the appendices. The use of standard open-weight models (Qwen, Nemotron) and established benchmarks (SHADE-Arena, MMLU) enhances reproducibility. The label noise estimation (1.4%) via LLM-as-a-judge is a transparent approach to data quality.
The primary limitation is the reliance on the model's internal representation of "belief." If a model's persona shifts such that it genuinely "believes" its deceptive output (due to training dynamics), the probe may fail. The paper acknowledges this with the "math-sandbagger" organism case. Additionally, the dataset is heavily skewed towards off-policy data, and while they test on-policy generalization, the gap between synthetic training data and real-world deployment nuances remains a concern.
This work has high impact on AI safety monitoring. By demonstrating that cheap, white-box probes can match or exceed expensive black-box monitors, it lowers the barrier to entry for effective safety monitoring in frontier deployments. The release of the FIBS dataset will likely become a standard resource for training deception detectors. The findings support the viability of interpretability-based safety measures over purely behavioral monitoring. The paper presents a highly effective white-box probe architecture and dataset for detecting LLM deception, outperforming black-box monitors in agentic and introspective settings. It provides a rigorous, scalable solution to the critical problem of monitoring frontier AI agents for misaligned behavior, with significant implications for deployment safety.
Computer-use agents (CUAs) are rapidly improving, yet synthesizing diverse and reliable software environments at scale remains challenging. Native application environments require substantial construction and integration effort, which limits application coverage and increases the risk of application-specific overfitting. Browser-hosted environments provide limited interaction depth and workflow complexity. These limitations motivate SoftwareGym, an automated pipeline for synthesizing GUI software environments with diverse interfaces, executable functionality, and runtime-verified behavior. It generates structurally diverse designs, realizes them as runnable applications, and uses GUI evaluation with iterative repair to ensure functional reliability. Using SoftwareGym, we synthesize 182 applications across twelve categories and achieve a mean GUI task pass rate of 91.5%. We construct SoftwareGym-Bench, a 120-task evaluation suite spanning the same twelve categories, and use trajectories collected from these environments for CUA training. The resulting model improves Pass@3 from 41.39% to 56.97% on the strictly out-of-domain OSWorld benchmark and achieves 40.0% on SoftwareGym-Bench. These results demonstrate the potential of synthesized software environments as scalable and transferable supervision for CUA training.
Primary: Shanghai Jiaotong University
All Institutions: Shanghai Jiaotong University, Zhejiang University
The paper introduces a scalable, automated pipeline for synthesizing diverse and functionally reliable GUI software environments, demonstrating that training on these synthesized trajectories significantly improves the out-of-domain generalization of compact computer-use agents.
The paper proposes SoftwareGym, an automated multi-agent pipeline for synthesizing GUI software environments. The methodology is structured into three stages: Design (generating diverse functional specs and UI blueprints using layout-archetype distance and tree-edit distance to ensure structural diversity), Implementation (coding Avalonia applications with source-level verification), and Evaluation (GUI-based testing with iterative repair). The use of a "runtime-verified" approach where agents interact with the generated software to verify functionality before it enters the training pool is a strong technical contribution. The diversity metrics (Hamming distance for layout, Tree-Edit Distance for structure) are well-defined and effectively address the common issue of LLM-generated UIs converging to similar patterns.
The experiments are extensive. The authors synthesize 182 applications across 12 categories. They demonstrate that training a 9B model (Qwen3.5-9B) solely on these synthesized trajectories improves Pass@3 on the out-of-domain OSWorld benchmark from 41.39% to 56.97%. This is a significant improvement for a compact model, outperforming larger specialized agents like EvoCUA-32B and OpenCUA-32B. The ablation studies convincingly show the contribution of structured style design and GUI-guided repair. The creation of SoftwareGym-Bench (120 tasks) provides a dedicated evaluation suite, though its size is relatively small compared to OSWorld.
The paper provides code and model links. The pipeline relies on specific LLMs (GPT-5.5, Seed 2.1 Pro) for synthesis and evaluation, which may limit reproducibility for groups without access to these specific frontier models or APIs. However, the detailed description of the verification metrics and the release of the 182 applications significantly enhance reproducibility of the dataset and benchmark.
The synthesized applications are based on Avalonia (a cross-platform .NET UI framework), which may not fully represent the diversity of native desktop environments (e.g., Windows WPF, macOS SwiftUI, Linux GTK). The 40.0% score on SoftwareGym-Bench, while competitive, is lower than the top API-based agents (GPT-5.5 at 73.3%), suggesting that while the training data is effective for transfer, the 9B model still lags behind frontier API models on complex tasks. The cost of synthesis ($70.76 per app) is non-trivial, though lower than manual development.
This work addresses a critical bottleneck in Computer-Use Agent (CUA) research: the scarcity of diverse, executable software environments for training. By demonstrating that synthesized, verified environments can provide transferable supervision, this paper opens a path toward scaling CUA training without relying solely on manual data collection or limited browser-based environments. It is likely to be widely adopted as a data source or a methodological template for environment synthesis in the agent community. The paper introduces a scalable, automated pipeline for synthesizing diverse and functionally reliable GUI software environments, demonstrating that training on these synthesized trajectories significantly improves the out-of-domain generalization of compact computer-use agents.
Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning. We hypothesize that these failures reflect a shared deficit in visual transition reasoning, and test whether this capability can serve as a shared training primitive, one that different models can learn from different supervision sources and reuse across different tasks, with a systematic training recipe. Existing benchmarks document these deficits separately but do not support controlled comparisons across scenes, actions, and reasoning operations. We therefore introduce WOVEN, a training source and benchmark for visual transition reasoning that organizes transition supervision by scene, action, and reasoning type, using diverse, realistic rollouts from video-pretrained generative models: 36,076 examples across 20 scene types, 5 action types, and 8 reasoning types. We first evaluate 38 frontier MLLMs (e.g., GPT-5.4 and Qwen3-VL-235B-A22B) and find a substantial and systematic deficit: even the strongest models fall far below humans, and the failures recur across model families and persist with scale. We then train MLLMs at multiple scales on WOVEN and find that they learn a shared capability that transfers broadly: training subsets of only about 2,000 items each collectively improve 22 of 26 external benchmarks by up to 27.3 percentage points, and WOVEN data can replace 30-50% of a task's own training data with comparable accuracy. Controlled comparisons further yield a training recipe for visual world modeling, validated prospectively on held-out benchmarks: select supervision by the reasoning operation it teaches rather than by the actions, scenes, or domains it shows, and prefer larger changes to the visual state for robustness. Our work establishes visual transition reasoning as a reusable foundation for systematic visual world-model training in MLLMs.
Primary: Northwestern University
All Institutions: Northwestern University, Carnegie Mellon University, UNC Chapel Hill
The paper establishes visual transition reasoning as a shared, trainable primitive for multimodal LLMs and provides a systematic training recipe validated by a new benchmark, WOVEN. By demonstrating that small, targeted subsets of transition supervision can broadly improve performance across diverse spatial, physical, and temporal benchmarks, the work offers a data-efficient and theoretically grounded approach to enhancing visual world modeling capabilities in current MLLMs.
The paper proposes a novel framework for "visual transition reasoning," formalizing it as inference over $(s, a, s')$ triplets. The core methodological contribution is the WOVEN dataset, which uses video-pretrained generative models (Wan2.2) to create controlled rollouts with known actions and endpoints. This allows for the construction of multiple-choice questions with error-typed distractors, enabling fine-grained analysis of model failures. The training methodology involves supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO) on controlled subsets of the data. The systematic ablation study comparing supervision by reasoning operation versus action type is a strong methodological choice that isolates the source of transfer.
The experimental evaluation is extensive and rigorous. The authors evaluate 38 frontier MLLMs to establish a baseline deficit. They then demonstrate that training on small subsets (~2,000 items) of WOVEN significantly improves performance on 22 of 26 external benchmarks. A key strength is the causal intervention experiment, where replacing 30-50% of task-specific training data with WOVEN data yields comparable accuracy, proving the utility of the dataset as a data-efficient substitute. The correlation analysis ($r=0.86$) between WOVEN accuracy gains and external benchmark gains provides strong evidence for the "shared primitive" hypothesis.
The paper provides high reproducibility. Code and datasets are released on GitHub and HuggingFace. The generation process using Wan2.2 is specified, and the training recipes (SFT + GRPO) are detailed. The use of standard benchmarks for evaluation ensures that results can be verified against existing baselines.
The reliance on synthetic video generation (Wan2.2) may introduce domain gaps compared to real-world physics, although the authors argue for realism. The evaluation is primarily on multiple-choice tasks, which may not fully capture open-ended reasoning capabilities. The "frontier" models evaluated include some very recent releases (GPT-5.4, Qwen3-VL), which may limit immediate reproducibility for labs without access to these specific API versions or weights.
This paper has significant potential to influence how multimodal models are trained for embodied AI and physical reasoning. By identifying visual transition reasoning as a reusable primitive, it offers a data-efficient path to improving spatial and temporal understanding. The systematic recipe for selecting supervision (by reasoning operation rather than action) provides actionable guidelines for practitioners building world models. The paper establishes visual transition reasoning as a shared, trainable primitive for multimodal LLMs and provides a systematic training recipe validated by a new benchmark, WOVEN. By demonstrating that small, targeted subsets of transition supervision can broadly improve performance across diverse spatial, physical, and temporal benchmarks, the work offers a data-efficient and theoretically grounded approach to enhancing visual world modeling capabilities in current MLLMs.
Generative Recommenders increasingly use Reinforcement Learning (RL) to optimize outcomes that may appear hundreds or thousands of interactions after a decision. Standard RL propagates reward according to temporal proximity rather than explicit relevance, attenuating the signal assigned to distant but important actions and creating a source of sample inefficiency. We introduce Explanatory World Models (EWM), an attribution framework that uses explanations to identify the earlier events that plausibly contributed to a realized outcome. We instantiate EWM with a frontier language model and use its explanations as learning signals using a conservative hierarchical operator that recursively redistributes reward while preserving the undiscounted episode return. To investigate what constitutes a useful explanation, we operationalize three complementary variants inspired by Halpern's actual causation, Lombrozo's explanatory simplicity, and Deutsch's hard-to-vary explanations. We evaluate EWM on two controlled long-horizon RL benchmarks, a public sequential-recommendation benchmark, and an industry-scale generative recommender. On Crafter and WordCraft, EWM substantially outperforms PPO and LaRe, with its largest gains when rewards are most delayed. On KuaiRand, EWM outperforms SFT and LaRe offline and A2C online. Finally, in an offline evaluation of an industry generative recommender, EWM considerably improves Recall and increases behavioral proxies. These results suggest that explanations can provide structured and reusable credit for learning from delayed outcomes.
Primary: University of Cambridge
All Institutions: University of Cambridge
The paper proposes Explanatory World Models (EWM) to improve credit assignment in RL by using LLM-generated causal explanations to redistribute delayed rewards. This approach significantly outperforms standard RL baselines in long-horizon tasks and industrial recommendation settings, demonstrating the potential of integrating causal reasoning with reinforcement learning to overcome sample inefficiency.
The paper introduces Explanatory World Models (EWM), a framework that leverages Large Language Models (LLMs) to generate causal explanations for delayed rewards in Reinforcement Learning (RL). The core innovation is the use of these natural language explanations to identify non-temporal dependencies between actions and outcomes, thereby improving credit assignment. The method employs a "conservative hierarchical operator" to redistribute reward based on the LLM's identified causal links while preserving the undiscounted episode return. The theoretical grounding draws from Halpern's actual causation, Lombrozo's explanatory simplicity, and Deutsch's hard-to-vary explanations, providing a robust conceptual foundation for what constitutes a "useful" explanation in this context. The integration of LLMs as a credit assignment mechanism rather than just a policy or value estimator is a significant methodological shift.
The evaluation is comprehensive, spanning controlled long-horizon benchmarks (Crafter, WordCraft), a public sequential recommendation dataset (KuaiRand), and an industry-scale generative recommender. The results show substantial outperformance of PPO and LaRe, particularly in scenarios with high reward delay, which is the primary pain point the paper addresses. The inclusion of an industry-scale offline evaluation adds significant practical relevance, demonstrating that the method scales beyond toy problems. The gains in Recall and behavioral proxies on the industrial dataset are strong indicators of real-world utility.
The paper provides detailed descriptions of the LLM variants and the reward redistribution operator. However, the specific prompts used to elicit explanations from the "frontier language model" and the exact hyperparameters for the conservative operator are critical for reproducibility. While the appendix is referenced, the main text suggests a level of detail that should allow for replication if the code is released (which is not provided in the text snippet). The use of a "frontier" model implies potential dependency on proprietary APIs, which could hinder open reproduction.
The reliance on LLMs introduces inference costs and latency, which may be prohibitive for real-time online RL in some settings. The method's effectiveness depends heavily on the quality of the LLM's causal reasoning, which can be inconsistent. Furthermore, the "conservative" nature of the reward redistribution might limit the magnitude of improvement compared to more aggressive credit assignment strategies. The paper does not extensively discuss the failure modes of the LLM explanations or how the system handles incorrect causal attributions.
This work has significant implications for the field of RL, particularly for applications involving sparse or delayed rewards such as robotics, long-horizon planning, and recommendation systems. By bridging the gap between symbolic causal reasoning (via LLMs) and statistical learning (RL), it offers a pathway to more sample-efficient agents. The findings could inspire further research into hybrid neuro-symbolic approaches for credit assignment and the use of generative models as world models for causal inference. The paper proposes Explanatory World Models (EWM) to improve credit assignment in RL by using LLM-generated causal explanations to redistribute delayed rewards. This approach significantly outperforms standard RL baselines in long-horizon tasks and industrial recommendation settings, demonstrating the potential of integrating causal reasoning with reinforcement learning to overcome sample inefficiency.
Reinforcement learning (RL) has emerged as a key paradigm for improving large language model (LLM) agents, enabling stronger reasoning, planning, and tool-use in interactive environments. However, existing agent RL methods remain fundamentally constrained by an on-policy reasoning bottleneck: when tasks exceed the model's current capability boundary, effective trajectories become extremely sparse and training often collapses. We introduce Rollout-Time Intervention (RTI), a technique designed to elicit successful trajectories beyond the policy's intrinsic exploration capacity. RTI operates by leveraging reward-side evidence (i.e., evaluation rubrics) during rollout to convert erroneous trajectories into successful ones, providing effective learning signals for tasks that the policy rarely solves on its own. Although intervention trajectories are inherently off-policy, we find that they can effectively guide policy optimization when trajectories with excessive distributional shifts are filtered out. Extensive experiments across five benchmarks demonstrate that RTI consistently outperforms RL baselines, enabling the post-trained policy to approach the performance of substantially stronger contemporary LLMs and, more importantly, transcend its original capability boundary. The code is available at https://github.com/Alibaba-NLP/qqr.
Primary: Alibaba Group
All Institutions: Alibaba Group, Alibaba Group Amap, Alibaba Group HKUST (Guangzhou)
The paper introduces Rollout-Time Intervention (RTI), a technique that leverages reward-side evidence to correct failed agent trajectories during rollout, enabling LLMs to learn beyond their intrinsic exploration limits. By combining a multi-agent correction pipeline with distribution gating and token reweighting, RTI effectively mitigates the on-policy bottleneck in reinforcement learning, demonstrating substantial improvements in complex agentic tasks and providing a robust framework for training agents on tasks where successful trajectories are initially sparse.
The paper proposes Rollout-Time Intervention (RTI), a method to address the "on-policy bottleneck" in RL training for LLM agents. The core idea is to use a multi-agent system (Validator-Corrector-Verifier) to repair failed trajectories during the rollout phase, effectively creating successful off-policy examples. The methodology is technically sound, addressing key challenges of off-policy learning: (1) preventing reward hacking/leakage via the Verifier, (2) managing distribution shift via perplexity-based gating, and (3) improving credit assignment via reasoning-shift-aware token reweighting. The use of self-intervention (using the policy model itself for correction) is a clever and resource-efficient design choice, though it relies on the model having sufficient latent capability to correct its own errors when guided by rubrics.
The evaluation is extensive, covering five benchmarks including a novel Navigation benchmark and the challenging Claw-Eval-Hard20. The results show consistent improvements over strong baselines like GRPO and teacher-guided methods (LUFFY, OPD). The ablation studies are rigorous, clearly isolating the contribution of each component (Post-Check, PPL Gating, Reweighting). The analysis of training dynamics (showing that the policy eventually succeeds without intervention) is particularly convincing evidence that the method teaches the model rather than just hacking the reward.
The authors provide a GitHub link and state that comprehensive details, including prompts and hyperparameters, are in the appendix. The use of standard benchmarks (tau2-bench) and open-source backbones (Qwen3) enhances reproducibility. However, the specific implementation of the MAS prompts and the "Claw-Eval" benchmark details might require careful reading of the appendix to fully replicate.
The method relies heavily on the availability of "reward-side evidence" or evaluation rubrics, which may not be available for all agentic tasks. The perplexity gating threshold requires tuning. Additionally, the method is primarily tested on reasoning and tool-use tasks; its applicability to other types of agent interactions (e.g., physical robotics) is not explored. The reliance on self-intervention means that if the model lacks the fundamental knowledge to correct the error, the intervention may fail, though the paper argues this is less of an issue than with external teachers.
This work has significant implications for the training of LLM agents, offering a practical way to overcome the sparsity of rewards in complex environments. By enabling models to learn from corrected failures, it could accelerate the development of more capable autonomous agents. The technique is modular and could potentially be integrated into other RL frameworks beyond GRPO. The paper introduces Rollout-Time Intervention (RTI), a technique that leverages reward-side evidence to correct failed agent trajectories during rollout, enabling LLMs to learn beyond their intrinsic exploration limits. By combining a multi-agent correction pipeline with distribution gating and token reweighting, RTI effectively mitigates the on-policy bottleneck in reinforcement learning, demonstrating substantial improvements in complex agentic tasks and providing a robust framework for training agents on tasks where successful trajectories are initially sparse.
In-context-learning Tabular Foundation Models (TFMs) are uniquely suited to scenarios with limited resources or privacy constraints, as they solve new tasks without additional training. However, their architecture consists of multiple transformer blocks, resulting in high computational and memory footprints. Meanwhile, traditional alternatives like Gradient-Boosted Decision Trees (GBDTs) remain highly competitive and run efficiently on CPUs, suggesting that tabular tasks may not inherently require deep architectures. This contrast raises a fundamental question: can TFMs be simplified by removing architectural redundancies without sacrificing performance? To answer this, we introduce a closed-form, label-free affine translator that replaces a contiguous window of up to 96% of the blocks, leaving the rest unchanged. Evaluating eight frontier TFMs on 171 classification datasets, we reveal that tabular transformers are vastly overparameterized: replacing half of a model's blocks costs at most $0.002$ median AUROC across all architectures, with the half-depth TabFM still outperforming every other full-depth TFMs and tuned GBDTs. Furthermore, different model families are overparameterized in different ways: TabPFN-3 relies on its encoders, maintaining performance regardless of which single block is retained. Other single-axis models spread computation thinly, needing only their first block, while dual-axis models concentrate it in one critical block. Ultimately, removing depth saves up to 1.5B parameters, speeds up GPU inference by up to $6\times$ at a median cost of $0.03$ AUROC, while a single-block TabPFN-3 serves cached queries $2.4\times$ faster using $6\times$ less memory at negligible cost.
Primary: University of Oxford
All Institutions: University of Oxford
The paper demonstrates that Tabular Foundation Models are effectively shallow by showing that up to 96% of their transformer blocks can be replaced with a simple, label-free affine map with negligible performance loss, revealing significant architectural redundancy and enabling substantial efficiency gains.
The paper introduces a "closed-form, label-free affine translator" to replace contiguous windows of transformer blocks in Tabular Foundation Models (TFMs). The method fits a linear map between the hidden states of the start and end of the bypassed window using ridge regression on an unlabeled calibration set. The window selection is guided by minimizing the forward KL divergence between the full model's output distribution and the compressed model's output distribution. The methodology is clever in that it requires no gradient updates, no labeled data, and preserves the original classification head, allowing for a direct comparison of output probabilities. The per-feature parameterization for dual-axis models is a necessary and well-motivated extension.
The evaluation is extensive, covering 8 frontier TFMs from 6 families across 171 classification datasets (including clinical and standard benchmarks). The results show that replacing 50% of the blocks costs negligible performance (median AUROC drop <= 0.002), and even reducing to a single block retains competitive performance for many models. The ablation study comparing the affine translator to naive layer dropping (identity map) is crucial, showing that the blocks are not inactive but their computation is linearly absorbable. The comparison with GBDTs and the efficiency analysis (speedup/memory reduction) further solidify the practical relevance.
The paper provides a GitHub link to the code. The experimental setup is detailed, including dataset splits, hyperparameter tuning for GBDTs, and specific model configurations. The use of standard benchmarks and clear metrics (AUROC, R^2, latency, memory) ensures reproducibility.
The study is limited to classification and regression tasks; generative tasks are not covered. The pre-training study is limited to a small "nanoTabPFN" model, so it is unclear if the findings scale to much larger architectures. The per-feature translator for dual-axis models can have a high parameter count on wide datasets, potentially negating some efficiency gains.
This paper has significant implications for the deployment of foundation models in resource-constrained environments (e.g., edge devices, clinical servers). It challenges the assumption that deep architectures are necessary for tabular data, suggesting that current TFMs are overparameterized. This could lead to more efficient model design and reduced computational costs for inference. The paper demonstrates that Tabular Foundation Models are effectively shallow by showing that up to 96% of their transformer blocks can be replaced with a simple, label-free affine map with negligible performance loss, revealing significant architectural redundancy and enabling substantial efficiency gains.
A solid understanding of the predictive success of deep neural networks (DNNs) remains elusive. For example, while many data-dependent metrics exist that seek to predict model quality of DNNs, these metrics may be expensive to compute for large datasets and/or may be impossible to compute when training datasets are not released. A practical and effective predictor of model quality involves analyzing DNN weight matrices, as prior work has established that heavy-tailed spectral distributions in trained weight matrices can be strongly predictive of model quality. However, such metrics lack a rigorous statistical justification. In this work, we aim to understand why these metrics perform well, from a statistical perspective. To do so, we employ a probabilistic framework to derive the marginal likelihood for a DNN, evaluated using only the final layer weights. Across large-scale convolutional DNNs and LLMs, we find that this marginal likelihood predicts model quality. To understand the role heavy-tailed spectra play in predictive performance, we derive the limiting log-marginal likelihood for spectra that follow the (heavy-tailed) High-Temperature Marchenko-Pastur (HTMP) distribution. We show that this limiting quantity is convex in a heaviness parameter, and we derive an optimal heaviness that empirically predicts over-fitting for large-scale DNNs.
Primary: University of Melbourne
All Institutions: University of Melbourne, University of California, University of Queensland, ICSI, LBNL, CIRES
The paper provides a rigorous statistical justification for the predictive power of heavy-tailed weight spectra by deriving a data-free Bayes Free Energy metric and analyzing its limiting behavior under the High-Temperature Marchenko-Pastur distribution. It successfully connects Random Matrix Theory to practical model quality prediction, offering new insights into the distinct learning dynamics of computer vision and language models.
The paper proposes a probabilistic framework to derive the marginal likelihood (Bayes Free Energy) of a Deep Neural Network (DNN) using only the final layer weights. By assuming an entropic regularization framework and an interpolating regime, the authors derive a data-free metric that correlates with model quality. They then employ Random Matrix Theory (RMT), specifically the High-Temperature Marchenko-Pastur (HTMP) distribution, to analyze the limiting behavior of this free energy. The core theoretical contribution is showing that the limiting free energy is convex in a "heaviness" parameter and identifying an optimal ratio of spectral parameters that predicts overfitting. This provides a rigorous statistical justification for the empirical observation that heavy-tailed weight spectra are predictive of model quality.
The experiments are extensive, covering both Computer Vision (ResNet, DenseNet on ImageNet/CIFAR) and Large Language Models (Pythia, Qwen3.5). The authors demonstrate strong Spearman correlations between the derived free energy metric and test accuracy/error. They further validate the theoretical predictions by tracking the HTMP parameters during training, showing that CV models optimize the heaviness ratio while LLMs optimize the scale parameter. The inclusion of both CV and LLMs strengthens the generalizability of the findings, although the failure of the metric on Vision Transformers and dropout models is a notable gap.
The paper provides a GitHub repository link for code and experimental details. The theoretical derivations are detailed in the appendices, including proofs for the limiting free energy and convexity. The use of standard model zoos (torchvision, Pythia) ensures that the baseline models are accessible, making the experimental results highly reproducible.
The primary limitation is that the metric is derived for the final layer weights and regression tasks, and does not generalize well to Vision Transformers or models with dropout before the final layer. Additionally, the prior precision parameter $\lambda$ often needs to be tuned post-hoc for CV models, which limits the "data-free" and "a priori" nature of the metric in practical applications. The theoretical assumptions (interpolation, specific spectral convergence) may not hold for all modern architectures.
This work bridges the gap between phenomenological theories (HT-SR) and rigorous statistical learning theory. By providing a data-free metric for model quality, it offers a practical tool for evaluating models without access to training data, which is increasingly important for foundation models. The insights into the distinct training dynamics of CV models vs. LLMs (optimizing heaviness vs. scale) could inform future training strategies and regularization techniques. The paper provides a rigorous statistical justification for the predictive power of heavy-tailed weight spectra by deriving a data-free Bayes Free Energy metric and analyzing its limiting behavior under the High-Temperature Marchenko-Pastur distribution. It successfully connects Random Matrix Theory to practical model quality prediction, offering new insights into the distinct learning dynamics of computer vision and language models.
Recent incidents have highlighted the challenge of monitoring LLM agents and the danger of models deceiving people. We show that white-box deception detection via probes can be scaled up to frontier monitoring settings by collecting the largest deception dataset to date for training probes and introducing a novel probe architecture which can aggregate information across many layers and tokens. Our probes achieve 98.8% AUC in SHADE-Arena, surpassing an Opus 5.5 text-monitoring baseline, and show improved efficacy as the underlying model is scaled up. To push our probes to their limit, we test them on several cases where deception cannot be determined from the context alone. In these cases, which we refer to as introspective deception, the ground truth can only be determined through careful elicitation or thorough knowledge of a model's training data. In one such evaluation, we show that probes can distinguish transcripts containing a model's true hidden goal from other goals with an AUC of up to 99.7%. Our probes also readily detect deception on prominent open-weight models which lie about politically sensitive topics, and about their beliefs when put under pressure. We release our training dataset, dubbed FIBS, to help drive frontier deployment of effective probes, and encourage the community to expand upon it with further examples of deception and sabotage.
Primary: FAR.AI
All Institutions: FAR.AI, Coefficient Giving
The paper presents a highly effective white-box probe architecture and dataset for detecting LLM deception, outperforming black-box monitors in agentic and introspective settings. It provides a rigorous, scalable solution to the critical problem of monitoring frontier AI agents for misaligned behavior, with significant implications for deployment safety.
The paper introduces a rigorous framework for white-box deception detection, distinguishing between "contextual falsehoods" (detectable from context) and "introspective falsehoods" (requiring knowledge of model beliefs). The core methodological contribution is the "FIBS" dataset (163k transcripts) and a novel probe architecture that applies factored causal attention over the layer-token grid, allowing for aggregation of information across many layers and tokens. This is a significant step up from standard linear probes or single-layer MLPs. The use of "contrast scores" (prefilling "Yes" vs "No" and measuring the difference) is a clever practical technique to mitigate context-specific biases in probe outputs.
The experimental setup is robust and comprehensive. The authors evaluate on SHADE-Arena, a standard benchmark for agentic sabotage, showing that their probes outperform a strong black-box baseline (Opus 5.5) with much lower computational cost. Crucially, they test generalization to "introspective" settings, including model organisms with hidden goals and politically sensitive topics where black-box monitors are structurally blind. The results showing high AUC (up to 99.7%) in distinguishing true hidden goals from false denials are particularly strong, addressing a key gap in prior work (e.g., Cooney et al. 2026).
The paper commits to releasing the FIBS dataset and trained probes. Detailed hyperparameters and implementation details are provided in the appendices. The use of standard open-weight models (Qwen, Nemotron) and established benchmarks (SHADE-Arena, MMLU) enhances reproducibility. The label noise estimation (1.4%) via LLM-as-a-judge is a transparent approach to data quality.
The primary limitation is the reliance on the model's internal representation of "belief." If a model's persona shifts such that it genuinely "believes" its deceptive output (due to training dynamics), the probe may fail. The paper acknowledges this with the "math-sandbagger" organism case. Additionally, the dataset is heavily skewed towards off-policy data, and while they test on-policy generalization, the gap between synthetic training data and real-world deployment nuances remains a concern.
This work has high impact on AI safety monitoring. By demonstrating that cheap, white-box probes can match or exceed expensive black-box monitors, it lowers the barrier to entry for effective safety monitoring in frontier deployments. The release of the FIBS dataset will likely become a standard resource for training deception detectors. The findings support the viability of interpretability-based safety measures over purely behavioral monitoring. The paper presents a highly effective white-box probe architecture and dataset for detecting LLM deception, outperforming black-box monitors in agentic and introspective settings. It provides a rigorous, scalable solution to the critical problem of monitoring frontier AI agents for misaligned behavior, with significant implications for deployment safety.
Computer-use agents (CUAs) are rapidly improving, yet synthesizing diverse and reliable software environments at scale remains challenging. Native application environments require substantial construction and integration effort, which limits application coverage and increases the risk of application-specific overfitting. Browser-hosted environments provide limited interaction depth and workflow complexity. These limitations motivate SoftwareGym, an automated pipeline for synthesizing GUI software environments with diverse interfaces, executable functionality, and runtime-verified behavior. It generates structurally diverse designs, realizes them as runnable applications, and uses GUI evaluation with iterative repair to ensure functional reliability. Using SoftwareGym, we synthesize 182 applications across twelve categories and achieve a mean GUI task pass rate of 91.5%. We construct SoftwareGym-Bench, a 120-task evaluation suite spanning the same twelve categories, and use trajectories collected from these environments for CUA training. The resulting model improves Pass@3 from 41.39% to 56.97% on the strictly out-of-domain OSWorld benchmark and achieves 40.0% on SoftwareGym-Bench. These results demonstrate the potential of synthesized software environments as scalable and transferable supervision for CUA training.
Primary: Shanghai Jiaotong University
All Institutions: Shanghai Jiaotong University, Zhejiang University
The paper introduces a scalable, automated pipeline for synthesizing diverse and functionally reliable GUI software environments, demonstrating that training on these synthesized trajectories significantly improves the out-of-domain generalization of compact computer-use agents.
The paper proposes SoftwareGym, an automated multi-agent pipeline for synthesizing GUI software environments. The methodology is structured into three stages: Design (generating diverse functional specs and UI blueprints using layout-archetype distance and tree-edit distance to ensure structural diversity), Implementation (coding Avalonia applications with source-level verification), and Evaluation (GUI-based testing with iterative repair). The use of a "runtime-verified" approach where agents interact with the generated software to verify functionality before it enters the training pool is a strong technical contribution. The diversity metrics (Hamming distance for layout, Tree-Edit Distance for structure) are well-defined and effectively address the common issue of LLM-generated UIs converging to similar patterns.
The experiments are extensive. The authors synthesize 182 applications across 12 categories. They demonstrate that training a 9B model (Qwen3.5-9B) solely on these synthesized trajectories improves Pass@3 on the out-of-domain OSWorld benchmark from 41.39% to 56.97%. This is a significant improvement for a compact model, outperforming larger specialized agents like EvoCUA-32B and OpenCUA-32B. The ablation studies convincingly show the contribution of structured style design and GUI-guided repair. The creation of SoftwareGym-Bench (120 tasks) provides a dedicated evaluation suite, though its size is relatively small compared to OSWorld.
The paper provides code and model links. The pipeline relies on specific LLMs (GPT-5.5, Seed 2.1 Pro) for synthesis and evaluation, which may limit reproducibility for groups without access to these specific frontier models or APIs. However, the detailed description of the verification metrics and the release of the 182 applications significantly enhance reproducibility of the dataset and benchmark.
The synthesized applications are based on Avalonia (a cross-platform .NET UI framework), which may not fully represent the diversity of native desktop environments (e.g., Windows WPF, macOS SwiftUI, Linux GTK). The 40.0% score on SoftwareGym-Bench, while competitive, is lower than the top API-based agents (GPT-5.5 at 73.3%), suggesting that while the training data is effective for transfer, the 9B model still lags behind frontier API models on complex tasks. The cost of synthesis ($70.76 per app) is non-trivial, though lower than manual development.
This work addresses a critical bottleneck in Computer-Use Agent (CUA) research: the scarcity of diverse, executable software environments for training. By demonstrating that synthesized, verified environments can provide transferable supervision, this paper opens a path toward scaling CUA training without relying solely on manual data collection or limited browser-based environments. It is likely to be widely adopted as a data source or a methodological template for environment synthesis in the agent community. The paper introduces a scalable, automated pipeline for synthesizing diverse and functionally reliable GUI software environments, demonstrating that training on these synthesized trajectories significantly improves the out-of-domain generalization of compact computer-use agents.
Unified atomistic modeling has the potential to accelerate discovery in chemistry, materials science, and biology by bridging data-rich chemical domains and data-scarce biological contexts. However, existing generative approaches to atomistic modeling remain highly specialized to scientific disciplines (chemistry vs. biology) or do not leverage both high-volume organic (molecule) and inorganic (material) data for general-purpose pretraining. To this end, we introduce Zatom-2, an atomistic generative model pretrained on approximately five million structures from the OMol25 and OMat24 electronic structure datasets. Zatom-2 features a multiscale Transformer architecture coupled with conditional flow matching that supports force conditioning and foundational pretraining tasks such as generation, structure prediction, and prediction of molecular and material energies and forces. Empirically, Zatom-2 achieves better molecular distribution fidelity than Zatom-1 and achieves strong performance on existing molecule and material generation benchmarks. Zatom-2 demonstrates the ability to control sample generation across low- and high-force regimes, and enhances protein generation in a low-data setting through joint generative-predictive pretraining and transfer learning, increasing protein backbone designability in a length extrapolation setting from 67.8% without pretraining to 74.8% after finetuning on 2,000 protein domains.
Primary: University of California, Berkeley
All Institutions: University of California, Berkeley, Lawrence Berkeley National Laboratory, University of Cambridge, University of Oxford, University of Washington, University of Oxford (Inferred from authors: Mahoney, Erichson, Jacobson, Blau, Perez, Anand, Abrudan, Panescu, Li, Knowles, Morehead, Cretu)
Zatom-2 introduces a unified atomistic generative model that leverages multitask pretraining on large-scale molecular and material datasets to enable cross-domain transfer to protein generation. By combining a novel atom1 tokenization scheme with conditional flow matching and physics-grounded auxiliary objectives (energy/force prediction), the model demonstrates superior distribution fidelity and, crucially, the ability to transfer geometric representations from small molecules to larger biomolecules, achieving significant gains in protein designability in low-data and out-of-distribution settings.
The paper introduces Zatom-2, a unified atomistic generative model that bridges the gap between small molecule/material generation and protein structure prediction. The core methodological contribution is the "atom1" tokenization scheme for molecules/materials (analogous to "atom14" for proteins), which allows a single Transformer backbone to handle variable atom counts without separate diffusion tracks for atom identity. The model employs conditional flow matching for coordinate generation and incorporates a novel "force conditioning" mechanism that uses the mean force norm as a global descriptor to control the relaxation state of generated structures. A key innovation is the multitask pretraining curriculum that jointly optimizes for generation, structure prediction, and energy/force prediction (MLIP objectives). The authors hypothesize and demonstrate that the physics-grounded supervision from energy/force prediction shapes the latent representations in a way that improves generative fidelity and transferability, particularly in low-data regimes.
The experimental evaluation is extensive, covering pretraining on ~5 million structures from OMol25 and OMat24. The paper demonstrates that Zatom-2 outperforms its predecessor (Zatom-1) in distribution fidelity and achieves competitive results on standard benchmarks like QM9, MP20, and GEOM-Drugs. A significant portion of the evaluation focuses on the "force consistency" metric, showing the model can reliably distinguish between high-force (rattled) and low-force (relaxed) regimes. The most compelling results are in the transfer learning section, where Zatom-2 pretrained on small molecules/materials is fine-tuned on a small protein dataset (SCOPe-2k). The model shows significant improvements in designability and novelty, especially in out-of-distribution (longer sequence) settings, compared to training from scratch or using protein-specific models like RFdiffusion3. The scaling study confirms that both data coverage and model capacity yield complementary gains.
The paper provides a strong reproducibility statement, including a link to the open-source code repository (https://github.com/Zatom-AI/nucleus) which contains documentation, data loading, and training/inference materials. The appendix details the model architecture, hyperparameters, and compute resources used (NVIDIA A100 GPUs). The authors also provide an AI Use Statement, disclosing the use of LLMs for code and writing assistance, which adds to the transparency of the research process.
The model does not enforce exact rotational equivariance, relying instead on data augmentation and pair-biased attention, which may limit its performance in tasks requiring strict symmetry preservation compared to equivariant diffusion models. The MLIP (energy/force) prediction accuracy is lower than specialized potentials, though it serves as an auxiliary task. The protein generation experiments are limited to a small dataset (2k domains) and do not yet match the performance of state-of-the-art protein-specific models in in-distribution settings, though it shows promise in extrapolation. The paper acknowledges that precise force matching within regimes remains challenging.
This work has significant potential to accelerate discovery in chemistry, materials science, and biology by providing a unified framework for atomistic modeling. The ability to transfer knowledge from data-rich chemical domains to data-scarce biological contexts could reduce the computational cost of training specialized models. The force conditioning mechanism offers a new tool for controlling the physical state of generated structures, which is valuable for simulating non-equilibrium processes. The open-sourcing of the code and the use of large public datasets (OMol25, OMat24) will facilitate further research in this area. Zatom-2 introduces a unified atomistic generative model that leverages multitask pretraining on large-scale molecular and material datasets to enable cross-domain transfer to protein generation. By combining a novel atom1 tokenization scheme with conditional flow matching and physics-grounded auxiliary objectives (energy/force prediction), the model demonstrates superior distribution fidelity and, crucially, the ability to transfer geometric representations from small molecules to larger biomolecules, achieving significant gains in protein designability in low-data and out-of-distribution settings.
Tabular data underpin prediction and decision-making in medicine, finance, government and science, but often contain sensitive individual-level information, creating a need for accurate prediction while preserving privacy. Traditional private learning provides formal privacy guarantees, but requires slow dataset-specific optimisation, suffers substantial utility loss under strong privacy, and is often difficult to apply correctly. Tabular foundation models adapt rapidly to new datasets, but existing models lack formal privacy guarantees, and are highly vulnerable to membership-inference attacks, limiting their use on sensitive data. Here we introduce PrivTab, an easy to use tabular foundation model for differentially private classification that embeds a privacy mechanism within its architecture. Pretrained on simulated datasets, PrivTab uses in-context learning to transform sensitive rows into compact, provably private summaries---effectively learning how to learn under privacy. PrivTab outperforms private linear and neural-network baselines under moderate-to-strong privacy, shows negligible membership leakage, maintains well-calibrated predictions under strong privacy, and reduces dataset fitting time by 10,000 times, requiring only a single forward pass. By combining formal privacy, speed, and easy of use, PrivTab brings recent advances in AI to applications where sensitive individual-level data have limited their adoption.
Primary: University of Helsinki
All Institutions: University of Helsinki, Aalto University, University of Oxford
PrivTab introduces a novel architectural integration of differential privacy into tabular foundation models, enabling provably private classification with negligible utility loss and orders-of-magnitude speedup over traditional private training methods. By embedding a differentially private cross-attention mechanism that compresses sensitive data into a fixed-size private summary, the model leverages the post-processing property of DP to ensure all downstream predictions are private, while noise-aware pretraining allows the model to learn how to predict effectively through the privacy noise. This approach resolves the tension between the high utility of foundation models and the rigorous guarantees of differential privacy, offering a practical, auditable, and efficient solution for sensitive tabular data applications.
The paper introduces PrivTab, a tabular foundation model that integrates differential privacy (DP) directly into the architecture via a "Differentially Private Multi-Head Cross-Attention" (DP-MHCA) layer. This is a significant architectural shift from traditional DP-SGD, which applies noise during iterative optimization. By compressing the context dataset into a fixed-size, noisy summary in a single forward pass, PrivTab leverages the post-processing property of DP to ensure that all subsequent predictions are private without additional privacy cost. The methodology is rigorous, featuring a formal proof of the privacy guarantee (verified in Lean 4), a noise-aware pretraining objective that teaches the model to predict through noisy summaries, and a clear separation between the privacy-critical path (the summary module) and the prediction path. The use of Gaussian DP (GDP) and adaptive composition is handled correctly, and the sensitivity analysis is sound.
The evaluation is comprehensive, utilizing the TabArena benchmark with 33 datasets across six privacy regimes. PrivTab demonstrates superior AUC and Log Loss compared to DP-Linear Regression and DP-MLP baselines, particularly in moderate-to-strong privacy settings where traditional methods suffer significant utility loss. The computational efficiency is a standout result, with fitting times reduced by ~10,000x (milliseconds vs. minutes). The membership inference attack analysis convincingly demonstrates that non-private tabular foundation models (TabPFN, TabICL) are highly vulnerable, whereas PrivTab maintains strong privacy. The inclusion of a private preprocessing case study adds practical relevance, showing how to handle feature scaling within the privacy budget.
High. The authors provide a public GitHub repository containing the code, Lean 4 formal verification scripts, and empirical auditing tools. The pretraining details, including the simulator used (TabICL simulator) and hyperparameters, are described in detail. The use of standard benchmarks (TabArena) and clear evaluation metrics (AUC, Log Loss, Elo) facilitates comparison with other works.
The model is currently limited to classification tasks, not regression. The privacy model assumes one user per row, which may not hold in all real-world scenarios (e.g., longitudinal data). The advantage diminishes on very large datasets or under very weak privacy constraints, where traditional DP-SGD methods can leverage more data. The reliance on public descriptions for clipping bounds in preprocessing is a practical constraint that may not always be met.
This work has the potential to significantly lower the barrier to entry for differentially private machine learning. By removing the need for dataset-specific private training and hyperparameter tuning, it makes DP accessible to domain experts who are not privacy specialists. The architectural insight—that privacy can be embedded into the foundation model's in-context learning mechanism—could inspire similar approaches in other modalities (text, image) where privacy is a concern. The formal verification aspect sets a new standard for trust in private ML implementations. PrivTab introduces a novel architectural integration of differential privacy into tabular foundation models, enabling provably private classification with negligible utility loss and orders-of-magnitude speedup over traditional private training methods. By embedding a differentially private cross-attention mechanism that compresses sensitive data into a fixed-size private summary, the model leverages the post-processing property of DP to ensure all downstream predictions are private, while noise-aware pretraining allows the model to learn how to predict effectively through the privacy noise. This approach resolves the tension between the high utility of foundation models and the rigorous guarantees of differential privacy, offering a practical, auditable, and efficient solution for sensitive tabular data applications.
Modern language models undergo reinforcement learning with verifiable rewards (RLVR) on top of already-trained checkpoints. A key promise of RLVR is the discovery of new reasoning strategies. In principle, a model can sample novel ideas absent from its prior training data. In practice, however, augmenting RLVR with strong novelty incentives has seen limited success and can degrade model quality. Because verifiable rewards supervise only a narrow slice of the model's knowledge and behavior, such degradations are difficult to recover from. Instead, we decouple exploration from optimization in a framework we call Exploration-Distillation (ExpDis). We train one or more explorer policies with a novelty bonus in the reward, filter their trajectories for correctness and quality, and distill them into a separate student policy. The student policy is then trained without a novelty bonus. We repeat the above procedure for several rounds, alternating between exploration and optimization. This decoupling allows us to aggressively scale exploration without degrading the student policy. Across seven mathematical reasoning benchmarks and two model families, ExpDis outperforms DAPO at the same wall-clock budget. Moreover, we observe improved pass@$k$ scaling, indicating that ExpDis produces models that generate more diverse correct solutions.
Primary: Columbia University
All Institutions: Columbia University
The paper introduces a decoupled exploration-distillation framework for RLVR that effectively boosts reasoning performance and diversity without degrading general capabilities. By separating the aggressive exploration policy from the stable optimization policy, it resolves the tension between novelty and correctness, offering a robust and scalable improvement over standard RLVR methods.
The paper proposes Exploration-Distillation (ExpDis), a framework that decouples exploration from optimization in Reinforcement Learning with Verifiable Rewards (RLVR). The core insight is that applying novelty bonuses directly to the policy being optimized leads to degradation of general capabilities because the verifiable reward only supervises a narrow slice of the model's knowledge. By training separate "explorer" policies with aggressive novelty incentives (using Random Network Distillation) and distilling only their correct, high-quality trajectories into a "student" policy via SFT followed by standard RLVR, the method isolates the benefits of exploration from the risks of instability. The methodology is sound, building on established concepts like intrinsic motivation and expert iteration, but the specific application to LLM reasoning with the decoupled architecture is a novel and effective contribution.
The experiments are rigorous and well-controlled. The authors compare against strong baselines (DAPO, GRPO, Dr. GRPO) and a direct novelty-bonus variant of DAPO. They demonstrate that ExpDis outperforms DAPO at the same wall-clock budget across multiple model families (Qwen3, Ministral) and benchmarks (AIME, MATH, etc.). Crucially, they show that while direct novelty bonuses degrade general knowledge benchmarks (MMLU, GPQA), ExpDis preserves or improves these capabilities while boosting reasoning performance. The analysis of pass@$k$ scaling and diversity metrics provides strong evidence that the method genuinely expands the model's solution space rather than just sharpening existing modes.
The paper provides high reproducibility. Code and checkpoints are released on GitHub and HuggingFace. Hyperparameters, including the novelty bonus weight schedule and filtering criteria, are detailed in the appendix. The use of standard datasets (DAPO-Math-17K) and public benchmarks further aids reproducibility.
The method requires training multiple models (explorers + student), which increases memory and compute overhead compared to single-model RLVR, although the paper argues this is offset by the ability to use aggressive exploration. The current evaluation is limited to mathematical reasoning; it is unclear if the benefits generalize to other domains like code or open-ended generation without similar verifiable rewards. The optimal number of explorers and rounds may be sensitive to the specific task and model scale.
This work has significant implications for the scaling of LLM reasoning. As models become proficient at known strategies, the need for discovering new strategies grows. ExpDis provides a safe mechanism to do so, potentially enabling the discovery of novel reasoning paths that standard RLVR would miss due to entropy collapse. It offers a practical path for practitioners to enhance model diversity and capability without the risk of catastrophic forgetting or quality degradation. The paper introduces a decoupled exploration-distillation framework for RLVR that effectively boosts reasoning performance and diversity without degrading general capabilities. By separating the aggressive exploration policy from the stable optimization policy, it resolves the tension between novelty and correctness, offering a robust and scalable improvement over standard RLVR methods.
Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning. We hypothesize that these failures reflect a shared deficit in visual transition reasoning, and test whether this capability can serve as a shared training primitive, one that different models can learn from different supervision sources and reuse across different tasks, with a systematic training recipe. Existing benchmarks document these deficits separately but do not support controlled comparisons across scenes, actions, and reasoning operations. We therefore introduce WOVEN, a training source and benchmark for visual transition reasoning that organizes transition supervision by scene, action, and reasoning type, using diverse, realistic rollouts from video-pretrained generative models: 36,076 examples across 20 scene types, 5 action types, and 8 reasoning types. We first evaluate 38 frontier MLLMs (e.g., GPT-5.4 and Qwen3-VL-235B-A22B) and find a substantial and systematic deficit: even the strongest models fall far below humans, and the failures recur across model families and persist with scale. We then train MLLMs at multiple scales on WOVEN and find that they learn a shared capability that transfers broadly: training subsets of only about 2,000 items each collectively improve 22 of 26 external benchmarks by up to 27.3 percentage points, and WOVEN data can replace 30-50% of a task's own training data with comparable accuracy. Controlled comparisons further yield a training recipe for visual world modeling, validated prospectively on held-out benchmarks: select supervision by the reasoning operation it teaches rather than by the actions, scenes, or domains it shows, and prefer larger changes to the visual state for robustness. Our work establishes visual transition reasoning as a reusable foundation for systematic visual world-model training in MLLMs.
Primary: Northwestern University
All Institutions: Northwestern University, Carnegie Mellon University, UNC Chapel Hill
The paper establishes visual transition reasoning as a shared, trainable primitive for multimodal LLMs and provides a systematic training recipe validated by a new benchmark, WOVEN. By demonstrating that small, targeted subsets of transition supervision can broadly improve performance across diverse spatial, physical, and temporal benchmarks, the work offers a data-efficient and theoretically grounded approach to enhancing visual world modeling capabilities in current MLLMs.
The paper proposes a novel framework for "visual transition reasoning," formalizing it as inference over $(s, a, s')$ triplets. The core methodological contribution is the WOVEN dataset, which uses video-pretrained generative models (Wan2.2) to create controlled rollouts with known actions and endpoints. This allows for the construction of multiple-choice questions with error-typed distractors, enabling fine-grained analysis of model failures. The training methodology involves supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO) on controlled subsets of the data. The systematic ablation study comparing supervision by reasoning operation versus action type is a strong methodological choice that isolates the source of transfer.
The experimental evaluation is extensive and rigorous. The authors evaluate 38 frontier MLLMs to establish a baseline deficit. They then demonstrate that training on small subsets (~2,000 items) of WOVEN significantly improves performance on 22 of 26 external benchmarks. A key strength is the causal intervention experiment, where replacing 30-50% of task-specific training data with WOVEN data yields comparable accuracy, proving the utility of the dataset as a data-efficient substitute. The correlation analysis ($r=0.86$) between WOVEN accuracy gains and external benchmark gains provides strong evidence for the "shared primitive" hypothesis.
The paper provides high reproducibility. Code and datasets are released on GitHub and HuggingFace. The generation process using Wan2.2 is specified, and the training recipes (SFT + GRPO) are detailed. The use of standard benchmarks for evaluation ensures that results can be verified against existing baselines.
The reliance on synthetic video generation (Wan2.2) may introduce domain gaps compared to real-world physics, although the authors argue for realism. The evaluation is primarily on multiple-choice tasks, which may not fully capture open-ended reasoning capabilities. The "frontier" models evaluated include some very recent releases (GPT-5.4, Qwen3-VL), which may limit immediate reproducibility for labs without access to these specific API versions or weights.
This paper has significant potential to influence how multimodal models are trained for embodied AI and physical reasoning. By identifying visual transition reasoning as a reusable primitive, it offers a data-efficient path to improving spatial and temporal understanding. The systematic recipe for selecting supervision (by reasoning operation rather than action) provides actionable guidelines for practitioners building world models. The paper establishes visual transition reasoning as a shared, trainable primitive for multimodal LLMs and provides a systematic training recipe validated by a new benchmark, WOVEN. By demonstrating that small, targeted subsets of transition supervision can broadly improve performance across diverse spatial, physical, and temporal benchmarks, the work offers a data-efficient and theoretically grounded approach to enhancing visual world modeling capabilities in current MLLMs.
Reconstructing complete, scene-aligned 3D objects from casual images requires integrating sparse, uncertain observations and inferring surfaces hidden by occlusions. We present GATOR, a generative and agentic framework that recovers textured object assets and their scene-relative pose from one or more images. Our local modality mixer couples patch-aligned RGB, target-mask, and pointmap features before cross-view reasoning, preserving scene context while distinguishing the target from its surroundings. Text-guided semantic conditioning complements these spatial cues with category names and object captions through stage-specific adapters for structure, geometry, and appearance generation. The generated asset initializes a multimodal agent, providing instance-specific geometry and pose for targeted structural and texture refinement through an observation-guided edit-render-review loop. Across synthetic objects, cluttered tabletops, and indoor scenes, GATOR achieves strong geometric and appearance fidelity while recovering scene-relative pose from sparse observations. Time-budget comparisons and scene-level simulation further demonstrate the reconstruction efficiency and simulation readiness. Project page: https://research.nvidia.com/labs/lpr/gator/
Primary: NVIDIA
All Institutions: NVIDIA
[One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. GATOR introduces a novel generative-agentic framework for 3D object reconstruction that couples a modality mixer for initial asset generation with an iterative agentic refinement loop, achieving strong geometric and appearance fidelity from casual images while recovering scene-relative pose, though the lack of detailed quantitative results in the provided text limits the full assessment of its empirical superiority over existing feed-forward and iterative baselines.
The paper proposes GATOR, a framework combining a "local modality mixer" for feature fusion (RGB, mask, pointmap) with a "multimodal agent" for iterative refinement. The core novelty lies in the agentic loop (edit-render-review) that uses the initial generative output to guide targeted structural and texture corrections. This hybrid approach of one-shot generation followed by agentic refinement is a significant architectural shift from purely feed-forward reconstruction models. The use of stage-specific adapters for text-guided conditioning is a standard but effective technique. The methodology is sound, leveraging recent trends in agentic AI for 3D tasks, though the specific implementation details of the "agent's" decision-making process (e.g., how it selects which parts to refine) are critical to its success and likely complex.
The paper claims strong geometric and appearance fidelity on synthetic objects, cluttered tabletops, and indoor scenes. It includes time-budget comparisons and scene-level simulation readiness tests. However, the provided text is a placeholder with section headers but no actual quantitative results, tables, or figures. Without specific metrics (e.g., Chamfer Distance, FID, PSNR) or comparisons to state-of-the-art baselines (like Zero-1-to-3, Wonder3D, or recent agentic 3D papers), the experimental evaluation cannot be fully verified. The claim of "strong fidelity" is unsubstantiated in the provided text.
The project page is provided, which is a positive sign for reproducibility. However, the lack of detailed hyperparameters, agent prompt engineering details, and training data specifics in the provided text makes independent reproduction difficult. The "agentic" component often relies on proprietary LLMs or specific prompt strategies that may not be fully open-sourced.
The primary limitation is the reliance on an agentic loop, which can be computationally expensive and slow compared to feed-forward methods, despite the "time-budget" claims. The performance on real-world, uncontrolled casual images (as opposed to synthetic or curated datasets) is not clearly demonstrated in the provided text. The "scene-aligned" pose recovery from sparse observations is a difficult problem, and the robustness of this component to severe occlusions or ambiguous viewpoints is a potential weakness.
This work has high potential impact in robotics, AR/VR, and digital twin creation, where accurate 3D asset recovery from casual photos is crucial. The agentic approach could be extended to other 3D tasks like scene editing or animation. It represents a step towards more interactive and intelligent 3D reconstruction pipelines. [One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. GATOR introduces a novel generative-agentic framework for 3D object reconstruction that couples a modality mixer for initial asset generation with an iterative agentic refinement loop, achieving strong geometric and appearance fidelity from casual images while recovering scene-relative pose, though the lack of detailed quantitative results in the provided text limits the full assessment of its empirical superiority over existing feed-forward and iterative baselines.
Unified multimodal diffusion large language models (dLLMs) offer a single architecture for both image generation and multimodal understanding, but their iterative decoding requires tens to hundreds of forward passes. Existing few-step distillation methods largely focus on either image generation or text generation, making it unclear how to compress a fully discrete multimodal dLLM into a single efficient student while preserving both generation and understanding. We introduce Omni-Diffusion-Distill, a unified two-stage distillation framework that retains strong generation and understanding capabilities while substantially reducing the inference cost of a unified multimodal dLLM. Omni-Diffusion-Distill aligns the distillation of both generation and understanding, for both images and text, in the discrete token space. In the first stage, the student is trained to skip decoding steps by replaying cached teacher trajectories, and in the second stage the student is refined on intermediate states along its own rollouts. We further remedy two sources of degradation in unified distillation with a pairwise collision penalty that reduces repetition under parallel text decoding, and entropy-matched guidance that prevents entropy collapse caused by fitting the sharpened teacher distribution in image generation. Omni-Diffusion-Distill achieves state-of-the-art trade-offs between decoding efficiency and generation and understanding performance for multimodal dLLMs, reducing image generation from 128 to 8 decoding steps and multimodal understanding from 512 to 64, giving 18.2x and 21.2x wall-clock speedups. Under these budgets, it scores 0.828 on GenEval and 83.0 on DPG-Bench for text-to-image generation, while reaching GPT judge scores of 20.0 on MM-Vet and 57.2 on COCO captioning (twice the teacher's 28.4 at the same steps) for multimodal understanding.
Primary: Meta
All Institutions: Meta
[One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper introduces a unified two-stage distillation framework for discrete multimodal dLLMs that effectively compresses inference steps for both image generation and text understanding, achieving state-of-the-art efficiency-performance trade-offs through targeted remedies for entropy collapse and parallel decoding repetition.
The paper proposes Omni-Diffusion-Distill, a two-stage distillation framework specifically designed for unified multimodal diffusion large language models (dLLMs) that operate in a fully discrete token space. The methodology is well-structured, addressing the specific challenges of compressing iterative decoding for both image generation and text understanding within a single architecture. Stage 1 employs off-policy trajectory replay to teach the student to skip decoding steps, while Stage 2 utilizes on-policy refinement on the student's own rollouts to mitigate exposure bias. The core technical contributions are two targeted remedies for modality-specific degradation: an entropy-matched guidance mechanism to prevent entropy collapse in image generation (caused by fitting sharpened CFG targets), and a pairwise collision penalty to reduce repetition in parallel text decoding. The adaptation of distribution matching distillation (DMD) to discrete tokens via an auxiliary network and direct gradient specification is a sound technical adaptation of continuous diffusion techniques to the discrete domain.
The experimental evaluation is comprehensive and rigorous. The authors distill the Lumina-DiMOO model and compare against strong baselines (Di[M]O, CDLM, T3D) across 17 benchmarks covering Text-to-Image (T2I), Image-to-Image (I2I), and Multimodal Understanding (MMU). The results demonstrate significant efficiency gains (18.2x and 21.2x wall-clock speedups) with minimal performance degradation, often outperforming the few-step teacher. The inclusion of ablation studies for the collision penalty and entropy-matched guidance, along with detailed analysis of degeneration metrics (repetition, entropy), provides strong evidence for the effectiveness of the proposed components. The use of GPT-5.5 for open-ended evaluation is a modern and appropriate choice, though it introduces some dependency on external models.
The paper provides high reproducibility. It details the training data mixture, optimization settings, and algorithms for both stages. The specific hyperparameters for the loss terms and the bisection method for entropy matching are described. While the code is not explicitly linked in the provided text, the level of detail in the appendices (algorithms, loss derivations, data sources) suggests that reproduction is feasible for a skilled practitioner. The use of standard benchmarks and clear evaluation protocols further supports reproducibility.
The primary limitation is the scope of validation; the method is only tested on one specific unified dLLM architecture (Lumina-DiMOO). It is unclear how well the framework generalizes to other unified models with different attention mechanisms or tokenization schemes. Additionally, the student model still trails the full-step teacher on certain fine-grained tasks like text rendering and detailed captioning. The reliance on a large teacher model and the computational cost of collecting teacher trajectories, while one-time, remains a barrier for smaller labs.
This work has significant potential impact on the deployment of unified multimodal models. By drastically reducing inference costs while maintaining quality, it makes real-time multimodal interaction more feasible. The techniques developed for handling entropy collapse and parallel decoding artifacts are likely to be applicable to other discrete generative models beyond dLLMs. The framework bridges the gap between high-quality iterative samplers and efficient few-step inference, a critical area for practical AI applications. [One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper introduces a unified two-stage distillation framework for discrete multimodal dLLMs that effectively compresses inference steps for both image generation and text understanding, achieving state-of-the-art efficiency-performance trade-offs through targeted remedies for entropy collapse and parallel decoding repetition.
User simulators are gaining attention as environments to train LLM assistants that can better cope with the nuances of multi-turn conversations with real users. Prior work on user simulation has emphasized generating realistic user utterances, leaving unexplored their potential as a source of direct feedback on the assistant's responses. Such evaluation signals may not be explicitly communicated in language, but can instead reside in the latent reasoning of users. In this work, we introduce user simulators with intrinsic feedback, namely, models equipped with latent evaluation states: internal thoughts, reactions, and scalar ratings that evaluate the assistant from a first-person user perspective at each conversation turn. We train LLM-based simulators using interaction logs between real users and assistants and demonstrate that intrinsic feedback improves simulation realism and critiquing behaviors versus prompted simulator baselines, while also providing turn-level evaluations that existing fine-tuned simulators lack. Further, evaluations generated by our trained simulators correlate better with human judgments than those of prompted simulators of the same size. Lastly, our experiments reveal that simulators with intrinsic feedback provide effective reward signals for multi-turn RL training, matching setups that rely on external judge LLMs for rewards while cutting RL compute by more than half.
Primary: University of Toronto
All Institutions: University of Toronto, Google DeepMind
The paper introduces user simulators with latent evaluation states that provide intrinsic feedback for both realistic simulation and efficient RL training. By integrating evaluation signals directly into the simulator's latent reasoning, the authors demonstrate a method that improves simulation realism and reduces the computational cost of training LLM assistants via multi-turn reinforcement learning, offering a compelling alternative to external judge-based reward systems.
The paper proposes a novel framework for user simulators equipped with "Latent Evaluation States" (LESs). Unlike prior work that treats simulators purely as utterance generators or uses latent states only for goal alignment, this approach explicitly models the user's internal evaluation of the assistant (thoughts, ratings, reactions) as part of the generation process. The methodology involves fine-tuning LLMs (Gemma3, Qwen2.5) on interaction logs where these latent states are synthetically generated by a larger model (Gemini 3.1-Pro). The core innovation is the dual utility of these states: they improve the realism of the generated user turns and provide an intrinsic, turn-level reward signal for Reinforcement Learning (RL) training of assistant models, eliminating the need for external judge LLMs.
The experiments are comprehensive, covering simulation realism (perplexity, human alignment scores), feedback accuracy (correlation with human ratings), and downstream RL performance. The paper demonstrates that LES-based simulators outperform prompted baselines and standard SFT simulators in realism and feedback correlation. Crucially, it shows that using intrinsic rewards from the simulator for multi-turn GRPO training yields assistants that perform comparably to those trained with external judge rewards, while reducing computational overhead by more than 50%. The use of WildChat and SimulatorArena datasets provides a solid empirical foundation.
The paper provides detailed prompts for latent state generation and judging, along with specific hyperparameters for training. However, the reliance on a proprietary model (Gemini 3.1-Pro) for data augmentation and evaluation limits full reproducibility for labs without access to such models. The code is not explicitly linked in the provided text, though the methodology is described in sufficient detail to be replicated with open-source alternatives.
The primary limitation is the dependency on a large, potentially proprietary model to generate the synthetic latent states for training data. Additionally, the RL experiments are conducted on relatively small assistant models (1.5B parameters), and it remains to be seen if the intrinsic reward signal scales effectively to larger, more complex assistant models. The "decontamination" of WildBench is a good practice, but the overall dataset size for RL training (100k conversations) is modest compared to state-of-the-art RL efforts.
This work has significant implications for the field of LLM alignment and evaluation. By integrating evaluation into the simulator itself, it offers a more efficient and potentially more aligned path for training multi-turn conversational agents. The reduction in computational cost for RL training is a practical advantage that could accelerate the development of more robust AI assistants. It shifts the paradigm from "simulator as environment" to "simulator as critic," which is a valuable conceptual advance. The paper introduces user simulators with latent evaluation states that provide intrinsic feedback for both realistic simulation and efficient RL training. By integrating evaluation signals directly into the simulator's latent reasoning, the authors demonstrate a method that improves simulation realism and reduces the computational cost of training LLM assistants via multi-turn reinforcement learning, offering a compelling alternative to external judge-based reward systems.
Real-time whole-body controllers for legged robots typically plan through a fixed nominal model and degrade when the deployed dynamics change. Adaptive methods typically require a model structure that contact dynamics do not provide, or they need offline training for each anticipated condition. We present Look-back and Look-ahead Adaptive Model Predictive Path Integral control (LLA-MPPI). The method converts whole-body adaptation into selection over a bank of GPU-batched contact simulators with different physical or structural parameters. Windowed prediction errors select the simulator that best explains recent motion. A whole-body MPPI planner optimizes controls through the selected model. The framework requires no offline training, and its selected hypotheses are physically interpretable. Across four simulated tasks, it achieves 97.5% success while the strongest baseline reaches 74% and an oracle with the true model reaches 98.5%. Hardware validation on a Unitree Go2 shows the robot walking under a payload added mid-run, walking after one leg is disabled, and pushing a box to its goal while increasing its mass on the fly. Code, videos, and project details are available at: https://lla-control.github.io
Primary: Massachusetts Institute of Technology
All Institutions: Massachusetts Institute of Technology
LLA-MPPI introduces a novel adaptive control framework that leverages GPU-batched physics simulation to select the best-fitting dynamics model from a bank of hypotheses, enabling real-time whole-body control of legged robots under significant model mismatch without offline training. The paper demonstrates near-oracle performance in simulation and successful hardware validation, establishing a new standard for adaptive model-based control in contact-rich environments.
The paper proposes LLA-MPPI, a framework that decouples adaptive model identification from trajectory optimization. The core innovation is the use of a "bank" of GPU-batched physics simulators (via MuJoCo Warp) to perform model selection. Instead of using classical estimators (like Kalman filters or particle filters) that require specific linear or affine structures, the method treats adaptation as a classification problem over a discrete set of physical hypotheses. The "Look-back" stage runs on the GPU, evaluating thousands of candidate dynamics models in parallel against recent state transitions to select the best-fitting model. The "Look-ahead" stage runs on the CPU, using MPPI (Model Predictive Path Integral) control to plan trajectories using the selected model. This asynchronous, heterogeneous architecture is technically sound and cleverly leverages modern hardware capabilities (GPU parallelism for wide/shallow simulation, CPU threads for narrow/deep planning). The removal of the need for differentiable or closed-form dynamics is a significant methodological advantage for contact-rich robotics.
The evaluation is rigorous and comprehensive. The authors test the method on four distinct simulated tasks (asymmetric payload, leg lock, slope/friction variation, variable-mass box pushing) and validate on hardware (Unitree Go2). The comparison against an oracle (true model) shows the method achieves 97.5% success vs. 98.5% for the oracle, indicating near-perfect adaptation. Comparisons against baselines (Nominal MPPI, Disturbance Observer, Continuous Parameter Estimator) show substantial improvements in success rate and fall rate. The inclusion of a comparison with Domain Randomized PPO is valuable, highlighting the trade-off between offline training costs and online adaptability. The hardware results are particularly convincing, demonstrating real-world applicability in scenarios like mid-run payload addition and leg failure.
The paper provides a project URL with code, videos, and details. The specific hardware (RTX 2080 Ti, i9-9980XE) and software versions (MuJoCo 3.4.0, MuJoCo-Warp 1.12.0) are clearly stated. The hyperparameters for the model bank size, window size, and switching margins are detailed for each task. This level of detail supports high reproducibility.
The method relies on a finite, design-time hypothesis bank. If the true dynamics fall outside the discretized range of the bank, adaptation will fail. The paper acknowledges this and suggests future work on adaptive bank refinement. Additionally, the method requires significant GPU resources to maintain the bank of simulators, which may not be available on all robotic platforms. The distinguishability of models depends on excitation; if the robot's motion does not excite a particular uncertainty (e.g., friction when not slipping), the model selection may be ambiguous, though the hysteresis mechanism helps mitigate this.
This work has high potential impact in the field of legged robotics and adaptive control. By demonstrating that high-fidelity, contact-rich dynamics can be adapted to in real-time without offline training or complex analytical derivations, it opens the door to more robust and versatile robotic systems. The technique of using GPU-batched simulation for model selection could be extended to other domains involving complex, non-differentiable dynamics, such as soft robotics or multi-agent systems. LLA-MPPI introduces a novel adaptive control framework that leverages GPU-batched physics simulation to select the best-fitting dynamics model from a bank of hypotheses, enabling real-time whole-body control of legged robots under significant model mismatch without offline training. The paper demonstrates near-oracle performance in simulation and successful hardware validation, establishing a new standard for adaptive model-based control in contact-rich environments.
Animal evidence shows that precise voluntary movements arise from rotational neural population dynamics in motor cortex, but their physical effects remain unknown. We developed a robotic analog of biological motor systems with artificial muscles, multimodal sensors, and a neural network controller trained via reinforcement learning. The robotic analog exhibited accurate movements, robustness to damage, and neural population dynamics akin to animals. This task-driven, embodied model illuminates the causal link between neural population dynamics and motor outcomes. We discovered that neural rotations generate oscillatory maneuvers orthogonal to the reaching direction, optimizing trajectory adjustments, which is confirmed by primate neural data. The model also revealed counterintuitive neural energy principles under sensor and motor redundancies, and striking Eureka moments during motor learning, bridging biological and artificial systems. These findings provide new perspectives on how neural dynamics contribute to accurate and flexible movement, inspiring future intelligent robots with animal-like mobility.
Primary: Peking University
All Institutions: Peking University, State Key Laboratory of Transvascular Implantation Devices, The Second Affiliated Hospital of Zhejiang University School of Medicine, IDG/McGovern Institute for Brain Research, Peking-Tsinghua Center for Life Sciences, School of Advanced Manufacturing and Robotics, School of Life Sciences
The paper introduces a robotic analog of biological motor systems to causally link neural rotational dynamics to physical movement outcomes. By combining a biomimetic robot arm with an RNN controller trained via reinforcement learning, the authors demonstrate that rotational neural dynamics drive orthogonal trajectory adjustments, a finding validated in primate data, thereby providing a new embodied framework for decoding the functional role of neural population dynamics in motor control.
The paper proposes a "robotic analog" of biological motor systems, combining a continuum robot arm with liquid crystal elastomer (LCE) artificial muscles, multimodal sensors (vision, IMUs, self-sensing muscles), and a Recurrent Neural Network (RNN) controller trained via Soft Actor-Critic (SAC) reinforcement learning. The core methodological innovation lies in using this embodied, transparent system to causally link neural population dynamics (specifically rotational dynamics) to physical motor outcomes. The authors employ dimensionality reduction techniques like jPCA and CCA to analyze the latent dynamics of the RNN controller, comparing them to primate motor cortex data. The approach is rigorous, moving beyond simple simulation to physical embodiment to test hypotheses about neural dynamics that are difficult to test in vivo.
The experiments are extensive, covering simulation and real-world physical tests. The robot achieves sub-5mm reaching errors and demonstrates robustness to muscle failure without retraining. Key findings include the emergence of rotational dynamics in the RNN layers that drive orthogonal maneuvers, a phenomenon confirmed in primate data. The paper also analyzes neural energy efficiency under different sensory feedback conditions and identifies "Eureka moments" during learning where performance and dynamics shift abruptly. The validation against primate neural data strengthens the biological relevance of the findings.
The paper provides detailed descriptions of the hardware fabrication (LCE fibers, IMU setup) and the simulation environment (MuJoCo, constitutive models). However, specific code repositories or hyperparameters for the RL training are not explicitly provided in the text, which may limit immediate reproducibility. The physical setup is complex and likely expensive to replicate.
The system is limited to a single arm and relatively simple reaching tasks. The "neural" dynamics are those of an artificial RNN, not biological neurons, so while the analogy is strong, it is not a direct proof of biological mechanisms. The "Eureka moment" finding, while interesting, is based on a specific training trajectory and may not generalize to all RL algorithms or tasks.
This work bridges the gap between computational neuroscience and robotics, offering a new paradigm for studying motor control. It provides insights into how neural dynamics contribute to movement precision and flexibility, which could inspire more biologically plausible robot controllers. The findings on rotational dynamics and energy efficiency have implications for both understanding brain function and designing efficient robotic systems. The paper introduces a robotic analog of biological motor systems to causally link neural rotational dynamics to physical movement outcomes. By combining a biomimetic robot arm with an RNN controller trained via reinforcement learning, the authors demonstrate that rotational neural dynamics drive orthogonal trajectory adjustments, a finding validated in primate data, thereby providing a new embodied framework for decoding the functional role of neural population dynamics in motor control.