Last 7 Days (October 20 – October 26, 2026)
Humanoid mobility over complex terrain requires accurate foot placement and coordinated whole-body motion capable of robust adaptation. Reinforcement learning (RL) policies trained without motion references generalize well across terrains, yet eliciting effective whole-body coordination through reward design and exploration alone remains challenging. In contrast, reference-based methods yield natural motions but struggle to adapt beyond the predefined trajectories in their training data. We introduce a learning framework that bridges these two approaches by using motion data strictly as a training prior. An explicit, self-supervised, attention-based contact planner predicts future body-environment contacts, while a contact-conditioned tracker translates these plans into coordinated whole-body motion. Together, they establish contact as an interface between sensory observations and control actions. This abstraction allows the policy to internalize the contact-aware structure of locomotion rather than directly replicating kinematics. Consequently, a single policy can exploit data-driven motion priors for natural whole-body coordination while retaining the flexibility to discover environment-dependent behaviors through reinforcement learning. Building on the learned contact planner and tracker, a subsequent fine-tuning stage bootstraps their acquired capabilities to explore environment variations beyond those covered by the original motion data. Operating entirely from onboard observations without any reference inputs at deployment, this framework enables versatile humanoid mobility in challenging environments.
Primary: National University of Singapore
All Institutions: National University of Singapore, Robotic Systems Lab (ETH Zurich)
The paper introduces a contact-centric framework for humanoid mobility that uses attention-based planning to predict body-environment interactions, enabling a single policy to generalize across diverse terrains and skills without reference motions at deployment. This represents a significant step forward in humanoid control by decoupling contact planning from whole-body execution, allowing for more robust and adaptable behaviors in complex environments.
The paper proposes a novel framework for humanoid mobility that decouples the "what" (contact planning) from the "how" (whole-body tracking). The core innovation is the use of an attention-based mechanism to predict explicit contact locations on a terrain map, serving as an intermediate representation between perception and control. This is a significant conceptual shift from prior work that either relies on full-body trajectory tracking (which lacks generalization) or pure RL (which struggles with complex reward shaping for whole-body coordination). The three-stage training pipeline (Tracker Pretraining, Planner Supervision, RL Fine-tuning) is well-structured and logically sound, effectively leveraging motion data as a prior without binding the deployment policy to specific trajectories. The use of FiLM modulation for body-specific attention queries is a clever architectural choice that allows a single network to handle multiple contact points.
The experimental evaluation is robust and comprehensive. The authors conduct extensive real-world experiments on the Unitree G1 robot, demonstrating a wide variety of skills including ladder climbing, crawling, vaulting, and failure recovery. The simulation-based ablations are rigorous, isolating the contributions of the contact representation, the attention mechanism, and the RL fine-tuning stage. The comparison with MGMT (a strong baseline using diffusion-based motion generation) shows clear advantages in success rates and contact placement accuracy, particularly on sparse terrains. The qualitative comparisons with other methods (PHP, LLP) are fair and highlight the unique capabilities of the proposed approach. The inclusion of failure recovery experiments is a strong point, demonstrating the robustness of the learned policy.
The paper provides detailed descriptions of the network architectures, training procedures, and reward functions. The use of standard components (MLP trackers, attention planners, PD control) makes the method relatively accessible to reproduce. However, the specific details of the motion retargeting pipeline and the LiDAR mapping framework are described at a high level, which might make exact reproduction challenging without access to the code. The lack of a public code repository at the time of review (arXiv preprint) is a minor drawback, but the methodological details are sufficient for a skilled researcher to implement a similar system.
The paper acknowledges several limitations. The current perception is limited to 2.5D terrain maps, which restricts the system's ability to handle overhead contacts or complex 3D manipulation. The system is designed for local goal-conditioned locomotion and does not address global navigation. The contact representation cannot distinguish between motions with identical contact sequences but different kinematics (e.g., backflip vs. vertical jump). These are reasonable limitations for the current scope, but they highlight areas for future work.
This paper has significant potential impact on the field of humanoid robotics. By establishing contact as a general interface for mobility, it provides a scalable framework for adding new skills to humanoid robots without requiring exhaustive manual augmentation of motion datasets. The approach bridges the gap between data-driven imitation and reward-driven exploration, offering a path toward more versatile and robust humanoid control. The insights gained from this work could be applied to other robotic systems that require complex whole-body coordination, such as quadrupeds or manipulators. The paper introduces a contact-centric framework for humanoid mobility that uses attention-based planning to predict body-environment interactions, enabling a single policy to generalize across diverse terrains and skills without reference motions at deployment. This represents a significant step forward in humanoid control by decoupling contact planning from whole-body execution, allowing for more robust and adaptable behaviors in complex environments.
We give an algorithm for computing an $\eps$-approximate fixed point of any $d$-dimensional nonexpansion with respect to the $\ell_\infty$ norm in time $\log(1/\eps)^{O(\log^2 d)}$. At its core, our method uses a boosting argument to geometrically shrink a search region. Among other consequences, our algorithm implies that simple stochastic games can be solved in quasipolynomial time.
Primary: Unknown
All Institutions: Unknown
The paper presents a quasipolynomial-time algorithm for computing approximate fixed points of $\ell_\infty$-contractions, resolving a major open problem in the complexity of simple stochastic games. By introducing a novel boosting-based geometric search strategy, the authors improve the running time from subexponential to quasipolynomial, providing a significant theoretical advance with broad implications for game theory, verification, and optimization.
The paper introduces a deterministic algorithm for computing $\epsilon$-approximate fixed points of $\ell_\infty$-contractions in quasipolynomial time, specifically $\log(1/\epsilon)^{O(\log^2 d)}$. The core innovation is a "boosting" argument applied to a geometric search problem. The authors define a "thin labeling" subroutine that recursively splits dimensions based on median side lengths to identify which faces of a search box can be safely shrunk. They then amplify the accuracy of these thin labelings using a technique analogous to AdaBoost, treating the labeling problem as an extremal amplification game. This approach cleverly reduces the subexponential dependence on dimension $d$ found in prior work (Feodorov and Haslebacher) to a quasipolynomial dependence. The theoretical framework is rigorous, with detailed proofs of the potential functions and correctness of the boosting steps.
As a purely theoretical computer science and optimization paper, there are no empirical experiments, datasets, or benchmark comparisons. The "evaluation" is entirely analytical, proving tight bounds on the number of arithmetic operations and function evaluations required. The significance is derived from the theoretical breakthrough in complexity bounds rather than empirical performance.
The algorithm is described in full pseudocode (Algorithms 1, 2, 3, 4). The theoretical guarantees are proven rigorously. However, without code or empirical benchmarks, "reproducibility" in the practical sense is limited to the ability of other theorists to verify the proofs. The unit-cost real machine model is used for analysis, which is standard in this field but limits direct practical implementation without bit-complexity analysis (which is noted as straightforward but not fully detailed).
The algorithm is deterministic and applies to $\ell_\infty$ norms. While it solves Simple Stochastic Games (SSGs) in quasipolynomial time, it does not resolve the open question of whether SSGs are in P. The quasipolynomial bound, while a significant improvement over subexponential, is still super-polynomial, meaning it may not be practical for very large dimensions or high-precision requirements. The paper focuses on the worst-case complexity; smoothed analysis or average-case performance is not addressed.
This result has significant implications for the complexity of several fundamental problems in game theory and verification, including Simple Stochastic Games, stochastic parity games, mean-payoff games, and modal $\mu$-calculus model checking. By providing a quasipolynomial-time algorithm for these problems, it advances the state of the art in understanding the computational hardness of these decision problems. It also contributes to the broader field of fixed-point computation, which is central to reinforcement learning (value iteration) and distributed computing. The use of boosting for geometric search is a novel methodological contribution that could inspire further research in algorithmic game theory. The paper presents a quasipolynomial-time algorithm for computing approximate fixed points of $\ell_\infty$-contractions, resolving a major open problem in the complexity of simple stochastic games. By introducing a novel boosting-based geometric search strategy, the authors improve the running time from subexponential to quasipolynomial, providing a significant theoretical advance with broad implications for game theory, verification, and optimization.
Preference optimization can improve every ranking metric while building a hidden, shared representation shift that the relative loss barely sees, one that can eventually destroy generation. We trace this failure to the common-mode offset, a shared directional shift in the model's hidden states that appears across token positions, prompts, and both chosen and rejected responses. Because it moves chosen and rejected representations together, the relative ranking loss barely reacts to its accumulation. As this offset grows, it projects onto the output layer to form a vocabulary well, elevating a single token's probability until it dominates decoding and causes generation collapse. The isolated offset alone reconstructs the vocabulary well, injecting the learned offset into an untreated base model recreates the collapse where norm-matched random directions do not, and masking this common component from the loss prevents the well in our flagship setting. Under the per-token ranking loss we study, the same collapse appears in 14 of 17 base-model lineages and under full-parameter fine-tuning. Moreover, the offset is not merely an artifact of collapse, since it is already present in textbook DPO and SimPO runs that form no well. Textbook DPO keeps it weakly aligned with any single vocabulary direction even with three times our base budget. Trained from base models for that budget, however, full-parameter SimPO drives it into collapse on both Llama-2 and Llama-3.1, and the same kind of offset grows, without a well, from a public SFT checkpoint under the official SimPO recipe. Across 17 UltraFeedback runs, the better a run ranks held-out responses, the less often two public reward models prefer its samples to the base model's. These findings expose a blind spot in standard ranking metrics and introduce a representation-level signal that can warn of generation collapse before it occurs.
Primary: The University of Tokyo
All Institutions: The University of Tokyo, TIER IV, Inc.
The paper identifies a hidden "common-mode offset" in preference optimization that causes generation collapse despite improved ranking metrics, providing a causal explanation and effective mitigation strategy. This is a significant contribution to the understanding of LLM alignment failures, offering both a diagnostic signal and a practical solution that is likely to influence future alignment research and practice.
The paper introduces a novel diagnostic framework for preference optimization failures, identifying a "common-mode offset" in the final hidden states as the root cause of vocabulary collapse. The methodology is rigorous, combining theoretical analysis of loss invariances with extensive empirical validation. The authors define the offset as the mean displacement of hidden states between trained and base models, demonstrating that this shared vector moves chosen and rejected representations in tandem, thereby evading the relative ranking loss. They propose a "dose" metric to quantify the risk of collapse and introduce a mitigation strategy involving centering the rejected-side loss and applying norm/loudness penalties to the offset. The causal interventions (injecting the offset into base models, masking it from the loss) are well-designed and effectively isolate the mechanism.
The experimental scope is impressive, covering 17 base-model lineages (0.5B to 14B parameters) across five tokenizer families, using both LoRA and full-parameter fine-tuning. The use of two distinct datasets (driving preferences and UltraFeedback) and multiple objectives (DPO, SimPO, Plackett-Luce ranking) strengthens the generalizability of the findings. The discovery that ranking metrics improve while generation collapses (anticorrelation with external reward models) is a significant empirical contribution. The cross-family validation of the mitigation strategy up to three times the base budget is convincing.
The paper provides detailed descriptions of training configurations, measurement protocols, and penalty terms. The authors state that code and data will be released upon publication. The specific hyperparameters (learning rates, batch sizes, LoRA ranks) are clearly defined. The measurement of "dose" and "loudness" is well-defined, allowing for replication. The reliance on specific public checkpoints (e.g., Llama-3-8B SFT from SimPO authors) aids reproducibility.
The mitigation strategy is calibrated on specific datasets and may require tuning for other domains. The analysis is primarily focused on the final hidden state, and the interaction with earlier layers is not deeply explored. The "diffuse degradation" observed in SimPO is noted but not fully resolved by the proposed offset control, indicating a second failure mode that remains unaddressed. The study is limited to English-centric models and datasets.
This work has high impact for the LLM alignment community by exposing a critical blind spot in standard evaluation metrics. It provides a practical diagnostic tool (monitoring the common-mode offset) that can be integrated into training pipelines to prevent generation collapse. The findings challenge the assumption that improving ranking metrics guarantees healthy generation, prompting a re-evaluation of alignment objectives. The proposed penalties offer a simple, effective remedy for a widespread issue. The paper identifies a hidden "common-mode offset" in preference optimization that causes generation collapse despite improved ranking metrics, providing a causal explanation and effective mitigation strategy. This is a significant contribution to the understanding of LLM alignment failures, offering both a diagnostic signal and a practical solution that is likely to influence future alignment research and practice.
AI agents increasingly act as analysts whose decision can often be cast as hypothesis tests. Classical tests control the Type I error rate, the probability of rejecting a true null hypothesis, for fixed procedures. However, AI analysts introduce internal randomness and can evolve continuously over time. As AI analysts evolve, the risk of approving versions with Type I error rates above the target can accumulate, and adaptive use of evaluation feedback can lead to overfitting. We introduce continual validation, which evaluates each proposed AI analyst version before deployment using simulated data under the null hypothesis, with scenarios that can be randomized and expanded over time. We prove that, regardless of how updates are chosen or how many are proposed, the lifetime probability of ever deploying a version whose Type I error rate exceeds the allowed level remains bounded. This guarantees that every subsequent decision controls the Type I error rate at the target level across these scenarios. At the same time, updates with sufficiently low Type I error rates pass validation with high probability, allowing the analyst to continuously improve its performance without sacrificing rigor. In two examples involving evolving AI analysts for bioequivalence studies and product A/B tests, Type I error rates reached 0.098 and 0.183 without continual validation, compared with at most 0.045 and 0.044 with it, while retaining 82% and 96% of the power from optimally selected valid updates.
Primary: Purdue University
All Institutions: Purdue University, Anthropic, Lilly Purdue Innovation Institute
The paper proposes a rigorous statistical framework, "continual validation," that guarantees Type I error control for AI agents that evolve over time by using sequential hypothesis testing with a lifetime error budget. By proving that the probability of deploying an invalid version remains bounded regardless of the number of adaptive updates, the work addresses a critical safety gap in the deployment of self-improving AI analysts, offering a practical method to maintain statistical rigor while allowing for continuous improvement in power and automation.
The paper introduces "continual validation," a framework for controlling Type I error in evolving AI agents. The core innovation is the use of sequential hypothesis testing with a lifetime error budget. By freezing each proposed version, simulating null data, and using exact binomial confidence bounds with a shrinking error budget ($\alpha_r$), the authors prove that the probability of ever deploying an invalid version remains bounded. This addresses the "garden of forking paths" problem in adaptive AI systems. The theoretical contribution is solid, leveraging standard statistical tools (Clopper-Pearson bounds, union bounds) in a novel context of agent evolution. The "allowed paths" proposition is a clever practical addition that reduces computational cost by validating deterministic decision rules rather than stochastic LLM outputs.
The experiments are rigorous and well-motivated. The authors test the framework on two distinct domains: bioequivalence studies (pharmacokinetics) and product A/B testing. They compare continual validation against four baselines: no validation, static benchmark, improvement test, and Learn-then-Test. The results clearly demonstrate that without continual validation, Type I error rates inflate significantly (up to 0.78 in one case), while continual validation keeps them below the target (0.05) while retaining high statistical power (82-96% of oracle). The inclusion of off-grid scenarios to test robustness is a strong point, showing the method generalizes beyond the specific validation grid.
The paper provides detailed algorithmic descriptions and specifies the exact statistical bounds used. The use of open-source tools (GrowthBook) and public LLMs (Claude, Qwen, DeepSeek) enhances reproducibility. However, the specific prompts and internal logic of the "developer" agent are not fully detailed, which might make exact replication of the evolutionary trajectories difficult, though the statistical guarantees hold regardless of the specific updates proposed.
The primary limitation is the reliance on the ability to simulate null scenarios. If the null hypothesis space is complex or hard to simulate, the method's applicability is limited. Additionally, the computational cost of simulating $m_r$ datasets for each update can be high, although the "allowed paths" optimization mitigates this. The guarantee is strictly for the simulated scenarios; while robustness is shown empirically, there is no formal guarantee for unobserved data distributions.
This paper has significant implications for the deployment of AI agents in high-stakes domains like healthcare and finance. As AI agents become more autonomous and self-improving, the lack of statistical guarantees for their decisions is a major barrier to adoption. This work provides a rigorous framework to ensure that AI analysts do not inflate error rates as they evolve, potentially becoming a standard protocol for validating AI-driven statistical decisions. It bridges the gap between classical statistical theory and modern adaptive AI systems. The paper proposes a rigorous statistical framework, "continual validation," that guarantees Type I error control for AI agents that evolve over time by using sequential hypothesis testing with a lifetime error budget. By proving that the probability of deploying an invalid version remains bounded regardless of the number of adaptive updates, the work addresses a critical safety gap in the deployment of self-improving AI analysts, offering a practical method to maintain statistical rigor while allowing for continuous improvement in power and automation.
Masked diffusion models (MDMs) generate sequences by progressively unmasking tokens, but errors introduced during decoding are typically irreversible. Recent work has shown that iterative correction can improve sample quality, yet existing theory fails to explain this given a learned, imperfect denoiser model. We re-formulate MDM inference as a Markov chain on the clean sequence space, defined by a remask-denoise kernel that randomly re-introduces masks and then denoises. This helps us analyze the actual denoising trajectory towards the data distribution. The analysis motivates Random Remask-Denoise (R$^2$D) inference, a purely test-time, finetuning-free, reward-free procedure. To further improve performance, we propose Trajectory Correction Training (TCT), a denoiser-training procedure that exposes the model to its own errors along the generation trajectory. On logical tasks, our approach exhibits a striking near-log-linear test-time scaling, achieving over 97% accuracy on hard Sudoku puzzles and over 99% accuracy on 3-SAT. Our approach also applies to pretrained models on actual language tasks, beating iterative correction baselines and the official Q-mode accuracy of LLaDA2.1-mini on HumanEval+.
Primary: Carnegie Mellon University
All Institutions: Carnegie Mellon University
The paper presents a unified Markov-chain framework for masked diffusion inference, deriving a principled test-time scaling algorithm (R2D) and a corresponding training method (TCT) that significantly improve accuracy on logical and coding tasks.
The paper proposes a unified Markov chain framework for Masked Diffusion Model (MDM) inference, re-formulating the process as a remask-denoise kernel on the clean sequence space. This theoretical shift allows for a rigorous analysis of the denoising trajectory using KL divergence contraction bounds. The authors derive a single-step update bound that identifies three key factors: timestep schedule, task structure, and model quality. Based on this, they propose Random Remask-Denoise (R2D) inference, an adaptive, reward-free test-time scaling method that iterates until prediction stabilizes. Additionally, they introduce Trajectory Correction Training (TCT), a training procedure that exposes the model to its own errors along the generation trajectory, effectively teaching it to correct mistakes. The methodology is sound, bridging the gap between diffusion theory and practical iterative correction by providing a principled justification for why and when remasking helps.
The experiments are rigorous and cover both synthetic logical tasks (Sudoku-Hard, 3-SAT) and a real-world language task (HumanEval+ with LLaDA2.1-mini). The results demonstrate a striking near-log-linear test-time scaling effect on logical tasks, achieving >97% accuracy on hard Sudoku and >99% on 3-SAT. The method outperforms baselines like ReMDM and ProSeCo in both accuracy and compute efficiency (NFE). The application to the 16B LLaDA2.1-mini model shows that R2D can beat the official Q-mode accuracy on HumanEval+ without any fine-tuning, which is a strong practical result. Ablation studies confirm the importance of uniform remasking and argmax decoding for the scaling effect.
The paper provides detailed pseudo-code for both the inference algorithm (R2D) and the training procedure (TCT). Hyperparameters such as correction budgets, early-exit patience, and model architectures are specified. The use of standard datasets (SATLIB, Sudoku-Extreme) and a public model (LLaDA2.1-mini) enhances reproducibility. However, the specific implementation details of the TCT training loop (e.g., how batches are managed across chains) are described in the appendix but could benefit from more explicit code-level details.
The primary limitation is the computational cost of iterative correction, although the paper argues it is more efficient than baselines. The method relies on argmax decoding for the scaling effect, which may limit diversity in generation tasks where sampling is preferred. The logical tasks, while impressive, are discrete and constraint-heavy; the generalizability to other types of continuous or open-ended generation tasks is less clear, though the HumanEval+ result is promising. The theory assumes a Markovian remask step, which is a simplification of some standard inference procedures.
This work provides a theoretical foundation for test-time scaling in discrete diffusion models, a rapidly growing area. By showing that iterative correction can be viewed as a Markov chain convergence process, it offers a principled way to allocate compute during inference. The ability to improve accuracy of pretrained models without fine-tuning (as shown with LLaDA2.1-mini) is highly valuable for practitioners who cannot retrain large models. It may influence future designs of diffusion-based LLMs to incorporate self-correction capabilities natively. The paper presents a unified Markov-chain framework for masked diffusion inference, deriving a principled test-time scaling algorithm (R2D) and a corresponding training method (TCT) that significantly improve accuracy on logical and coding tasks.
We consider online monitoring of an AI system that produces a sequence of decisions. At each round, the system outputs a proposed decision together with a confidence score estimating the trustworthiness of the decision. The decision and confidence score need not be produced by the same model: for example, one agent may propose a decision, while a separate verifier agent evaluates it and outputs a correctness or safety score. Based on this information, we must decide whether to accept the AI decision automatically or consult an expert, such as a human auditor, who reviews the proposal and either accepts or corrects it. We formulate this task as an online constrained optimization problem in which the goal is to minimize the frequency of expert auditing while guaranteeing that the fraction of AI errors that go unaudited stays below a user-defined threshold. We derive a family of algorithms with provable error-control guarantees with no assumptions, including a method that is optimally efficient: it asymptotically attains the best achievable auditing rate for both adversarial and i.i.d. data streams. Experiments on screening harmful requests to large language models, qualitative text annotation, and data labeling show that our methods achieve valid error control in practice while substantially reducing the need for human auditing compared to baselines.
Primary: Stanford University
All Institutions: Stanford University, University of California, University of Toronto
The paper introduces a rigorous online learning framework for selective auditing of AI decisions, providing distribution-free error control guarantees and asymptotically optimal auditing rates. By combining inverse probability weighting with online isotonic calibration, the authors develop methods that significantly reduce human oversight costs while maintaining strict reliability bounds, offering a practical and theoretically sound solution to the challenge of monitoring autonomous AI systems at scale.
The paper proposes a framework for "selective auditing" of AI decisions, formulated as an online constrained optimization problem. The core contribution is a family of algorithms that balance the cost of expert auditing against the risk of uncorrected AI errors. The primary method, "additive thresholding," uses inverse probability weighting to update a confidence threshold based on observed errors, providing distribution-free error control guarantees. A more sophisticated variant, "additive thresholding with confidence calibration," employs online isotonic regression (via exponentiated gradient descent on a simplex) to recalibrate confidence scores, ensuring pseudo-calibration while maintaining error bounds. The paper further derives an optimally efficient algorithm that asymptotically minimizes the auditing rate for both adversarial and i.i.d. data streams, connecting the problem to primal-dual methods in online learning. The theoretical analysis is rigorous, providing $O(1/T)$ error control bounds under linear error growth assumptions.
The experiments evaluate the methods on three tasks: screening harmful LLM requests, qualitative text annotation, and data labeling. The results demonstrate that the proposed methods achieve valid error control while significantly reducing the number of required human audits compared to baselines (such as fixed thresholds or the method from Kiyani et al.). The empirical validation supports the theoretical claims, showing that the calibration step improves efficiency without sacrificing reliability. However, the experimental section is somewhat brief, and the datasets are standard benchmarks rather than novel large-scale deployments, which limits the demonstration of scalability in real-world high-stakes environments.
The paper provides clear algorithmic descriptions (Algorithm 1) and theoretical definitions. However, specific hyperparameter choices (step sizes, grid points for calibration) are not fully detailed in the truncated text, and code availability is not explicitly confirmed in the provided excerpt. The reliance on standard online learning techniques (mirror descent, inverse probability weighting) suggests that implementation is feasible for practitioners familiar with these methods.
The theoretical guarantees assume that the number of errors grows linearly with time ($n_T = \Omega(T)$). In regimes where errors are very rare, the bounds may be loose or the methods may not provide strong guarantees, as noted in the text. The "optimal efficiency" result relies on discretizing confidence scores onto a grid, which introduces a discretization error that is not fully quantified in the main text. Additionally, the framework assumes a binary error indicator and a single confidence score, which may not capture the complexity of multi-class or multi-dimensional decision spaces.
This work has significant implications for the deployment of autonomous AI systems, particularly in safety-critical domains where human oversight is costly but necessary. By providing provable guarantees on error rates while minimizing human intervention, the paper offers a practical path toward scalable AI oversight. The connection to online learning and calibration makes the methods applicable to a wide range of sequential decision-making problems, potentially influencing future standards for AI reliability and auditing protocols. The paper introduces a rigorous online learning framework for selective auditing of AI decisions, providing distribution-free error control guarantees and asymptotically optimal auditing rates. By combining inverse probability weighting with online isotonic calibration, the authors develop methods that significantly reduce human oversight costs while maintaining strict reliability bounds, offering a practical and theoretically sound solution to the challenge of monitoring autonomous AI systems at scale.
We give an algorithm for computing an $\eps$-approximate fixed point of any $d$-dimensional nonexpansion with respect to the $\ell_\infty$ norm in time $\log(1/\eps)^{O(\log^2 d)}$. At its core, our method uses a boosting argument to geometrically shrink a search region. Among other consequences, our algorithm implies that simple stochastic games can be solved in quasipolynomial time.
Primary: Unknown
All Institutions: Unknown
The paper presents a quasipolynomial-time algorithm for computing approximate fixed points of $\ell_\infty$-contractions, resolving a major open problem in the complexity of simple stochastic games. By introducing a novel boosting-based geometric search strategy, the authors improve the running time from subexponential to quasipolynomial, providing a significant theoretical advance with broad implications for game theory, verification, and optimization.
The paper introduces a deterministic algorithm for computing $\epsilon$-approximate fixed points of $\ell_\infty$-contractions in quasipolynomial time, specifically $\log(1/\epsilon)^{O(\log^2 d)}$. The core innovation is a "boosting" argument applied to a geometric search problem. The authors define a "thin labeling" subroutine that recursively splits dimensions based on median side lengths to identify which faces of a search box can be safely shrunk. They then amplify the accuracy of these thin labelings using a technique analogous to AdaBoost, treating the labeling problem as an extremal amplification game. This approach cleverly reduces the subexponential dependence on dimension $d$ found in prior work (Feodorov and Haslebacher) to a quasipolynomial dependence. The theoretical framework is rigorous, with detailed proofs of the potential functions and correctness of the boosting steps.
As a purely theoretical computer science and optimization paper, there are no empirical experiments, datasets, or benchmark comparisons. The "evaluation" is entirely analytical, proving tight bounds on the number of arithmetic operations and function evaluations required. The significance is derived from the theoretical breakthrough in complexity bounds rather than empirical performance.
The algorithm is described in full pseudocode (Algorithms 1, 2, 3, 4). The theoretical guarantees are proven rigorously. However, without code or empirical benchmarks, "reproducibility" in the practical sense is limited to the ability of other theorists to verify the proofs. The unit-cost real machine model is used for analysis, which is standard in this field but limits direct practical implementation without bit-complexity analysis (which is noted as straightforward but not fully detailed).
The algorithm is deterministic and applies to $\ell_\infty$ norms. While it solves Simple Stochastic Games (SSGs) in quasipolynomial time, it does not resolve the open question of whether SSGs are in P. The quasipolynomial bound, while a significant improvement over subexponential, is still super-polynomial, meaning it may not be practical for very large dimensions or high-precision requirements. The paper focuses on the worst-case complexity; smoothed analysis or average-case performance is not addressed.
This result has significant implications for the complexity of several fundamental problems in game theory and verification, including Simple Stochastic Games, stochastic parity games, mean-payoff games, and modal $\mu$-calculus model checking. By providing a quasipolynomial-time algorithm for these problems, it advances the state of the art in understanding the computational hardness of these decision problems. It also contributes to the broader field of fixed-point computation, which is central to reinforcement learning (value iteration) and distributed computing. The use of boosting for geometric search is a novel methodological contribution that could inspire further research in algorithmic game theory. The paper presents a quasipolynomial-time algorithm for computing approximate fixed points of $\ell_\infty$-contractions, resolving a major open problem in the complexity of simple stochastic games. By introducing a novel boosting-based geometric search strategy, the authors improve the running time from subexponential to quasipolynomial, providing a significant theoretical advance with broad implications for game theory, verification, and optimization.
Diffusion and flow models provide expressive policy classes for robotic and sequential decision-making systems. However, their action likelihoods are generally intractable, limiting the direct use of widely adopted policy optimization methods such as PPO and GRPO. We propose Doob Policy Optimization (DoobPO), a framework that leverages Doob's $h$-transform for likelihood-free reinforcement learning fine-tuning of generative policies. DoobPO separates policy improvement from generative policy learning. It first learns a lightweight ratio model that represents the desired policy update, and then distills the resulting ratio-tilted target policy into the generative policy via conditional drift matching. This formulation applies uniformly to both diffusion and flow policies and supports a broad family of policy improvement rules, including PPO and GRPO. We prove that the practical conditional drift matching objective is gradient-equivalent to the ideal Doob drift matching objective. We further show that the induced diffusion process solves an endpoint-constrained stochastic optimal control problem. This establishes that it achieves the minimum-energy modification required to realize the desired policy tilt. Across MuJoCo continuous-control benchmarks, DoobPO achieves strong performance against competitive diffusion- and flow-policy baselines, showing a principled and practical route to RL fine-tuning for generative policies.
Primary: Northwestern University
All Institutions: Northwestern University
DoobPO introduces a likelihood-free reinforcement learning framework using Doob's $h$-transform to enable policy optimization for diffusion and flow-based policies. The paper makes a substantial technical contribution by unifying policy improvement and generative learning through conditional drift matching, supported by rigorous theoretical proofs and strong empirical results on MuJoCo benchmarks, positioning it as a significant advancement in the integration of generative models with reinforcement learning.
The paper proposes Doob Policy Optimization (DoobPO), a framework addressing the critical bottleneck of intractable likelihoods in diffusion and flow-based policies for reinforcement learning. By leveraging Doob's $h$-transform, the method decouples policy improvement from generative policy learning. It introduces a lightweight ratio model to represent policy updates and distills the resulting ratio-tilted target policy into the generative policy via conditional drift matching. The authors provide a rigorous theoretical foundation, proving that the practical conditional drift matching objective is gradient-equivalent to the ideal Doob drift matching objective and demonstrating that the induced diffusion process solves an endpoint-constrained stochastic optimal control problem. This establishes a principled, minimum-energy modification approach to realizing desired policy tilts, offering a unified formulation for both diffusion and flow policies.
The experimental evaluation is conducted on MuJoCo continuous-control benchmarks, comparing DoobPO against competitive diffusion- and flow-policy baselines. The results demonstrate strong performance, validating the practical utility of the proposed framework for RL fine-tuning of generative policies. While the benchmarks are standard in the field, the paper focuses primarily on continuous control, which may limit the generalizability of the findings to discrete or high-dimensional visual tasks without further evidence.
The paper is accepted at NeurIPS 2026, a venue with high standards for reproducibility. A project page is provided, which likely contains code and additional details. The theoretical proofs and clear separation of the ratio model and drift matching components suggest that the method is well-defined and implementable. However, without access to the full code repository details in the provided text, specific hyperparameter choices and implementation nuances remain partially opaque.
The primary limitation is the focus on MuJoCo continuous-control tasks, which may not fully capture the challenges of applying DoobPO to more complex, high-dimensional, or discrete action spaces. Additionally, the reliance on a lightweight ratio model introduces an approximation that could affect performance in highly stochastic or non-stationary environments. The computational overhead of maintaining and updating the ratio model alongside the generative policy is not extensively discussed in the provided abstract.
This work has significant potential to bridge the gap between expressive generative models and practical reinforcement learning. By enabling the use of standard policy optimization methods like PPO and GRPO with diffusion and flow policies, DoobPO could accelerate the adoption of generative models in robotics and sequential decision-making. The theoretical insights into stochastic optimal control and drift matching may also inspire new approaches in related fields such as optimal transport and control theory. DoobPO introduces a likelihood-free reinforcement learning framework using Doob's $h$-transform to enable policy optimization for diffusion and flow-based policies. The paper makes a substantial technical contribution by unifying policy improvement and generative learning through conditional drift matching, supported by rigorous theoretical proofs and strong empirical results on MuJoCo benchmarks, positioning it as a significant advancement in the integration of generative models with reinforcement learning.
Objects move, cameras pan, and weather drifts with the wind: much of what changes in sequential data is motion. Motion changes where things are, not what they are, and is therefore a symmetry of the data. Yet standard recurrent networks are not equivariant to it: their memory stays put while the input moves on. Flow equivariant recurrent networks first brought this symmetry into sequence models, for motions of constant velocity, or flows, by enumerating a bank of velocities. We extend this idea in three directions: from simple recurrent networks to convolutional LSTMs, from flows to motions whose velocity changes in time, and from enumerating velocities to estimating them. Treating a sequence as a field over time, we show that motion becomes a gauge symmetry: a global symmetry made local in time. Built on this view, our motion equivariant convolutional LSTM (MELSTM) uses its velocity model as the gauge field and is provably equivariant to every motion. Across future-frame prediction, classification of digits visible only through their motion, and action recognition under camera shake, MELSTM outperforms a convolutional LSTM and its flow equivariant counterpart with the same number of parameters: it is more accurate, generalizes to longer horizons and to unseen motions, and carries a fraction of the flow equivariant model's state.
Primary: The Kempner Institute at Harvard University
All Institutions: The Kempner Institute at Harvard University, South Dakota State University
The paper proposes a Motion Equivariant Convolutional LSTM that treats motion as a gauge symmetry, enabling equivariance to time-varying velocities. By using a velocity model to transport hidden states, the method outperforms flow-equivariant baselines in accuracy and generalization, offering a robust framework for sequential data with dynamic symmetries.
The paper introduces a rigorous theoretical framework extending equivariance from static spatial symmetries to dynamic, time-dependent motions. By framing motion as a gauge symmetry over the time axis, the authors derive a "Motion Equivariant Convolutional LSTM" (MELSTM). The core innovation is the use of a velocity model (acting as a gauge connection) to transport hidden and cell states between timesteps, ensuring equivariance to arbitrary motions rather than just constant-velocity flows. The mathematical derivation is sound, proving that standard ConvLSTMs are not flow-equivariant for generic kernels (measure-zero set) and establishing the necessary transformation laws for the velocity model to maintain equivariance. The extension to local motions via slot tracking is a clever architectural choice that addresses the correspondence problem in multi-object scenes.
The experiments are well-designed to isolate the impact of the transport mechanism. Comparisons are made against a standard ConvLSTM and a Flow Equivariant ConvLSTM (FERNN) with identical parameter counts. The tasks (Moving MNIST prediction, Motion-Only MNIST classification, KTH action recognition under camera shake) effectively test the model's ability to handle time-varying velocities. The results show that MELSTM outperforms baselines, particularly in generalization to unseen motions and longer prediction horizons. The finding that MELSTM is less sensitive to recent velocity changes than the enumerating FERNN is a significant empirical insight.
The code is available on GitHub. The paper provides detailed descriptions of the velocity estimation methods (phase correlation) and the architectural modifications. The use of parameter-free velocity models in the main experiments simplifies reproduction, though the paper notes that trainable velocity models are a future direction. The experimental setup is clearly defined, allowing for fair comparison.
The primary limitation is the reliance on parameter-free velocity estimation (phase correlation) in the main experiments, which may not generalize to all types of motion or complex scenes without learning. The model assumes a specific structure for local motions (slots), which may not capture all possible scene decompositions. The theoretical framework is complex and may be difficult to implement for non-abelian groups or more complex Lie algebras.
This work bridges the gap between geometric deep learning and sequential modeling, offering a principled way to incorporate dynamic symmetries into recurrent architectures. It has potential applications in video prediction, robotics, and any domain where objects move with time-varying velocities. The gauge theory perspective provides a new lens for understanding equivariance in time-series data, which could inspire further research in this area. The paper proposes a Motion Equivariant Convolutional LSTM that treats motion as a gauge symmetry, enabling equivariance to time-varying velocities. By using a velocity model to transport hidden states, the method outperforms flow-equivariant baselines in accuracy and generalization, offering a robust framework for sequential data with dynamic symmetries.
Program optimization aims to improve performance while preserving behavior. Coding agents can already make programs faster by changing algorithms, data structures, and execution strategies. However, existing equivalence checks mostly rely on tests, which agents have been shown to exploit, passing the tests without preserving the intended behavior. We propose agentic verified optimization, in which an agent produces a faster implementation together with a machine-checked proof of equivalence to the existing one. Because the existing implementation serves as the specification, the task avoids the specification-writing bottleneck of verified code generation, and recent advances in theorem proving make the proofs attainable. We formulate the task at the function and repository levels in Lean 4, where programs can be both executed and reasoned about. Few real-world applications are written in Lean, so we develop Aqueduct, an agentic framework that migrates C, C++, Go, and Rust programs into native Lean, validates each port by differential testing against its original. On the 189 reference implementations in Verina, agents produce faster proved rewrites for 156, including algorithmic changes such as replacing exponential recursion with dynamic programming. On 17 migrated applications, their 92 proved rewrites speed up 47 of 51 application scenarios. These application-level gains are smaller than the function-level ones, making the choice of functions to optimize an important problem. Current agents can thus already carry out verified optimization on both standalone functions and applications migrated from real programs. We discuss the remaining challenges and the steps toward verified optimization in production.
Primary: University of California
All Institutions: University of California, Amazon Web Services
The paper proposes agentic verified optimization, where LLM agents generate faster code with machine-checked proofs of equivalence, and demonstrates its feasibility on real-world software migrated to Lean 4. By using the existing code as the specification, it avoids the specification-writing bottleneck and prevents reward hacking, showing that current agents can perform verified optimizations that are both correct and faster, albeit with modest application-level gains.
The paper introduces "agentic verified optimization," a framework where LLM agents generate faster code implementations alongside machine-checked proofs of equivalence in Lean 4. The methodology is robust, leveraging the existing implementation as the specification to avoid the bottleneck of writing formal specs. It addresses the critical issue of reward hacking in test-based optimization by using formal proofs. The development of "Aqueduct," a framework for migrating C/C++/Go/Rust code to Lean with differential testing, is a significant technical contribution that enables the evaluation of this method on real-world software, not just toy problems.
The evaluation is extensive, covering 189 standalone functions from the Verina benchmark and 17 migrated real-world applications. The results show that agents can produce verified speedups in a majority of cases (82.5% for functions). The distinction between function-level microbenchmarks and application-level macrobenchmarks is well-handled, showing that while function-level gains are substantial, application-level gains are more modest due to I/O and unoptimized parts of the code. The comparison against a single-agent migration baseline for Aqueduct demonstrates the effectiveness of the multi-agent migration strategy.
The paper provides detailed descriptions of the agent setup (Codex with GPT-5.6 Sol), the Lean version, and the timing protocols. The use of deterministic generators for workloads and the open-source nature of Lean and the benchmarks (Verina, ProgramBench) suggests high reproducibility. However, the specific prompts and agent orchestration logic for Aqueduct would need to be released to fully replicate the migration results.
The primary limitation is the reliance on Lean 4, which has a small ecosystem of real-world applications, necessitating the migration step. The migration process itself is complex and may not capture all nuances of the original programs. Additionally, the speedups, while verified, are often modest at the application level (geometric mean 1.12x), limiting immediate practical impact for production systems where larger gains are typically expected.
This work bridges the gap between LLM-based code generation and formal verification, offering a path to trustworthy automated optimization. It has implications for software engineering, formal methods, and AI safety by providing a mechanism to ensure that AI-generated optimizations do not introduce subtle bugs. The framework could be extended to other proof assistants and languages, potentially transforming how performance-critical code is developed and maintained. The paper proposes agentic verified optimization, where LLM agents generate faster code with machine-checked proofs of equivalence, and demonstrates its feasibility on real-world software migrated to Lean 4. By using the existing code as the specification, it avoids the specification-writing bottleneck and prevents reward hacking, showing that current agents can perform verified optimizations that are both correct and faster, albeit with modest application-level gains.
We introduce Foundation4D, a unified feed-forward model for geometry estimation, 3D detection, semantic occupancy, and Gaussian reconstruction from multi-camera driving video. A shared backbone uses factorized spatiotemporal attention to efficiently combine observations across cameras and time. Our video inference pipeline uses pose graph optimization to refine camera trajectories and assembles object tracks across input windows. We also develop a data annotation pipeline and release PAI-Foundation4D, a benchmark with dense depth and semantic occupancy labels for 1,253 driving clips. With benchmark-specific fine-tuning, Foundation4D achieves the lowest depth AbsRel on Waymo and nuScenes, the highest detection mAP on nuScenes, and the highest occupancy mIoU on OpenOccupancy among the compared methods. Our task-combination study shows benefits for both detection and geometry: co-training detection with geometry and occupancy improves top-class mAP by 1.2 points over detection-only training, while co-training geometry with detection reduces camera-to-rig translation error by $13.4\%$ relative to geometry-only training. We further use the predicted geometry and trajectories as priors for lidarless NuRec, supporting reconstruction of driving scenes for simulation.
Primary: Waymo
All Institutions: Waymo
Foundation4D introduces a unified feed-forward model for 4D scene understanding in autonomous driving, achieving state-of-the-art results in geometry, detection, and reconstruction. The paper makes a strong technical contribution through its factorized spatiotemporal attention mechanism and joint training strategy, supported by the release of a new benchmark and rigorous empirical validation of multi-task learning benefits.
The paper proposes Foundation4D, a unified feed-forward model that jointly estimates geometry (depth), 3D detection, semantic occupancy, and Gaussian splatting reconstruction from multi-camera driving video. The core architectural contribution is a shared backbone utilizing factorized spatiotemporal attention to efficiently fuse observations across multiple camera views and temporal steps. This design allows the model to leverage temporal consistency for static scenes while handling dynamic objects. The method also introduces a video inference pipeline that employs pose graph optimization to refine camera trajectories and assembles object tracks across input windows, addressing the challenge of maintaining global consistency in long sequences. The joint training strategy is a key methodological strength, demonstrating that co-training detection with geometry and occupancy yields mutual benefits, improving both detection accuracy and geometric estimation precision.
The authors introduce PAI-Foundation4D, a new benchmark with dense depth and semantic occupancy labels for 1,253 driving clips, which is a significant contribution to the field. Experimental results show that Foundation4D achieves state-of-the-art performance on Waymo and nuScenes for depth estimation (lowest AbsRel), nuScenes for 3D detection (highest mAP), and OpenOccupancy for occupancy (highest mIoU). The ablation studies on task combination are particularly compelling, showing that co-training improves top-class mAP by 1.2 points and reduces camera-to-rig translation error by 13.4% compared to single-task baselines. The application to lidarless NuRec demonstrates the practical utility of the predicted geometry and trajectories for scene reconstruction in simulation.
The paper releases the PAI-Foundation4D benchmark, which enhances reproducibility for the evaluation metrics. However, the full model architecture and training code are not explicitly stated to be released in the provided text, though the detailed description of the factorized attention and pose graph optimization provides a clear roadmap for implementation. The reliance on specific proprietary datasets (Waymo) may limit immediate reproducibility for external groups without access to those resources.
The model is evaluated primarily on driving datasets, and its generalizability to other domains (e.g., robotics, indoor navigation) is not explored. The computational cost of the factorized spatiotemporal attention and pose graph optimization during inference may be high, potentially limiting real-time deployment on edge devices. The paper does not extensively discuss failure cases or robustness to extreme weather conditions or occlusions beyond standard benchmark evaluations.
This work has significant implications for the development of end-to-end autonomous driving systems by unifying perception and reconstruction tasks. The release of a new benchmark with dense labels will likely accelerate research in 3D scene understanding for autonomous vehicles. The demonstrated benefits of multi-task learning for geometry and detection provide valuable insights for practitioners designing unified perception stacks. The application to simulation via Gaussian reconstruction supports the growing trend of using neural rendering for training and testing autonomous driving algorithms. Foundation4D introduces a unified feed-forward model for 4D scene understanding in autonomous driving, achieving state-of-the-art results in geometry, detection, and reconstruction. The paper makes a strong technical contribution through its factorized spatiotemporal attention mechanism and joint training strategy, supported by the release of a new benchmark and rigorous empirical validation of multi-task learning benefits.
Amodal video segmentation asks for the complete mask of an object in every frame, including the parts hidden behind other objects. The best current methods generate these masks with fine-tuned video diffusion models, which need many denoising steps and an extra depth network, and give a different answer for every sample. Self-supervised video world models of the JEPA family are trained on a closely related task: predicting the representation of masked space-time regions from the visible ones. We therefore treat occlusion as masking. The tokens that the object's visible mask does not cover are dropped, the pre-trained V-JEPA 2.1 predictor fills them in, and a light head decodes the observed and predicted features into amodal masks. The resulting model, OccJEPA (Occlusion-as-masking JEPA), is deterministic and 33.5x faster than the diffusion-based state of the art, and more accurate on SAIL-VOS, on TAO-Amodal, including a held-out test split (+4.00 AP50) and visible masks tracked by SAM2, and on Kubric. Controlled experiments trace the gain to the predictor's pre-trained weights rather than to its capacity, for world models from 87M to 1.85B parameters. Code is available at https://github.com/zgp110/occjepa.
Primary: University of Wisconsin--Madison
All Institutions: University of Wisconsin--Madison
[One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper introduces OccJEPA, a deterministic and highly efficient method for amodal video segmentation that leverages pre-trained JEPA world models by treating occlusion as masking, achieving state-of-the-art accuracy with a 33.5x speedup over diffusion-based approaches.
The paper proposes OccJEPA, a method for amodal video segmentation that reframes occlusion as a masking problem solvable by pre-trained self-supervised video world models (specifically V-JEPA 2.1). Instead of using computationally expensive diffusion models with depth estimation, the method drops tokens corresponding to visible object masks and uses the JEPA predictor to fill in the latent representations of the occluded regions. A lightweight head then decodes these features into amodal masks. The core insight is that the predictive capability of JEPA models, trained to reconstruct masked spatiotemporal regions, naturally aligns with the task of inferring hidden object parts. The approach is deterministic, avoiding the stochasticity of diffusion sampling, and leverages the robustness of pre-trained weights rather than fine-tuning heavy generative models.
The experimental section demonstrates significant improvements over state-of-the-art diffusion-based methods. The model achieves a 33.5x speedup while maintaining or improving accuracy on SAIL-VOS, TAO-Amodal (including a held-out test split with +4.00 AP50), and Kubric. Ablation studies confirm that the performance gain stems from the pre-trained weights of the JEPA predictor rather than its architectural capacity, showing consistent results across model sizes from 87M to 1.85B parameters. The use of SAM2 for visible mask tracking ensures a fair comparison with other methods that rely on similar initialization.
The authors provide a public GitHub repository, which enhances reproducibility. The reliance on standard pre-trained models (V-JEPA 2.1) and established datasets (SAIL-VOS, TAO, Kubric) further supports reproducibility. However, the specific details of the "light head" architecture and training hyperparameters would need to be verified in the code repository for full replication.
The method relies on the quality of the visible masks provided by SAM2; errors in visible segmentation may propagate to the amodal prediction. The approach is tested primarily on standard benchmarks and may not generalize to highly dynamic or complex occlusion scenarios not covered by the training data of the pre-trained JEPA model. The deterministic nature, while a strength for consistency, may limit the exploration of multiple plausible amodal configurations in ambiguous cases.
This work bridges the gap between self-supervised representation learning and dense prediction tasks, suggesting that world models can be effectively repurposed for segmentation without generative overhead. The significant speedup makes amodal segmentation more practical for real-time applications. The finding that pre-trained weights are more critical than capacity for this task has implications for the design of future video understanding models. [One sentence main contribution]. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper introduces OccJEPA, a deterministic and highly efficient method for amodal video segmentation that leverages pre-trained JEPA world models by treating occlusion as masking, achieving state-of-the-art accuracy with a 33.5x speedup over diffusion-based approaches.
Preference optimization can improve every ranking metric while building a hidden, shared representation shift that the relative loss barely sees, one that can eventually destroy generation. We trace this failure to the common-mode offset, a shared directional shift in the model's hidden states that appears across token positions, prompts, and both chosen and rejected responses. Because it moves chosen and rejected representations together, the relative ranking loss barely reacts to its accumulation. As this offset grows, it projects onto the output layer to form a vocabulary well, elevating a single token's probability until it dominates decoding and causes generation collapse. The isolated offset alone reconstructs the vocabulary well, injecting the learned offset into an untreated base model recreates the collapse where norm-matched random directions do not, and masking this common component from the loss prevents the well in our flagship setting. Under the per-token ranking loss we study, the same collapse appears in 14 of 17 base-model lineages and under full-parameter fine-tuning. Moreover, the offset is not merely an artifact of collapse, since it is already present in textbook DPO and SimPO runs that form no well. Textbook DPO keeps it weakly aligned with any single vocabulary direction even with three times our base budget. Trained from base models for that budget, however, full-parameter SimPO drives it into collapse on both Llama-2 and Llama-3.1, and the same kind of offset grows, without a well, from a public SFT checkpoint under the official SimPO recipe. Across 17 UltraFeedback runs, the better a run ranks held-out responses, the less often two public reward models prefer its samples to the base model's. These findings expose a blind spot in standard ranking metrics and introduce a representation-level signal that can warn of generation collapse before it occurs.
Primary: The University of Tokyo
All Institutions: The University of Tokyo, TIER IV, Inc.
The paper identifies a hidden "common-mode offset" in preference optimization that causes generation collapse despite improved ranking metrics, providing a causal explanation and effective mitigation strategy. This is a significant contribution to the understanding of LLM alignment failures, offering both a diagnostic signal and a practical solution that is likely to influence future alignment research and practice.
The paper introduces a novel diagnostic framework for preference optimization failures, identifying a "common-mode offset" in the final hidden states as the root cause of vocabulary collapse. The methodology is rigorous, combining theoretical analysis of loss invariances with extensive empirical validation. The authors define the offset as the mean displacement of hidden states between trained and base models, demonstrating that this shared vector moves chosen and rejected representations in tandem, thereby evading the relative ranking loss. They propose a "dose" metric to quantify the risk of collapse and introduce a mitigation strategy involving centering the rejected-side loss and applying norm/loudness penalties to the offset. The causal interventions (injecting the offset into base models, masking it from the loss) are well-designed and effectively isolate the mechanism.
The experimental scope is impressive, covering 17 base-model lineages (0.5B to 14B parameters) across five tokenizer families, using both LoRA and full-parameter fine-tuning. The use of two distinct datasets (driving preferences and UltraFeedback) and multiple objectives (DPO, SimPO, Plackett-Luce ranking) strengthens the generalizability of the findings. The discovery that ranking metrics improve while generation collapses (anticorrelation with external reward models) is a significant empirical contribution. The cross-family validation of the mitigation strategy up to three times the base budget is convincing.
The paper provides detailed descriptions of training configurations, measurement protocols, and penalty terms. The authors state that code and data will be released upon publication. The specific hyperparameters (learning rates, batch sizes, LoRA ranks) are clearly defined. The measurement of "dose" and "loudness" is well-defined, allowing for replication. The reliance on specific public checkpoints (e.g., Llama-3-8B SFT from SimPO authors) aids reproducibility.
The mitigation strategy is calibrated on specific datasets and may require tuning for other domains. The analysis is primarily focused on the final hidden state, and the interaction with earlier layers is not deeply explored. The "diffuse degradation" observed in SimPO is noted but not fully resolved by the proposed offset control, indicating a second failure mode that remains unaddressed. The study is limited to English-centric models and datasets.
This work has high impact for the LLM alignment community by exposing a critical blind spot in standard evaluation metrics. It provides a practical diagnostic tool (monitoring the common-mode offset) that can be integrated into training pipelines to prevent generation collapse. The findings challenge the assumption that improving ranking metrics guarantees healthy generation, prompting a re-evaluation of alignment objectives. The proposed penalties offer a simple, effective remedy for a widespread issue. The paper identifies a hidden "common-mode offset" in preference optimization that causes generation collapse despite improved ranking metrics, providing a causal explanation and effective mitigation strategy. This is a significant contribution to the understanding of LLM alignment failures, offering both a diagnostic signal and a practical solution that is likely to influence future alignment research and practice.
Autonomous agents can increasingly propose ideas, modify code, and run experiments, creating an opportunity to accelerate scientific discovery using teams of collaborative AI researchers. To escape local minima in the discovery, we propose to encourage orthogonal exploration by constructing the team using agents with varying expertise. However, realizing this design requires more than diverse backgrounds. When experiments are costly, a team can test only a limited number of ideas, thereby creating the need to balance exploration and exploitation when selecting a subset of researchers to propose and execute new experiments. Furthermore, each result should shape what the team tries next. We therefore ask how AI researchers with different scientific backgrounds can build on one another's findings and sustain progress under a limited experimental budget. To address these challenges, we build a team of persistent AI researchers, each grounded in a different discipline, who keep private research histories, share experimental evidence, and receive research opportunities through adaptive allocation. We demonstrate the effectiveness of this design by deploying the team to search for linear attention architectures, where it outperforms GDN2 at 1.3B scale on language modeling, common-sense reasoning, and synthetic and real-world retrieval tasks. In controlled comparisons, removing any single organizational component from our design yields markedly weaker discoveries: without disciplinary backgrounds, the search plateaus early; without private histories, researchers make less progress despite identical shared evidence; and uniform selection falls short of adaptive allocation within the same budget. These results suggest that sustained autonomous discovery depends not only on stronger agents, but also on who explores, what they remember, and how research effort is allocated.
Primary: University of California, Santa Barbara (UCSB)
All Institutions: University of California, Santa Barbara (UCSB)
The paper introduces a multi-agent team structure with diverse disciplinary backgrounds, private memories, and adaptive budget allocation to autonomously discover linear attention architectures. By demonstrating that this organizational design outperforms ablated versions and achieves competitive results against state-of-the-art linear attention models at 1.3B scale, it provides a rigorous framework for collaborative AI research that balances exploration and exploitation, offering a scalable path toward autonomous scientific discovery in machine learning architecture design.
The paper proposes a multi-agent framework for autonomous architecture search, specifically targeting linear attention mechanisms. The core methodological contribution is the "organizational" structure of the agents: (1) Diverse disciplinary grounding to encourage orthogonal exploration, (2) Private research histories to maintain individual context, and (3) Adaptive allocation of experimental budget based on past performance. This moves beyond simple prompt engineering or single-agent loops by introducing a team dynamic that balances exploration and exploitation. The approach is conceptually sound and addresses a known limitation in LLM-based research agents (local minima due to homogeneous reasoning patterns).
The experiments are rigorous in scope, scaling from 150M to 1.3B parameters. The claim that the discovered architecture outperforms GDN2 (Gated DeltaNet) at 1.3B scale on language modeling, reasoning, and retrieval is significant if true. The ablation studies are critical to the paper's validity: removing disciplinary backgrounds, private histories, or adaptive allocation all lead to worse performance. This isolates the contribution of the team structure from the underlying LLM capabilities. However, the baseline comparison is limited to GDN2; comparisons with other recent linear attention variants (e.g., Mamba, RWKV, or standard Transformers at the same FLOP/param budget) would strengthen the claim.
The paper includes a detailed reproducibility statement, promising appendices with model configurations, hyperparameters, and agent trajectories. The use of FLOP-matched budget allocation is a strong signal of methodological rigor. However, without access to the actual code and agent prompts, full verification is difficult. The reliance on "persistent" agents implies a specific state-management infrastructure that must be clearly documented.
The primary limitation is the high computational cost of the search process, which may limit adoption by smaller labs. Additionally, the generalizability of the "disciplinary background" approach to other domains (e.g., computer vision or reinforcement learning) is not tested. The paper focuses heavily on linear attention, and it is unclear if the team structure yields similar gains in other architectural spaces. There is also a risk of "reward hacking" where agents optimize for the specific evaluation metrics rather than true generalization.
This work contributes to the emerging field of AI-driven scientific discovery. If the organizational principles (diversity, memory, adaptive allocation) are validated, they could be applied to other automated research tasks, such as hyperparameter tuning, dataset curation, or even hypothesis generation in biology. The potential to accelerate the discovery of efficient attention mechanisms is high, given the current industry focus on reducing inference costs. The paper introduces a multi-agent team structure with diverse disciplinary backgrounds, private memories, and adaptive budget allocation to autonomously discover linear attention architectures. By demonstrating that this organizational design outperforms ablated versions and achieves competitive results against state-of-the-art linear attention models at 1.3B scale, it provides a rigorous framework for collaborative AI research that balances exploration and exploitation, offering a scalable path toward autonomous scientific discovery in machine learning architecture design.
Auditory attention decoding (AAD) seeks to identify the attended talker from brief noninvasive brain recordings by comparing neural responses with competing speech streams. Recent electroencephalography (EEG) decoders report nearly perfect accuracy from one second of data, yet we show that this performance fails to generalize to unseen speech because models exploit trial leakage, eye gaze, and acoustic cues instead of speech evoked neural activity. We introduce CoupleFlow, which decodes attention through temporal coupling between neural and acoustic embeddings. Its predictions remain at chance when the brain signal contains no informative coupling, preventing shortcuts based on speech content or talker identity. A conditional flow matching head implements a likelihood ratio decision rule based on pointwise mutual information between brain and speech. This rule is Bayes optimal when the recording depends only on the attended talker and removes preferences for speech that is likely a priori. We evaluate CoupleFlow with content disjoint splits recovered from the audio and null tests that distinguish genuine temporal coupling from trial recognition. Across nine EEG and magnetoencephalography (MEG) datasets processed through a common pipeline, spanning four languages, high density EEG recordings, listeners with hearing impairment, and a room with multiple talkers, CoupleFlow outperforms linear and deep baselines while all null tests remain at chance. Finally, we derive an evidence accumulation law that predicts accuracy across window lengths from a single parameter per dataset and predicts that parameter from the strength of neural speech tracking. This law identifies recording signal to noise ratio as a limit on decoding speed and reliability and estimates the window length required to reach a target accuracy.
Primary: The Ohio State University
All Institutions: The Ohio State University
CoupleFlow introduces a null-calibrated auditory attention decoder that eliminates data leakage shortcuts via time-locked coupling scores and flow-matching likelihood ratios, establishing a rigorous benchmark and theoretical framework for neural speech tracking.
The paper proposes CoupleFlow, a decoder for Auditory Attention Decoding (AAD) that relies on temporal coupling between neural and acoustic embeddings rather than direct classification or stimulus reconstruction. The core innovation is a "null-calibrated" design where the decision score is based on the Pearson correlation between centered neural and acoustic embeddings. This ensures that if the neural signal contains no time-locked information about the speech (e.g., random noise or misaligned data), the score is zero, preventing shortcuts based on acoustic cues or trial identity. Additionally, a conditional flow-matching head is used to estimate pointwise mutual information (PMI), providing a Bayes-optimal likelihood ratio decision rule under the assumption that the recording depends only on the attended talker. The method includes rigorous anti-shortcut training terms, such as an audio-only adversary and VICReg, to ensure the model learns genuine neural coupling.
The evaluation is exceptionally rigorous, addressing a major flaw in previous AAD literature: data leakage. The authors introduce content-disjoint splits, ensuring test speech is never heard in training, and perform null tests (across-trial and circular shifts) to verify that accuracy stems from neural coupling rather than trial recognition. Experiments span nine EEG and MEG datasets, including multi-talker rooms (MAESTRO) and hearing-impaired listeners. The paper demonstrates that previous high accuracies were largely due to leakage, and while CoupleFlow's absolute accuracy is lower than leaked baselines, it is significantly higher than leakage-free baselines. A key contribution is the derivation of an "evidence accumulation law" that predicts accuracy as a function of window length and signal-to-noise ratio, providing a theoretical framework for understanding decoding speed limits.
The paper provides extensive details on preprocessing, network architecture, and training protocols. It explicitly states that a generative AI assistant was used for coding and drafting, but emphasizes that all results are generated by scripts from logged runs and verified by the authors. The exclusion of datasets that fail time-alignment checks (like NJU) is well-documented. The use of a common pipeline across datasets enhances comparability. However, the specific code repository is not provided in the text, which slightly limits immediate reproducibility, though the detailed descriptions are sufficient for re-implementation.
The method assumes the recording depends primarily on the attended talker, which is an approximation since ignored speech is weakly tracked. The flow-matching head adds computational overhead. The evidence accumulation law relies on certain statistical assumptions (independence of Fisher transforms) that may not hold perfectly in all regimes. The paper acknowledges that MEG results are evaluated only on match-mismatch tasks due to dataset constraints.
This paper has significant impact on the field of Brain-Computer Interfaces and computational neuroscience. By exposing the prevalence of data leakage in AAD studies, it sets a new standard for evaluation rigor. The null-calibrated approach is a generalizable principle for any neural decoding task where shortcut learning is a concern. The evidence accumulation law provides a practical tool for designing neuro-steered hearing devices, allowing engineers to predict the required recording duration for target accuracy based on sensor quality. CoupleFlow introduces a null-calibrated auditory attention decoder that eliminates data leakage shortcuts via time-locked coupling scores and flow-matching likelihood ratios, establishing a rigorous benchmark and theoretical framework for neural speech tracking.
Humanoid mobility over complex terrain requires accurate foot placement and coordinated whole-body motion capable of robust adaptation. Reinforcement learning (RL) policies trained without motion references generalize well across terrains, yet eliciting effective whole-body coordination through reward design and exploration alone remains challenging. In contrast, reference-based methods yield natural motions but struggle to adapt beyond the predefined trajectories in their training data. We introduce a learning framework that bridges these two approaches by using motion data strictly as a training prior. An explicit, self-supervised, attention-based contact planner predicts future body-environment contacts, while a contact-conditioned tracker translates these plans into coordinated whole-body motion. Together, they establish contact as an interface between sensory observations and control actions. This abstraction allows the policy to internalize the contact-aware structure of locomotion rather than directly replicating kinematics. Consequently, a single policy can exploit data-driven motion priors for natural whole-body coordination while retaining the flexibility to discover environment-dependent behaviors through reinforcement learning. Building on the learned contact planner and tracker, a subsequent fine-tuning stage bootstraps their acquired capabilities to explore environment variations beyond those covered by the original motion data. Operating entirely from onboard observations without any reference inputs at deployment, this framework enables versatile humanoid mobility in challenging environments.
Primary: National University of Singapore
All Institutions: National University of Singapore, Robotic Systems Lab (ETH Zurich)
The paper introduces a contact-centric framework for humanoid mobility that uses attention-based planning to predict body-environment interactions, enabling a single policy to generalize across diverse terrains and skills without reference motions at deployment. This represents a significant step forward in humanoid control by decoupling contact planning from whole-body execution, allowing for more robust and adaptable behaviors in complex environments.
The paper proposes a novel framework for humanoid mobility that decouples the "what" (contact planning) from the "how" (whole-body tracking). The core innovation is the use of an attention-based mechanism to predict explicit contact locations on a terrain map, serving as an intermediate representation between perception and control. This is a significant conceptual shift from prior work that either relies on full-body trajectory tracking (which lacks generalization) or pure RL (which struggles with complex reward shaping for whole-body coordination). The three-stage training pipeline (Tracker Pretraining, Planner Supervision, RL Fine-tuning) is well-structured and logically sound, effectively leveraging motion data as a prior without binding the deployment policy to specific trajectories. The use of FiLM modulation for body-specific attention queries is a clever architectural choice that allows a single network to handle multiple contact points.
The experimental evaluation is robust and comprehensive. The authors conduct extensive real-world experiments on the Unitree G1 robot, demonstrating a wide variety of skills including ladder climbing, crawling, vaulting, and failure recovery. The simulation-based ablations are rigorous, isolating the contributions of the contact representation, the attention mechanism, and the RL fine-tuning stage. The comparison with MGMT (a strong baseline using diffusion-based motion generation) shows clear advantages in success rates and contact placement accuracy, particularly on sparse terrains. The qualitative comparisons with other methods (PHP, LLP) are fair and highlight the unique capabilities of the proposed approach. The inclusion of failure recovery experiments is a strong point, demonstrating the robustness of the learned policy.
The paper provides detailed descriptions of the network architectures, training procedures, and reward functions. The use of standard components (MLP trackers, attention planners, PD control) makes the method relatively accessible to reproduce. However, the specific details of the motion retargeting pipeline and the LiDAR mapping framework are described at a high level, which might make exact reproduction challenging without access to the code. The lack of a public code repository at the time of review (arXiv preprint) is a minor drawback, but the methodological details are sufficient for a skilled researcher to implement a similar system.
The paper acknowledges several limitations. The current perception is limited to 2.5D terrain maps, which restricts the system's ability to handle overhead contacts or complex 3D manipulation. The system is designed for local goal-conditioned locomotion and does not address global navigation. The contact representation cannot distinguish between motions with identical contact sequences but different kinematics (e.g., backflip vs. vertical jump). These are reasonable limitations for the current scope, but they highlight areas for future work.
This paper has significant potential impact on the field of humanoid robotics. By establishing contact as a general interface for mobility, it provides a scalable framework for adding new skills to humanoid robots without requiring exhaustive manual augmentation of motion datasets. The approach bridges the gap between data-driven imitation and reward-driven exploration, offering a path toward more versatile and robust humanoid control. The insights gained from this work could be applied to other robotic systems that require complex whole-body coordination, such as quadrupeds or manipulators. The paper introduces a contact-centric framework for humanoid mobility that uses attention-based planning to predict body-environment interactions, enabling a single policy to generalize across diverse terrains and skills without reference motions at deployment. This represents a significant step forward in humanoid control by decoupling contact planning from whole-body execution, allowing for more robust and adaptable behaviors in complex environments.
Multi-robot coordination of nonlinear systems is naturally formulated as joint-space optimal control subject to constraints over groups of robots. Existing methods that approach joint optimality remain largely centralized, offline, and confined to collision-free motion planning, while decentralized, real-time counterparts target safe navigation without pursuing the joint optimum. This paper develops a decentralized, real-time planner that approaches the joint optimum as an anytime process. We extend a recent single-robot framework that searches dispersive trees of forward-propagated commands for differentially flat systems to coordinate a multi-robot team near-optimally. We first show that the inter-robot coupling can be effectively replaced by some local constraints, under which locally grown dispersive trees contain a near-optimal agreement. It is this structure that the robots pursue by decentralized column generation, negotiating rents on the shared space that guide the search of each robot's tree. The negotiation settles on an agreement whose near-optimality is certified at quiescence. Our protocol, WarpWeaver, implements this negotiation as an asynchronous real-time loop over a receding horizon, and demonstrates effective closure to quiesced agreements even when real-time deadlines cut the negotiation short. In simulation, WarpWeaver reduces total travel time by up to 22% against centralized offline planners on the Dynobench suite and by up to 37% against a decentralized real-time baseline, and nearly doubles the success rate of a task-specific planner on graph connectivity. Hardware experiments on six robots planning onboard demonstrate real-world implementability.
Primary: Massachusetts Institute of Technology
All Institutions: Massachusetts Institute of Technology
WarpWeaver introduces a decentralized, real-time multi-robot coordination protocol that achieves near-optimal joint control by extending dispersive forward tree search via decentralized column generation, providing certified near-optimality at quiescence and demonstrating significant improvements over centralized and decentralized baselines in both simulation and hardware.
The paper proposes WarpWeaver, a decentralized, real-time multi-robot coordination protocol that leverages dispersive forward tree search (DFT) for differentially flat systems. The core innovation is the formulation of joint optimal control as a decentralized column generation problem. The authors prove that inter-robot coupling can be replaced by local constraints (corridors) such that locally grown trees contain a near-optimal agreement. They utilize the dual variables of a restricted master problem (an ILP relaxation) as "rents" on shared space, which guide the local tree search of each robot. This creates a negotiation loop where robots propose columns (trajectories) based on current prices, and the pool of columns is updated until quiescence or a real-time deadline. The theoretical contribution is significant, providing a certificate of near-optimality at quiescence relative to the joint optimum, which is a strong guarantee for decentralized methods that often lack such bounds. The extension from single-robot DFT to multi-robot coordination via column generation is a clever and rigorous application of operations research techniques to robotics.
The evaluation is comprehensive, covering simulation benchmarks (Dynobench) and hardware experiments. In simulation, WarpWeaver outperforms centralized offline planners in travel time (up to 22% reduction) and decentralized real-time baselines (up to 37% reduction). It also demonstrates superior success rates on connectivity maintenance tasks compared to task-specific planners. The hardware experiments on six robots planning onboard validate the real-time feasibility and robustness of the approach. The use of high-fidelity nonlinear quadrotor models and differentially flat systems (unicycles, integrators) shows the method's generality. The comparison against both centralized (offline) and decentralized (real-time) baselines is appropriate and highlights the trade-off between optimality and real-time performance.
The paper provides detailed algorithmic descriptions (Algorithm 1) and specifies search parameters, command sets, and dynamics for various platforms in Table 1. The use of standard benchmarks like Dynobench aids reproducibility. However, specific implementation details of the "dispersive tree" generation and the exact solver for the dual problem are not fully detailed in the provided text, though references to prior single-robot work are made. The code availability is not explicitly stated in the abstract or intro, but the level of detail suggests it is reproducible by experts in the field.
The method assumes differentially flat dynamics, which limits its applicability to systems that are not flat or where flatness is difficult to establish. The computational cost of growing dispersive trees, while polynomial, can still be high for large teams or long horizons, though the decentralized nature mitigates this. The "rents" mechanism relies on the structure of the ILP, which may not generalize easily to non-convex or highly complex joint constraints beyond the Lipschitz predicates assumed. The real-time performance depends on communication delays and computational resources, which may vary in real-world deployments.
This work bridges the gap between rigorous optimal control theory and practical real-time multi-robot coordination. It provides a framework for decentralized planning with optimality guarantees, which is valuable for applications requiring high precision and safety, such as drone swarms, autonomous vehicle fleets, and industrial automation. The use of column generation for coordination is a novel paradigm that could inspire further research in distributed optimization for robotics. WarpWeaver introduces a decentralized, real-time multi-robot coordination protocol that achieves near-optimal joint control by extending dispersive forward tree search via decentralized column generation, providing certified near-optimality at quiescence and demonstrating significant improvements over centralized and decentralized baselines in both simulation and hardware.
Human video scales easily and does not need robot hardware, but existing methods require depth sensing, overlay the robot onto 2D video, or generate data only for the demonstrated objects. We present Real2Sem2Real (R2Sem2R), a real-to-semantic-to-real pipeline that turns one monocular RGB video of a human demonstration into thousands of robot demonstrations across the semantic object instances of a category. R2Sem2R tracks the SE(3) pose of each object with DINOv3-distilled Gaussian splats, outperforming existing methods and FoundationPose with estimated metric depth. R2Sem2R then aligns digital cousins, each generated from one or a few photographs, to the demonstrated objects with a semantic alignment. This alignment matches whole objects and the functional regions of target objects, such as a coffee machine's tray. R2Sem2R maps the human hand to a feasible robot grasp that extends to the cousins. Robot demonstrations are generated either in IsaacSim, for photorealistic rendering, or MuJoCo, for contact physics, and R2Sem2R can generate over 2,000 episodes in about two hours. On two real robots and four tasks, we train a vision-language-action policy on data from one human video per task, and policies trained on our data match or exceed policies trained on teleoperation across diverse object instances. The project page is available at https://real2sem2real.github.io/.
Primary: Stanford University
All Institutions: Stanford University, ETH Zurich
The paper presents a novel and effective pipeline for generating semantically augmented robot demonstrations from a single human RGB video. By combining robust RGB-only object tracking with semantic alignment of digital cousins, it enables the transfer of functional knowledge across object instances, resulting in policies that match or exceed those trained on teleoperation with significantly less human effort.
The paper proposes a coherent "Real-to-Semantic-to-Real" pipeline that addresses a critical bottleneck in robot learning: the cost of collecting diverse, object-specific demonstrations. The core technical contribution is a robust RGB-only 4D object tracker using DINOv3-distilled Gaussian splats, which outperforms depth-dependent baselines like FoundationPose. The novelty lies in the "semantic alignment" of digital cousins (generated from few photos) to the demonstrated objects, allowing the transfer of functional affordances (e.g., where to place a mug on a specific machine) rather than just geometric similarity. The integration of hand-to-gripper retargeting and parallel simulation in IsaacSim/MuJoCo is well-engineered, though the reliance on VLMs for keyframe selection and region segmentation introduces potential fragility.
The experimental setup is rigorous, involving two real robots (xArm7, Panda) and four distinct tasks that test different aspects of the pipeline (placement, pouring, hanging, reorientation). The comparison against teleoperation is the gold standard, and the results showing R2Sem2R matching or exceeding teleoperation with significantly less human effort are compelling. The ablation study isolating the effect of semantic augmentation (R2Sem2R vs. R2R2R+) is particularly valuable, clearly demonstrating the added value of the cousin alignment. The tracking accuracy benchmarks on YCBInEOAT provide strong evidence for the superiority of the proposed tracker.
The paper provides detailed descriptions of the pipeline components, including loss functions, initialization strategies, and simulation settings. The use of standard tools (IsaacSim, MuJoCo, DINOv3) aids reproducibility. However, the specific implementation details of the "semantic ICP" and the VLM prompts used for region selection are not fully detailed in the main text, which could hinder exact replication. The code availability is implied by the project page, but the paper itself is a preprint/submission, so full code release status is pending.
The method is limited to rigid objects and requires a metric measurement of the demonstrated objects, which is a non-trivial assumption. The IsaacSim kinematic replay lacks contact dynamics, limiting applicability to contact-rich tasks, though MuJoCo is used as an alternative. The pipeline is computationally intensive (2.5 hours for 2000 episodes) and requires careful setup of the scene and camera calibration. The reliance on monocular depth estimation for some components (though mitigated by the splat tracker) remains a potential source of error.
This work has significant potential to democratize robot learning by reducing the barrier to entry for data collection. By enabling the generation of thousands of diverse demonstrations from a single human video, it could accelerate the development of general-purpose robot policies. The approach is applicable to a wide range of manipulation tasks and could be extended to other embodiments and object categories. The paper presents a novel and effective pipeline for generating semantically augmented robot demonstrations from a single human RGB video. By combining robust RGB-only object tracking with semantic alignment of digital cousins, it enables the transfer of functional knowledge across object instances, resulting in policies that match or exceed those trained on teleoperation with significantly less human effort.
Consensus is a fundamental problem in distributed computing and at the core of state machine replication (SMR). Despite decades of active human research, discovering practical low-latency solutions to the problem is still open, as it requires combining protocol knowledge, correctness reasoning, implementation, and experimental evaluation. We report on how AI helped us discover A-greement, a new consensus protocol for geo-replicated state machines that significantly outperforms state-of-the-art solutions. Averaged across six deployments with 5 or 9 replicas, A-greement reduces the mean latency of a geo-distributed key-value store over the fastest baseline by 12.2% and 33.3% on write-heavy and read-heavy workloads, respectively, and matches the fastest baseline on read-only workloads. A-greement combines lease-authorized local reads, direct client learning for leader-ordered writes, and a fast fixed-quorum per-key write path, selected according to network topology and workload. At the heart of our discovery lies AI-greement, an agent harness for the "autonomous" discovery, implementation, experimentation, and verification of candidate protocols. AI-greement integrates (1) a concept-oriented protocol knowledge base, (2) a modular implementation skeleton, and (3) a Lean-based verification framework with reusable safety definitions, reference specifications, and proofs. AI-greement is of independent interest: its Lean formalization uncovers several previously unreported vulnerabilities in existing consensus protocols, and it provides guidelines on how domain-specific harnesses can turn general-purpose coding agents into effective tools for (distributed) computing research. Yet we argue that human consensus experts remain essential before, during, and after the search: agents reshape the researcher's role rather than end consensus research.
Primary: École Polytechnique Fédérale de Lausanne (EPFL)
All Institutions: École Polytechnique Fédérale de Lausanne (EPFL)
The paper presents AI-greement, a specialized agent harness that leverages formal verification and domain knowledge to autonomously discover and verify a new consensus protocol, A-greement, which outperforms state-of-the-art solutions in geo-distributed settings. This work is significant as it demonstrates a novel application of AI agents in the design of complex distributed systems, moving beyond code generation to algorithmic discovery and formal verification, thereby offering a new paradigm for tackling hard problems in distributed computing.
The paper proposes "AI-greement," a specialized agent harness designed to autonomously discover, implement, experiment with, and verify distributed consensus protocols. The methodology integrates three key components: a concept-oriented protocol knowledge base, a modular implementation skeleton, and a Lean-based formal verification framework. This approach moves beyond simple code generation by embedding domain-specific constraints and formal safety definitions directly into the agent's workflow. The resulting protocol, A-greement, is a hybrid design combining lease-authorized local reads, direct client learning for leader-ordered writes, and fast fixed-quorum per-key write paths. The use of Lean for verification is a significant methodological strength, as it allows the agent to reason about correctness rather than just performance, leading to the discovery of previously unreported vulnerabilities in existing protocols.
The evaluation is rigorous, comparing A-greement against state-of-the-art baselines across six different geo-distributed deployments with 5 and 9 replicas. The results show a 12.2% reduction in mean latency for write-heavy workloads and a 33.3% reduction for read-heavy workloads compared to the fastest baseline. The paper also validates the safety of the discovered protocol using the Lean framework. The inclusion of real-world network topologies and varied workload characteristics strengthens the empirical claims.
The paper provides details on the agent harness architecture and the Lean verification framework, which are critical for reproducibility. However, as the full text provided is a skeleton with section headers, specific implementation details, prompt engineering strategies, and code repositories are not visible in the excerpt. The mention of a "modular implementation skeleton" suggests that code artifacts are likely available or planned, but without the actual code or detailed prompt logs, full reproducibility of the *discovery process* is difficult to assess from the abstract and headers alone.
The primary limitation is the reliance on a highly specialized, domain-specific harness (AI-greement) which may not generalize easily to other areas of distributed computing or software engineering without significant adaptation. Additionally, the "autonomous" nature of the discovery is tempered by the authors' admission that human experts remain essential, suggesting the current level of autonomy is limited to search space reduction and verification assistance rather than fully independent invention. The performance gains, while significant, are specific to geo-replicated state machines and may not apply to other consensus contexts.
This paper has high potential impact on the field of AI for Systems. It demonstrates that LLMs can be effectively used not just for coding, but for the *design* of complex distributed algorithms when coupled with formal verification tools. The discovery of vulnerabilities in existing protocols via the Lean framework highlights the utility of AI-assisted formal methods. This work could inspire similar "agent harnesses" for other hard problems in computer science where correctness and performance are tightly coupled. The paper presents AI-greement, a specialized agent harness that leverages formal verification and domain knowledge to autonomously discover and verify a new consensus protocol, A-greement, which outperforms state-of-the-art solutions in geo-distributed settings. This work is significant as it demonstrates a novel application of AI agents in the design of complex distributed systems, moving beyond code generation to algorithmic discovery and formal verification, thereby offering a new paradigm for tackling hard problems in distributed computing.