Last 7 Days (August 18 – August 24, 2026)
Most modern optimizers form their momentum as an exponential moving average (EMA) of past gradients, forgetting every direction at one fixed rate. However, the inputs a deep network sees during training can be highly anisotropic, with a few directions queried frequently while most are seen rarely. Recent methods address this anisotropy by wrapping extra processing around this buffer, leaving the momentum update itself unchanged. We propose DeltaMomentum, which builds direction-awareness into the momentum update rule. The main observation is that the gradient of a linear layer splits into an input that acts as a key and an output-side error that acts as a value. Exploiting the key-value structure, DeltaMomentum updates the momentum buffer by the canonical delta rule, so each direction is forgotten at a rate set by how often it appears. We prove that it is a valid momentum, that it applies the input-side curvature correction without matrix inversion, and that it clears stale directions faster than EMA under both a fixed and a drifting optimum. It is a drop-in replacement for the momentum buffer of any optimizer, its coefficient transfers across widths under $μ$P, and its extra compute stays between $22.2\%$ and $25.0\%$ of a gated-MLP block's linear cost with no persistent memory. In FineWeb-Edu pretraining, AdamW with DeltaMomentum (DeltaAdamW) reaches AdamW's validation loss in up to $46.39 \pm 4.32\%$ fewer steps at 67M and $22.12 \pm 0.80\%$ at 370M over three seeds, and the gain persists at 1B on a Chinchilla-optimal budget. A Muon baseline tuned under the same protocol sits above DeltaAdamW at both language-model scales, and the gain holds for SGD, ResNet-18, and ViT-Tiny on CIFAR-10. Training-time diagnostics confirm the predicted mechanism, better gradient tracking and healthier input directions.
Primary: University of Pennsylvania
All Institutions: University of Pennsylvania, Jane Street
DeltaMomentum introduces a theoretically grounded, direction-aware momentum update rule that leverages the key-value structure of gradients to achieve faster convergence and implicit preconditioning. The paper presents a compelling combination of rigorous theory, detailed mechanistic analysis, and strong empirical results across multiple scales and architectures, establishing a new baseline for momentum-based optimization that is likely to influence future optimizer design.
The paper proposes "DeltaMomentum," a novel momentum update rule that replaces the standard Exponential Moving Average (EMA) with a delta-rule update based on the key-value structure of rank-1 gradient factorization in linear layers. The core insight is that treating the momentum buffer as an associative memory allows for direction-aware forgetting rates, where directions queried frequently are updated more aggressively. The methodology includes a theoretical derivation showing that this update implicitly performs input-side preconditioning (similar to K-FAC but without matrix inversion) and provides guarantees on convergence and tracking of non-stationary targets. The approach is theoretically sound, well-motivated by the anisotropy of deep learning landscapes, and offers a clean, drop-in replacement for existing optimizers.
The experimental evaluation is comprehensive and convincing. The authors conduct controlled ablations against AdamW (the only baseline differing in the first moment) and Muon (a strong structured optimizer baseline) on Llama-2-style language models at 67M, 370M, and 1B parameters on FineWeb-Edu. The results show significant step-efficiency gains (up to 46% fewer steps) and competitive wall-clock efficiency. Additional experiments on CIFAR-10 with ResNet-18 and ViT-Tiny demonstrate generalizability. The inclusion of mechanistic diagnostics (gradient alignment, function-space prediction error, feature covariance conditioning) strongly supports the theoretical claims. The compute overhead is quantified and shown to be manageable.
The paper provides detailed algorithmic descriptions, theoretical proofs, and mentions specific implementation details (e.g., normalized keys, $\mu$P scaling). The experimental setup is described with sufficient detail for reproduction, including data sources (FineWeb-Edu) and model architectures. The authors provide a clear recipe for hyperparameter transfer, enhancing reproducibility.
The paper acknowledges limitations regarding scale (up to 1B parameters) and domain (language modeling only). It does not yet demonstrate performance on larger models (e.g., 7B+) or other domains like vision-only or multimodal tasks, though the CIFAR results suggest potential. The theoretical analysis relies on quasi-static approximations, which may not perfectly capture the highly non-stationary dynamics of deep training, although the empirical diagnostics support the theory.
This work has significant potential to improve the efficiency of training large-scale neural networks, reducing compute costs and energy consumption. By providing a drop-in replacement for momentum, it can be easily adopted by the community. The theoretical insights into the role of momentum as an associative memory could inspire further research into optimizer design. DeltaMomentum introduces a theoretically grounded, direction-aware momentum update rule that leverages the key-value structure of gradients to achieve faster convergence and implicit preconditioning. The paper presents a compelling combination of rigorous theory, detailed mechanistic analysis, and strong empirical results across multiple scales and architectures, establishing a new baseline for momentum-based optimization that is likely to influence future optimizer design.
We introduce Accelerating Dexterity via Pre-Training (ADEPT), a large-scale reinforcement learning (RL) framework for learning sim-to-real transferable dexterity across high degree-of-freedom (DoF) robot embodiments that can solve long-horizon tasks directly from raw visuo-tactile perception. ADEPT pretrains a dexterous policy on a generic object reposing task, then post-trains downstream policies with this pretrained behavior as a prior. ADEPT enables learning new behaviors that are otherwise difficult to discover from scratch on multi-fingered robots and avoids learning the same set of skills over again for every new downstream task. The pretrained policy zero-shots the reposing phase of downstream tasks, but naïve RL fine-tuning rapidly degrades this capability during transfer. We address this with a stable post-training recipe combining behavior-cloning distillation, critic warm-up, and conservative on-policy updates. To safely exploit the full kinematic dexterity, we introduce a joint-space Geometric Fabric that mediates between the RL policy and the robot. We distill post-trained teachers into perceptive students that zero-shot sim-to-real transfer on two embodiments: a 23 DoF Kuka-Allegro with two RGB cameras, and a 29 DoF Flexiv-Sharpa with two RGB cameras and five vision-based tactile sensors, and can solve long-horizon tasks from challenging initial states with dexterity at human-level speed.
Primary: University of Toronto
All Institutions: University of Toronto, Vector Institute, NVIDIA
The paper presents a significant advancement in dexterous manipulation via RL, introducing a robust pre-training and post-training framework that enables stable and efficient learning of complex manipulation skills. The combination of behavioral cloning distillation, critic warm-up, and conservative updates effectively mitigates the instability often associated with fine-tuning RL policies, while the Geometric Fabric ensures safe and feasible control. The successful sim-to-real transfer on multiple high-DoF platforms demonstrates the practical utility and generalizability of the approach, making it a valuable contribution to the field of robotics and reinforcement learning.
The paper introduces ADEPT, a framework for learning dexterous manipulation policies using Reinforcement Learning (RL). The core methodological contribution is a two-stage training pipeline: 1) Pre-training a policy on a generic "reposing" task to learn basic object handling and kinematic feasibility, and 2) Post-training (fine-tuning) on specific downstream tasks. Crucially, the authors address the issue of "catastrophic forgetting" or instability during fine-tuning by introducing a stable recipe combining behavior cloning (BC) distillation, critic warm-up, and conservative on-policy updates. They also introduce a "Geometric Fabric" in the joint space to mediate between the RL policy and the robot, ensuring safe and feasible control. The approach leverages large-scale simulation data and transfers to real-world embodiments (Kuka-Allegro and Flexiv-Sharpa). The methodology is sound and addresses a significant bottleneck in dexterous manipulation: the difficulty of discovering complex manipulation skills from scratch via RL.
The evaluation is extensive and rigorous. The authors demonstrate zero-shot sim-to-real transfer on two distinct high-DoF robotic platforms. They show that ADEPT enables the robots to solve long-horizon tasks (like object reposing and manipulation) from challenging initial states. The results indicate that the pre-training phase significantly accelerates learning and improves the stability of the post-training phase compared to training from scratch. The comparison against baselines (likely standard RL or BC-only approaches) highlights the effectiveness of the proposed post-training recipe. The ability to transfer to real hardware without further tuning is a strong empirical result.
The paper provides detailed descriptions of the simulation environment, the robot models, and the training hyperparameters. The code is made available on GitHub, and the project website includes demo videos. The use of standard simulators (Isaac Gym) and well-known robot models enhances reproducibility. The authors explicitly discuss the limitations and the specific conditions under which the method works, aiding in replication.
The method relies heavily on high-quality simulation-to-real transfer, which can still be sensitive to domain gaps not captured in simulation (e.g., friction variations, sensor noise). The "Geometric Fabric" adds a layer of complexity to the control stack. The pre-training on a generic task requires careful design to ensure the learned priors are useful for diverse downstream tasks. The computational cost of pre-training on large-scale RL data is significant.
This work contributes to the advancement of general-purpose robotic manipulation, a key step towards more autonomous and versatile robots in unstructured environments. By making dexterous manipulation more accessible via RL, it lowers the barrier for research and application in this domain. The open-source release of code and models promotes further research and development in the field. The paper presents a significant advancement in dexterous manipulation via RL, introducing a robust pre-training and post-training framework that enables stable and efficient learning of complex manipulation skills. The combination of behavioral cloning distillation, critic warm-up, and conservative updates effectively mitigates the instability often associated with fine-tuning RL policies, while the Geometric Fabric ensures safe and feasible control. The successful sim-to-real transfer on multiple high-DoF platforms demonstrates the practical utility and generalizability of the approach, making it a valuable contribution to the field of robotics and reinforcement learning.
Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy, outcome-rewarded RL over the candidate slate, cannot teach it, for a structural reason we identify and name selector credit starvation: under a broadcast, sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run's own training artifacts confirms all three properties, each worsening monotonically with horizon. SkillGate removes the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separate action-local advantage reaching exactly the skill-naming tokens, positive only when a trajectory's single read is the correct one. On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.
Primary: Unknown (Affiliations not explicitly listed in text, authors appear to be from various Chinese institutions based on names and "corrauth" markers)
All Institutions: Unknown
SkillGate introduces a novel disjoint credit channel architecture that solves "selector credit starvation," enabling 9B agents to reliably select skills from large slates, significantly outperforming outcome-only RL and demonstrating that selection is a trainable, distinct capability from execution.
The paper introduces "SkillGate," a method that addresses the specific problem of in-policy skill selection in long-horizon agentic tasks. The core technical contribution is the identification of "selector credit starvation," a structural flaw in standard outcome-rewarded RL where the tokens responsible for selecting a skill receive vanishingly small and often incorrectly signed gradients. The proposed solution partitions the token support into two disjoint credit channels: an execution channel using standard group-normalized outcome advantages, and a selection channel using an action-local advantage that rewards only the specific tokens naming the correct skill. This separation prevents the selection decision from being punished by execution failures and ensures the selection signal is not diluted by trajectory length. The methodology is theoretically sound, addressing a known bottleneck in agent training with a precise, mechanistic fix.
The authors evaluate SkillGate on five agentic benchmarks (Claw-Eval, SkillsBench, SETA, SWE, Terminal-Bench 2.0) using a 16-candidate slate. They demonstrate that SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, outperforming outcome-only RL, supervised selection, and preference learning baselines. Crucially, they show that SkillGate achieves this with fewer reads and less exposure to misleading skills, indicating genuine improvement in selection capability rather than just increased tool usage. The ablation studies effectively isolate the contribution of the selection channel, showing that coarser credit assignment (group or trajectory level) fails to improve selection. The comparison with frontier models highlights that skill selection is a distinct capability not solved by scale alone.
The paper provides detailed implementation details, including the use of Qwen3.5-9B, GRPO, specific hyperparameters (learning rate, KL coefficient), and the structure of the training tasks. The description of the "standard mixed slate" and the construction of misleading candidates is clear. The audit methodology for demonstrating credit starvation is also well-described, allowing for replication of the diagnostic findings. However, the specific code for the "clean single-oracle utility" and the exact implementation of the disjoint credit masking would be necessary for full reproducibility, which is likely in a supplementary or code release not provided here.
The method requires that the correct skill is known during training to compute the selection advantage, limiting its applicability to settings with verifiable outcomes or oracle skills. The method assumes a "single read" constraint for the selection credit to be clean; reading multiple skills dilutes the signal. The evaluation is limited to 9B and 27B models, and while it outperforms larger frontier models in selection, it does not match their overall task success, suggesting that execution capability remains a bottleneck. The audit is performed on a single run's artifacts, and while the trends are clear, broader statistical significance across seeds is not explicitly detailed in the abstract/text provided.
This work has significant implications for the development of reliable autonomous agents. By solving the credit assignment problem for skill selection, it enables agents to effectively utilize large libraries of procedural knowledge, which is critical for complex, long-horizon tasks. The insight that selection and execution require disjoint credit channels is likely to influence future agent training methodologies, moving the field away from naive outcome-only RL for tool-use scenarios. It highlights the importance of fine-grained credit assignment in multi-decision trajectories. SkillGate introduces a novel disjoint credit channel architecture that solves "selector credit starvation," enabling 9B agents to reliably select skills from large slates, significantly outperforming outcome-only RL and demonstrating that selection is a trainable, distinct capability from execution.
Discrete diffusion models have demonstrated strong performance across a range of datasets, including natural language data and graph-structured data. Among many variants, score-entropy discrete diffusion (SEDD) has achieved particularly strong empirical results. In SEDD, new samples are generated by iteratively evaluating a sequence of concrete score functions, which are learned by minimizing a score-entropy loss. While much of the prior theoretical literature on discrete diffusion has focused on the sampling efficiency of SEDD under the assumption of small score estimation error, recent work has begun to investigate the finite-sample properties of score estimation itself. In this work, we take a different route by investigating the fundamental statistical limits of concrete score estimation. We focus on uniform and masking discrete diffusions, two of the most widely adopted discrete diffusion models. We establish a minimax lower bound under the score-entropy loss, and propose an MLE-based thresholding estimator that matches this lower bound up to constant and polylogarithmic factors that depend on neighboring density ratios. We further show that, for any target distribution, this density ratio is naturally controlled under both uniform and masking discrete diffusion models, yielding nearly matching minimax lower and upper bounds for the aggregated score estimation error. Our results imply that, with appropriate initialization and discretization, SEDD can achieve nearly optimal minimax sample complexity, as measured by the KL divergence between the target and generated distributions.
Primary: The University of Texas at Austin
All Institutions: The University of Texas at Austin, Cornell University
This paper establishes the minimax optimality of score-entropy discrete diffusion by deriving tight lower bounds and proposing a matching MLE-based estimator, providing crucial theoretical grounding for the statistical efficiency of SEDD in discrete state spaces.
The paper presents a rigorous theoretical analysis of Score-Entropy Discrete Diffusion (SEDD). The core methodological contribution is the establishment of minimax lower bounds for concrete score estimation under the score-entropy loss, coupled with the proposal of an MLE-based thresholding estimator that nearly matches this bound. The authors analyze two specific transition kernels (uniform and masking) and derive bounds on the neighboring density ratios, which are critical for controlling the estimation error. The methodology is mathematically dense, relying on information-theoretic lower bounds (Fano’s method/Poissonization) and concentration inequalities. It correctly identifies that previous work focused on sampling efficiency assuming good scores, whereas this work addresses the fundamental statistical limits of score estimation itself. The approach is sound and fills a significant gap in the theoretical understanding of discrete diffusion models.
The experimental section is minimal and serves primarily to validate the theoretical scaling laws rather than to demonstrate state-of-the-art performance. The authors use a synthetic uniform distribution over binary strings of varying dimensions. They verify that the KL divergence scales linearly with the aggregated score error and that the score estimation error scales inversely with the sample size ($1/n$). While these results are consistent with the theory, the lack of evaluation on real-world discrete data (e.g., natural language or graphs) limits the empirical impact. The experiments are illustrative of the theory but do not provide strong evidence of practical superiority over existing methods in complex settings.
The paper provides detailed algorithms (Algorithm 1) and theoretical proofs. The MLE thresholding estimator is simple to implement. However, the theoretical results are asymptotic and depend on constants that are not explicitly calculated for finite samples. The synthetic experiments are straightforward to reproduce. The lack of code release (URL: none) is a minor drawback, but the simplicity of the proposed estimator suggests that reproduction would be feasible for researchers in the field.
The primary limitation is the reliance on the assumption of bounded neighboring density ratios. While the authors show this is naturally controlled for uniform and masking diffusions, real-world data distributions may not satisfy this condition tightly, potentially leading to larger constants in the bounds. Furthermore, the analysis is non-asymptotic but does not account for the approximation error introduced by neural network parameterizations of the score function; it assumes the MLE estimator is computed exactly. The experiments are limited to synthetic data, leaving the applicability to high-dimensional real-world discrete data (like text) theoretically motivated but empirically unverified in this work.
This paper provides foundational theoretical justification for the use of SEDD, establishing that it achieves near-optimal sample complexity for distribution estimation in discrete spaces. This helps demystify why SEDD performs well empirically and guides future theoretical work on discrete diffusion. It clarifies the trade-offs between different diffusion kernels (uniform vs. masking) regarding density ratio control. The work is significant for the theory of generative models but has limited immediate impact on practitioners who rely on empirical performance rather than minimax optimality. This paper establishes the minimax optimality of score-entropy discrete diffusion by deriving tight lower bounds and proposing a matching MLE-based estimator, providing crucial theoretical grounding for the statistical efficiency of SEDD in discrete state spaces.
The matrix-aware optimizer Muon improves large model training by balancing updates across singular directions, yet its scaling behavior and end-to-end efficiency on large Diffusion Transformers (DiTs) remain unclear. We first establish Muon's scaling behavior on DiTs from 1.3B to 15B parameters, showing that its optimization and generative quality advantages over AdamW persist across model scales. However, at scale, the 5-step Newton--Schulz iteration (NS5) performed at every optimization step, together with full-momentum materialization, introduces substantial computation and communication overhead that can offset Muon's step-efficiency advantage. We introduce \emph{Periodic Row-wise Muon}, which performs a full NS5 spectral update once every \(K\) steps and applies a low compute and communication cost row-wise constrained update based on the current momentum at the remaining steps. We further co-design a distributed implementation that operates directly on sharded momentum during non-refresh steps and accelerates spectral refreshes through bucketed all-gather and communication--computation overlap. Across all scales, Muon improves the best observed generative quality over AdamW by 12.9--19.1\%. Compared with vanilla Muon, Periodic Row-wise Muon remains within 0.5\% in best generative quality on the 1.3B--4B models and improves it by 4.5\% at 9B. It reduces optimizer time by 46.9--54.3\%, end-to-end step time by 15.7--24.3\%, and logical communication volume by 66.7\%, while reaching its respective best generative quality with 33.7--64.8\% less active training time. These results show that Periodic Row-wise Muon preserves Muon's generative quality advantage while translating it into end-to-end training efficiency for large DiTs.
Primary: Meta
All Institutions: Meta
[One sentence main contribution]. The paper introduces Periodic Row-wise Muon, a distributed optimizer that alternates between full spectral updates and low-cost row-wise normalization to preserve the generative quality advantages of Muon while significantly reducing the training overhead of large Diffusion Transformers. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. This paper represents a solid contribution to the field of large-scale model optimization. It correctly identifies a critical bottleneck in the application of advanced matrix-aware optimizers (Muon) to modern architectures (DiTs). The proposed solution is not merely an engineering hack but is supported by geometric intuition and theoretical analysis regarding the stability of the polar factor. The systematic evaluation across multiple model scales provides strong empirical evidence for the method's efficacy. The integration of algorithmic changes with distributed systems optimizations (sharded momentum, communication-computation overlap) demonstrates a high level of technical maturity. While the novelty is incremental (alternating updates are a known pattern), the specific application and co-design for this context are valuable. The paper is well-written, rigorous, and addresses a timely problem in the scaling of diffusion models.
The paper proposes "Periodic Row-wise Muon," a hybrid optimization strategy for Diffusion Transformers (DiTs). It addresses the high computational and communication overhead of the vanilla Muon optimizer, which relies on a 5-step Newton-Schulz (NS5) iteration at every step. The core innovation is alternating between full NS5 spectral updates (every $K$ steps) and low-cost RowNorm updates (every other step). The authors provide a theoretical justification for this alternation, arguing that the ideal polar direction (targeted by NS5) is stable under small momentum perturbations, allowing RowNorm to maintain direction while NS5 periodically corrects the global spectral geometry. They also co-design a distributed implementation that operates on sharded momentum during non-refresh steps, significantly reducing communication volume. The methodology is sound, well-motivated by the specific bottlenecks of large-scale distributed training, and mathematically grounded.
The experimental evaluation is comprehensive and rigorous. The authors train DiTs ranging from 1.3B to 15B parameters on the GPIC dataset. They demonstrate that vanilla Muon improves generative quality (FD-DINO) over AdamW by 12.9-19.1%, but suffers from higher wall-clock time due to optimizer overhead. Periodic Row-wise Muon recovers this quality advantage while reducing optimizer time by ~50% and end-to-end step time by ~20%. The results are consistent across model scales, with the largest efficiency gains at the 15B scale. The ablation studies effectively isolate the contributions of the periodic schedule, the RowNorm geometry, and the distributed system optimizations. The use of multiple generation metrics (FID, MMD, HPSv2, GenEval) provides a robust assessment of quality.
The paper provides detailed algorithmic pseudocode, complexity analysis, and distributed execution details. The experimental setup is clearly described, including dataset sources, model architectures, and hyperparameters. The inclusion of theoretical proofs in the appendix adds to the rigor. However, as is common with large-scale distributed systems research, full reproducibility might require access to the specific internal distributed training infrastructure (FSDP2) and hardware topology (32-node H100 cluster). The code is not explicitly linked, but the description is sufficient for a competent engineer to implement.
The authors acknowledge several limitations: the evaluation is limited to one DiT family, one dataset (GPIC), and one hardware configuration. They do not explore layer-wise or adaptive scheduling for the period $K$ or the RowNorm multiplier $\gamma$. The refresh steps still incur full momentum communication, which remains a bottleneck, albeit less frequent. The results may vary with different hardware topologies or parallelism strategies.
This work has significant implications for the efficiency of training large-scale generative models. By making matrix-aware optimizers like Muon practically viable for large DiTs, it enables researchers and practitioners to leverage the optimization benefits of global spectral structure without prohibitive computational costs. This contributes to the broader goal of making large-scale AI training more accessible and energy-efficient. [One sentence main contribution]. The paper introduces Periodic Row-wise Muon, a distributed optimizer that alternates between full spectral updates and low-cost row-wise normalization to preserve the generative quality advantages of Muon while significantly reducing the training overhead of large Diffusion Transformers. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. This paper represents a solid contribution to the field of large-scale model optimization. It correctly identifies a critical bottleneck in the application of advanced matrix-aware optimizers (Muon) to modern architectures (DiTs). The proposed solution is not merely an engineering hack but is supported by geometric intuition and theoretical analysis regarding the stability of the polar factor. The systematic evaluation across multiple model scales provides strong empirical evidence for the method's efficacy. The integration of algorithmic changes with distributed systems optimizations (sharded momentum, communication-computation overlap) demonstrates a high level of technical maturity. While the novelty is incremental (alternating updates are a known pattern), the specific application and co-design for this context are valuable. The paper is well-written, rigorous, and addresses a timely problem in the scaling of diffusion models.
Many machine-learning systems set a threshold at a quantile of a calibration set: conformal predictors that promise 90% coverage by drawing their cutoff at the calibration set's 90th percentile, abstention gates that decline to answer when a model's score falls below the calibration set's tenth percentile, safety filters that block any output scoring above the 99th percentile of a reference set. All of them promise that the threshold will hold at the stated rate on new data. The promise assumes the calibration examples are independent, and in modern pipelines they usually are not: they share a prompt, a document, a reasoning trace. Survey statistics has known how to discount correlated data since 1965, by counting how many independent observations a sample is worth, but only for averages. We show that a threshold needs a different count. The count depends on how often clustered scores land on the same side of the threshold, and that changes with where the threshold is set. How similar the scores are as numbers does not enter. We prove a closed-form law for the resulting effective sample size and for the spread of the coverage a deployed system actually sees. Three consequences follow. The correction now used in the conformal literature is the wrong quantity, and can miss in either direction. A dataset has no single effective sample size. It has one for each level the threshold is set at. And the damage is invisible in coverage averaged over many runs, and fully felt by whoever deploys once. On a released calibration set of 25,028 examples, we measure the reliability of about 1,300.
Primary: Unknown
All Institutions: Unknown
The paper provides a rigorous theoretical and empirical correction for the effective sample size in clustered conformal prediction, demonstrating that standard validity checks fail to detect coverage dispersion issues caused by data clustering, thereby enhancing the reliability of uncertainty quantification in ML systems.
The paper presents a rigorous theoretical derivation of the "exceedance design effect," correcting the application of Kish's design effect from survey statistics to the specific context of conformal prediction thresholds. The core methodological insight is that the effective sample size for a quantile-based threshold depends on the intra-class correlation of the *binary exceedance indicators* (whether scores fall above/below the threshold), not the correlation of the scores themselves. This distinction is critical because the binary nature of the threshold operation changes the variance structure, particularly in the tails. The authors derive a closed-form law for the coverage dispersion and extend it to ragged clusters (varying cluster sizes) using a size-biased mean substitution. The approach is mathematically sound, leveraging Bahadur representations and empirical process theory under clustering assumptions.
The experimental evaluation is strong and directly supports the theoretical claims. The authors utilize a released process-reward calibration set (25,028 examples) to demonstrate that clustered data can have an effective sample size of ~1,300, a significant reduction. They perform extensive simulations across various copula families (Gaussian, t, Clayton) to validate the theoretical predictions of coverage dispersion. Crucially, they analyze a real-world LLM calibration scenario (CoNLL-2003 sentences clustered by document) to show that standard permutation-based validity checks fail to detect the violation of exchangeability caused by clustering, leading to overconfident and miscalibrated systems. The empirical evidence effectively bridges the gap between abstract statistical theory and practical ML deployment issues.
The paper provides detailed mathematical derivations and references specific code paths (e.g., `marginal_guarantee_exact.py`) and datasets (released PRM calibration set, CoNLL-2003). The theoretical results are self-contained, and the simulation parameters are clearly defined. The use of a released dataset enhances reproducibility for the empirical portion. However, the "released artifact" mentioned in §6.1 might require specific access or processing steps not fully detailed in the text alone, though the general methodology is clear.
The theoretical results rely on asymptotic approximations (large number of clusters $b$) and specific assumptions about the dependence structure (exchangeability within clusters, continuous marginal distributions). The paper acknowledges that "informative cluster sizes" (where cluster size is correlated with the score) can introduce first-order bias that the design effect correction alone does not address, requiring additional weighting strategies. The correction is specific to the coverage dispersion and does not fully resolve bias in the mean coverage under complex sampling designs without further adjustments.
This work has significant implications for the reliability and safety of machine learning systems, particularly in conformal prediction, abstention gates, and safety filtering. By providing a correct method to estimate effective sample sizes in the presence of clustering, it helps practitioners avoid overconfident uncertainty estimates. This is crucial for high-stakes applications where miscalibration can lead to harmful decisions. The paper also highlights a systemic oversight in the ML literature regarding the distinction between shift and dependence in calibration data, encouraging more rigorous statistical validation in future work. The paper provides a rigorous theoretical and empirical correction for the effective sample size in clustered conformal prediction, demonstrating that standard validity checks fail to detect coverage dispersion issues caused by data clustering, thereby enhancing the reliability of uncertainty quantification in ML systems.
Federated learning enables privacy-preserving collaborative training, but highly heterogeneous client data remain challenging, especially in graph federated learning where clients possess structurally diverse graphs. Existing personalized federated learning (PFL) methods ignore the intrinsic geometric properties of diverse graph structures. We propose FlatLand, a novel personalized federated learning method that embeds different clients' data in tailored Lorentz space of hyperbolic geometry. Our key insight is that hyperbolic geometry naturally accommodates the intrinsic negative curvature prevalent in real-world graphs, while the time-like dimension in Lorentz space provides a principled way to encode client-specific heterogeneity. We develop a parameter decoupling strategy that separates heterogeneous information (captured in time-like parameters) from common knowledge (preserved in space-like parameters), enabling direct aggregation without requiring client similarity estimation and extra calculation modules. Empirical results on diverse federated graph learning tasks demonstrate that FlatLand achieves superior performance, particularly in low-dimensional settings.
Primary: The Chinese University of Hong Kong
All Institutions: The Chinese University of Hong Kong, Yale University, The Hong Kong University of Science and Technology (Guangzhou), Huawei Technologies Co., Ltd.
FlatLand introduces a novel geometric decoupling strategy for personalized federated graph learning, leveraging Lorentzian hyperbolic geometry to separate heterogeneous client information from common knowledge, thereby enhancing performance and privacy in structurally diverse federated settings.
The paper proposes FlatLand, a Personalized Federated Learning (PFL) framework for Graph Neural Networks (GNNs) that utilizes Lorentzian hyperbolic geometry. The core innovation lies in decoupling client-specific heterogeneity into a "time-like" dimension and common knowledge into "space-like" dimensions within the hyperbolic embedding space. This allows for aggregation of the space-like parameters while keeping time-like parameters local, theoretically addressing the challenge of structural heterogeneity in federated graph learning. The approach combines geometric insights with practical FL constraints. While the application of hyperbolic geometry to GNNs is not entirely new, its specific integration into a parameter-decoupling PFL framework for heterogeneous graphs is a distinct and well-motivated contribution. The mathematical formulation appears sound, leveraging the properties of the Lorentz model to handle negative curvature inherent in graph structures.
The authors evaluate FlatLand on diverse federated graph learning tasks. The abstract claims superior performance, particularly in low-dimensional settings. The paper includes 9 figures and 8 tables, suggesting a comprehensive empirical study. However, without access to the specific baselines and dataset details in the full text, the magnitude of improvement and statistical significance must be inferred from the abstract's claims and the venue's reputation. The focus on "low-dimensional settings" is a specific niche; performance in high-dimensional Euclidean spaces or compared to state-of-the-art Euclidean PFL methods is less clear from the summary. The inclusion of ablation studies on the decoupling strategy is implied but not explicitly detailed in the abstract.
The paper is 34 pages with extensive appendices, which typically indicates good documentation of hyperparameters and experimental setups. The methodology relies on standard hyperbolic operations and FL communication protocols, which are generally reproducible. The authors list affiliations with major tech companies and universities, suggesting access to computational resources. However, the specific implementation details of the "tailored Lorentz space" per client and the optimization dynamics are critical for reproduction and are likely contained in the full text.
A key limitation of hyperbolic embeddings is numerical instability and optimization difficulty, especially in high dimensions, although the paper claims success in low dimensions. The "tailored" aspect implies per-client geometry, which might increase memory overhead or complexity in communication if not handled carefully, although the decoupling strategy aims to mitigate this. The effectiveness of the method depends heavily on the assumption that graph heterogeneity can be effectively captured by the time-like dimension; if heterogeneity is more complex, the model might underfit. Additionally, the claim of "no client similarity estimation" is a strong advantage, but the quality of the decoupling relies on the learning dynamics, which might be sensitive to initialization or learning rates.
This work contributes to the growing field of geometric deep learning and privacy-preserving AI. By providing a principled way to handle heterogeneous data in federated settings without sharing sensitive client-specific parameters, it advances the practical applicability of FL in domains like social networks or biological graphs where data is inherently non-Euclidean and heterogeneous. The privacy-by-design aspect, avoiding the sharing of similarity matrices, is a significant benefit for sensitive applications. FlatLand introduces a novel geometric decoupling strategy for personalized federated graph learning, leveraging Lorentzian hyperbolic geometry to separate heterogeneous client information from common knowledge, thereby enhancing performance and privacy in structurally diverse federated settings.
For AI agents to be useful beyond simple chat, they must hold sensitive user context such as calendars, credentials, health records, and financial data. We study whether the mere presence of such secrets in a model's context window introduces hidden correlations into the model's benign outputs, allowing reconstruction even when the model correctly refuses direct extraction. We further study whether an adversary can actively engineer prompts that amplify this effect, using the model as a covert carrier to transmit secrets through seemingly innocuous text. In both cases, this limited leakage is exploited using a novel adaptive attack that assumes black-box access to the underlying model. In controlled experiments across eight proprietary models, we find that 2-digit in-context secrets are reconstructed with near-perfect accuracy and 4-digit secrets at 82\% exact match, all from outputs the model produces in response to ordinary, non-adversarial requests. We observe that more capable models leak more: stronger instruction-following amplifies sensitivity to in-context secrets, suggesting leakage is a byproduct of capability as opposed to a patchable bug. We show this leakage enables two practical attacks: (1) a trained classifier that infers semantic predicates about user memories (e.g., health conditions, financial events) from routine natural-language outputs, and (2) an RL-trained adversary that extracts full Social Security Numbers from a production-style agent.
Primary: University of California, San Diego
All Institutions: University of California, San Diego, University of Maryland
This paper presents a rigorous and impactful analysis of inadvertent context leakage in language models, demonstrating that sensitive information can be reconstructed from benign outputs through statistical side-channels, even when the model explicitly refuses to disclose it. The introduction of adaptive black-box decoders to exploit these channels provides a new and critical perspective on LLM privacy, with significant implications for the deployment of AI agents handling sensitive user data.
The paper introduces a novel threat model termed "inadvertent context leakage," moving beyond traditional direct extraction or prompt injection attacks. The core methodology involves training adaptive, black-box decoders (using LoRA-finetuned classifiers) to detect statistical correlations between in-context secrets and benign model outputs. The authors isolate specific leakage channels, including response length, stylistic features, and digit frequency profiles, demonstrating that these channels persist even when the model explicitly refuses to output the secret. The approach is rigorous, employing a controlled experimental setup across eight proprietary frontier models and utilizing both passive inference and active RL-optimized prompt injection to engineer covert channels.
The experimental evaluation is comprehensive and convincing. The authors test on eight leading proprietary models (Claude, Gemini, GPT, Grok) using a variety of secret lengths (1-8 digits) and semantic predicates. Key findings include near-perfect reconstruction of 2-digit secrets and 82% exact match for 4-digit secrets on Claude Opus 4.6. The paper provides strong ablation studies, including the "suppression" hypothesis (that stronger instructions to hide data paradoxically increase leakage by distorting the output distribution), cross-model transferability of decoders, and the effectiveness of the attack against production-style agents (SSN extraction). The results are statistically significant and robust across different prompt templates and context sizes.
The paper provides detailed descriptions of the attack pipelines, decoder architectures (Qwen-2.5-1.5B backbone with LoRA), and hyperparameters. The use of public APIs for proprietary models limits exact reproducibility of the victim models' internal states, but the attack methodology is clearly defined. The authors provide prompt templates and describe the GRPO optimization process for the active attack. The data collection process (1,000 secrets, 10 responses per pair) is well-documented, allowing other researchers to replicate the study on accessible models or verify the API interactions.
The study is limited to numeric secrets and binary semantic predicates; generalization to open-vocabulary, free-text secrets is not addressed due to the combinatorial explosion of the value space. The experiments are conducted in a black-box setting via APIs, which may not capture all side-channels available in local deployments. The "resistant" tier of models (Grok, GPT-nano) suggests that leakage is model-dependent, and the mechanisms for resistance are not fully understood or mitigated. The study assumes the adversary can query the model, which may not always be the case in all deployment scenarios.
This work has significant implications for the privacy of AI agents and the security of LLM deployments. It challenges the assumption that instruction-following and refusal mechanisms are sufficient for data protection. The findings suggest that privacy in LLMs requires fundamental architectural changes or information-theoretic guarantees rather than just prompt engineering or output filtering. This could drive new research into privacy-preserving model training, secure inference protocols, and standardized privacy benchmarks for AI agents. This paper presents a rigorous and impactful analysis of inadvertent context leakage in language models, demonstrating that sensitive information can be reconstructed from benign outputs through statistical side-channels, even when the model explicitly refuses to disclose it. The introduction of adaptive black-box decoders to exploit these channels provides a new and critical perspective on LLM privacy, with significant implications for the deployment of AI agents handling sensitive user data.
Hidden chain-of-thought (CoT) traces, especially those from frontier proprietary large reasoning models (LRMs), are valuable model assets. Yet whether these hidden CoTs can be directly extracted from black-box models remains largely unexplored. In this work, we systematically study whether hidden CoTs can be extracted near-verbatim from black-box LRMs through API interactions. We identify a previously overlooked reasoning replay surface between tool calls and develop EchoCoT, a multi-step attack that iteratively extracts hidden CoTs using API-returned fidelity signals. We further develop an LLM-based optimization framework that automatically searches for an effective universal injection trajectory across various datasets. We evaluate EchoCoT on three open-source and five frontier proprietary LRMs. On open-source LRMs, EchoCoT achieves up to 66.4\% near-verbatim extraction success, with the extracted trace length within 10\% of the target and at least 90\% of tokens exactly matching the target CoT. The same injection trajectory also generalizes to unseen datasets, achieving up to 80\% extraction success under the same criterion. For tested frontier proprietary LRMs, a substantial fraction of extracted CoTs closely align with provider-reported reasoning lengths and available CoT summaries. EchoCoT can also extract very long CoTs: on Gemini-2.5, it extracts 33,463 tokens from a 32,948-token target. These results establish hidden-CoT extraction as a practical security risk and highlight the need to better protect hidden CoT assets.
Primary: CISPA Helmholtz Center for Information Security
All Institutions: CISPA Helmholtz Center for Information Security
EchoCoT establishes a practical and effective method for extracting hidden Chain-of-Thought traces from black-box Large Reasoning Models, exposing a significant security vulnerability in current API designs and setting a new benchmark for CoT privacy research.
The paper proposes EchoCoT, a sophisticated black-box attack that exploits the "reasoning replay surface" inherent in tool-calling mechanisms of Large Reasoning Models (LRMs). The core insight is that while standard multi-turn conversations discard hidden CoTs, tool calls preserve them within the context window. By iteratively injecting instructions via a scratchpad tool, the authors force the model to reproduce its hidden reasoning. The novelty lies in the automated optimization framework (LLM-based Inject-Reflect-Distill) that searches for universal injection trajectories using only API-provided proxy signals (token counts and summaries) as fidelity feedback. This is a clever adaptation of prompt optimization techniques to the specific constraints of black-box CoT extraction. The methodology is technically sound and demonstrates a deep understanding of LRM internal states and API behaviors.
The evaluation is extensive and rigorous. The authors test EchoCoT on three open-source LRMs (DeepSeek, Qwen, GLM) with ground-truth CoTs available for precise measurement, and five frontier proprietary models (Gemini, Claude) where they use proxy metrics and qualitative analysis. The results are striking: on open-source models, they achieve near-verbatim extraction (Token-EM > 0.90) for a significant fraction of samples. The cross-dataset transferability is also strong, with optimized trajectories working well on unseen datasets. The ability to extract very long CoTs (33k+ tokens) is a significant empirical finding. The baselines (Direct Prompting, CoT Synthesis, REP) are appropriate, and EchoCoT vastly outperforms them.
The paper provides a detailed description of the attack pipeline, optimization framework, and evaluation metrics. The code is available on GitHub. The use of public APIs for proprietary models ensures that the attack can be reproduced by others, provided they have access to the same models. The open-source model evaluations are fully reproducible.
The attack relies on the availability of tool-calling interfaces and specific API signals (reasoning token counts, summaries). Models that disable tool calls or hide these signals would be resistant. The attack is also computationally expensive due to the iterative nature and the need for multiple LLM calls for optimization. Furthermore, the success rate, while high, is not 100%, and failures often stem from length mismatches or refusals. The paper notes that defenses are possible but currently insufficient.
This work has profound implications for the security and privacy of frontier LRM providers. It demonstrates that hidden CoTs, considered valuable intellectual property and safety assets, are not secure against determined black-box attackers. This could lead to industrial espionage, leakage of sensitive training data, or bypassing of safety filters embedded in the reasoning process. It highlights a critical gap in the security design of LRM APIs and necessitates new defense mechanisms, such as obfuscating reasoning steps, limiting tool-call context, or detecting adversarial tool usage. EchoCoT establishes a practical and effective method for extracting hidden Chain-of-Thought traces from black-box Large Reasoning Models, exposing a significant security vulnerability in current API designs and setting a new benchmark for CoT privacy research.
Multi-agent LLM systems split work across models, so answering often requires knowledge that sits in another agent's context: a Sharer has encoded information that a Receiver needs to complete its task. They usually communicate by exchanging text, which puts autoregressive decoding on the critical path and reduces the exchange to a discrete message written without sight of the receiver's state. Recent latent protocols instead translate the sharer's key-value (KV) cache into the receiver's: C2C supports heterogeneous models but requires both to read the same input, while LCF-X removes this shared-context requirement through position-free sharer-cache pooling. Three restrictions remain: LCF-X compresses the sharer alone, supplies the same layer-local summary to every receiver position with no joint cross-layer memory to retrieve from, and assumes matched layer count and KV geometry. We introduce XKV, which lifts all three: learned-query attention pools both caches; self-attention over receiver-aligned layer tokens, with a learned layer map reconciling different depths, mixes the pooled summaries into a compact joint memory; and a shared position decoder lets every raw receiver cache position retrieve its own per-head-gated residual in the receiver's native KV geometry. Both models stay frozen and may differ in family, depth, KV-head count, head dimension, and tokenizer; only the translator is trained. Across 45 dataset-model-pair settings (six heterogeneous and three same-model ordered pairings, five datasets), XKV attains the highest macro score and best average rank, improving on LCF-X on every dataset (by 4.6 exact-match and 4.2 F1 points on ROPES) and surpassing text communication on four of the five, while training 76% fewer parameters and translating a cache pair 10.3x faster (5.8 vs. 59.9 ms); end to end, XKV is 26% faster than LCF-X and 6.8x faster than text communication.
Primary: Amazon
All Institutions: Amazon
[One sentence main contribution]. XKV introduces a dual-cache latent communication protocol that pools both sharer and receiver KV caches into a joint memory, enabling efficient, receiver-aware, and geometry-agnostic knowledge transfer between heterogeneous frozen LLMs. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper makes a solid technical contribution to the emerging area of latent communication in multi-agent systems. By addressing the limitations of prior work (LCF-X) through dual-cache pooling and position-specific retrieval, it achieves state-of-the-art accuracy and efficiency on split-evidence reasoning tasks. The rigorous evaluation across heterogeneous model pairs and the clear demonstration of efficiency-accuracy trade-offs make this a valuable addition to the literature. The method is well-motivated, technically detailed, and empirically validated.
The paper proposes XKV, a "dual-cache" latent communication protocol for heterogeneous frozen LLMs. The core innovation lies in pooling both the sharer's and receiver's KV caches using learned-query attention to create a joint memory, rather than just compressing the sharer's cache as in prior work (LCF-X). This joint memory is then queried by each receiver position to retrieve position-specific, receiver-native KV residuals. The method includes a learned layer map to handle depth mismatches and a shared position decoder. The approach is technically sound, leveraging standard attention mechanisms in a novel configuration for cross-model state transfer. It addresses specific bottlenecks in previous latent communication methods: receiver-independence, layer-local single-summary bottlenecks, and matched-geometry assumptions.
The evaluation is comprehensive, covering 45 dataset-model-pair settings across five benchmarks (ROPES, MuSiQue, QASC, StrategyQA, HotpotQA-bridge) and nine ordered pairs of Qwen, Gemma, and Llama models. XKV consistently outperforms text-to-text (T2T) and the latent baseline LCF-X in accuracy/F1 scores. Crucially, it demonstrates significant efficiency gains: 10.3x faster communication latency than LCF-X and 6.8x faster end-to-end than T2T, with 76% fewer parameters. The ablation studies effectively isolate the contribution of the dual-cache pooling and position-specific retrieval. The results are robust and clearly presented.
The paper provides detailed mathematical formulations for the pooling, layer alignment, and decoding steps. It specifies the model families, dataset splits, and evaluation metrics. The claim that both base models are frozen and only the translator is trained simplifies the reproduction of the inference pipeline. The latency measurements are specific and comparable. While code is not explicitly linked in the text provided, the methodological description is sufficient for reproduction by a competent researcher.
The method assumes a "cross-context" setting where evidence is split between two models. It does not address multi-turn conversations or many-agent systems with complex topologies, though this is noted as future work. The performance gain over LCF-X, while consistent, is moderate in absolute terms (e.g., +4.6 EM on ROPES). The reliance on a learned translator introduces a small overhead, albeit minimal compared to the savings. The evaluation is limited to small-to-medium models (0.6B-3B); scaling to larger models (7B+) is not demonstrated.
This work advances the field of multi-agent LLM systems by enabling more efficient and accurate latent communication. By removing the need for text decoding/encoding cycles and supporting heterogeneous models, it lowers the barrier to building distributed LLM systems. The efficiency gains could lead to reduced latency and cost in deployed agent frameworks. The approach of joint state pooling is a generalizable technique that could inspire similar methods in other domains requiring cross-model information transfer. [One sentence main contribution]. XKV introduces a dual-cache latent communication protocol that pools both sharer and receiver KV caches into a joint memory, enabling efficient, receiver-aware, and geometry-agnostic knowledge transfer between heterogeneous frozen LLMs. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper makes a solid technical contribution to the emerging area of latent communication in multi-agent systems. By addressing the limitations of prior work (LCF-X) through dual-cache pooling and position-specific retrieval, it achieves state-of-the-art accuracy and efficiency on split-evidence reasoning tasks. The rigorous evaluation across heterogeneous model pairs and the clear demonstration of efficiency-accuracy trade-offs make this a valuable addition to the literature. The method is well-motivated, technically detailed, and empirically validated.
Self-normalized concentration inequalities are standard tools in bandit and reinforcement-learning analyses. A widely used weighted extension claims an analogous time-uniform guarantee for discounted least-squares estimators in non-stationary problems. A simple scalar Gaussian counterexample with a fixed parameter shows that the claimed bounded radius is crossed with probability one. For fixed discount and regularization parameters, we further show that, when $δ\leq1/2$ and $T/δ$ is sufficiently large, any deterministic anytime boundary valid uniformly over the stated conditionally sub-Gaussian model class must be at least of order $R\sqrt{\log(T/δ)}$ at some time by horizon $T$; for nondecreasing boundaries, this order is required at time $T$. We identify the proof error: different terminal times use different Gaussian mixing distributions, so the fixed-time mixtures do not form one supermartingale, and the stopping-time argument does not repair this failure. Finally, we show that the weighted inequality remains valid at each fixed deterministic time, give valid finite- and infinite-horizon corrections, and discuss consequences for downstream analyses.
Primary: Academia Sinica
All Institutions: Academia Sinica, National Tsing Hua University
This paper provides a crucial correction to the theoretical analysis of discounted least-squares in non-stationary bandits, identifying a fundamental flaw in the proof of a widely used time-uniform concentration inequality and establishing the necessity of logarithmic confidence costs for valid anytime bounds.
The paper employs a rigorous theoretical analysis to identify a fundamental flaw in the proof of a widely used self-normalized concentration inequality for discounted least-squares estimators. The methodology involves constructing a scalar Gaussian counterexample to disprove the time-uniform bound claimed in prior work (RVC19weightedLB) and deriving matching lower bounds for any valid anytime boundary. The authors correctly identify the logical error in the original proof: the use of different Gaussian mixing distributions at different times prevents the construction of a single supermartingale required for Ville's inequality, and the proposed "stopping-time repair" is invalid because it conditions on a sequence of directions rather than a fixed one. This is a classic example of "e-hacking" or pseudo-maximization in sequential analysis. The correction provided involves explicit confidence allocation (union bound) which restores validity but at the cost of a logarithmic factor in the horizon.
As a theoretical note, the paper does not present empirical experiments in the traditional sense (e.g., training models on datasets). Instead, the "experiments" are the mathematical proofs of the counterexample and the lower bounds. The scalar Gaussian example serves as a synthetic proof-of-concept that the theoretical bounds fail. The evaluation is purely analytical, relying on probability theory and concentration inequalities.
The mathematical derivations, proofs, and counterexamples are fully detailed in the text and appendices. The logic is self-contained and reproducible by any researcher with a background in stochastic processes and bandit theory. The identification of the error in prior work is precise and verifiable.
The primary limitation is that this is a "negative" result paper; it identifies an error and provides a correction, but does not propose a new, more efficient algorithm or a significantly tighter bound than the corrected union-bound approach. The correction (union bound) is standard but may be loose in practice compared to what a correct time-uniform bound might achieve if one existed (though the lower bound suggests the logarithmic factor is necessary for non-decreasing boundaries). The scope is limited to the specific discounted least-squares setting and does not address all possible variations of non-stationary bandit algorithms.
This paper has significant impact on the theoretical foundations of non-stationary bandit and reinforcement learning literature. By correcting a widely cited inequality, it invalidates the theoretical guarantees of several downstream papers (RCG20-NSGLB, KT20, TV20, etc.) that relied on the flawed anytime bound. It forces the community to re-evaluate the regret bounds of these algorithms, potentially requiring them to pay an additional logarithmic factor or rely on fixed-time analysis with union bounds. It serves as an important cautionary tale regarding the application of Ville's inequality and supermartingale arguments in time-varying settings. This paper provides a crucial correction to the theoretical analysis of discounted least-squares in non-stationary bandits, identifying a fundamental flaw in the proof of a widely used time-uniform concentration inequality and establishing the necessity of logarithmic confidence costs for valid anytime bounds.
Most modern optimizers form their momentum as an exponential moving average (EMA) of past gradients, forgetting every direction at one fixed rate. However, the inputs a deep network sees during training can be highly anisotropic, with a few directions queried frequently while most are seen rarely. Recent methods address this anisotropy by wrapping extra processing around this buffer, leaving the momentum update itself unchanged. We propose DeltaMomentum, which builds direction-awareness into the momentum update rule. The main observation is that the gradient of a linear layer splits into an input that acts as a key and an output-side error that acts as a value. Exploiting the key-value structure, DeltaMomentum updates the momentum buffer by the canonical delta rule, so each direction is forgotten at a rate set by how often it appears. We prove that it is a valid momentum, that it applies the input-side curvature correction without matrix inversion, and that it clears stale directions faster than EMA under both a fixed and a drifting optimum. It is a drop-in replacement for the momentum buffer of any optimizer, its coefficient transfers across widths under $μ$P, and its extra compute stays between $22.2\%$ and $25.0\%$ of a gated-MLP block's linear cost with no persistent memory. In FineWeb-Edu pretraining, AdamW with DeltaMomentum (DeltaAdamW) reaches AdamW's validation loss in up to $46.39 \pm 4.32\%$ fewer steps at 67M and $22.12 \pm 0.80\%$ at 370M over three seeds, and the gain persists at 1B on a Chinchilla-optimal budget. A Muon baseline tuned under the same protocol sits above DeltaAdamW at both language-model scales, and the gain holds for SGD, ResNet-18, and ViT-Tiny on CIFAR-10. Training-time diagnostics confirm the predicted mechanism, better gradient tracking and healthier input directions.
Primary: University of Pennsylvania
All Institutions: University of Pennsylvania, Jane Street
DeltaMomentum introduces a theoretically grounded, direction-aware momentum update rule that leverages the key-value structure of gradients to achieve faster convergence and implicit preconditioning. The paper presents a compelling combination of rigorous theory, detailed mechanistic analysis, and strong empirical results across multiple scales and architectures, establishing a new baseline for momentum-based optimization that is likely to influence future optimizer design.
The paper proposes "DeltaMomentum," a novel momentum update rule that replaces the standard Exponential Moving Average (EMA) with a delta-rule update based on the key-value structure of rank-1 gradient factorization in linear layers. The core insight is that treating the momentum buffer as an associative memory allows for direction-aware forgetting rates, where directions queried frequently are updated more aggressively. The methodology includes a theoretical derivation showing that this update implicitly performs input-side preconditioning (similar to K-FAC but without matrix inversion) and provides guarantees on convergence and tracking of non-stationary targets. The approach is theoretically sound, well-motivated by the anisotropy of deep learning landscapes, and offers a clean, drop-in replacement for existing optimizers.
The experimental evaluation is comprehensive and convincing. The authors conduct controlled ablations against AdamW (the only baseline differing in the first moment) and Muon (a strong structured optimizer baseline) on Llama-2-style language models at 67M, 370M, and 1B parameters on FineWeb-Edu. The results show significant step-efficiency gains (up to 46% fewer steps) and competitive wall-clock efficiency. Additional experiments on CIFAR-10 with ResNet-18 and ViT-Tiny demonstrate generalizability. The inclusion of mechanistic diagnostics (gradient alignment, function-space prediction error, feature covariance conditioning) strongly supports the theoretical claims. The compute overhead is quantified and shown to be manageable.
The paper provides detailed algorithmic descriptions, theoretical proofs, and mentions specific implementation details (e.g., normalized keys, $\mu$P scaling). The experimental setup is described with sufficient detail for reproduction, including data sources (FineWeb-Edu) and model architectures. The authors provide a clear recipe for hyperparameter transfer, enhancing reproducibility.
The paper acknowledges limitations regarding scale (up to 1B parameters) and domain (language modeling only). It does not yet demonstrate performance on larger models (e.g., 7B+) or other domains like vision-only or multimodal tasks, though the CIFAR results suggest potential. The theoretical analysis relies on quasi-static approximations, which may not perfectly capture the highly non-stationary dynamics of deep training, although the empirical diagnostics support the theory.
This work has significant potential to improve the efficiency of training large-scale neural networks, reducing compute costs and energy consumption. By providing a drop-in replacement for momentum, it can be easily adopted by the community. The theoretical insights into the role of momentum as an associative memory could inspire further research into optimizer design. DeltaMomentum introduces a theoretically grounded, direction-aware momentum update rule that leverages the key-value structure of gradients to achieve faster convergence and implicit preconditioning. The paper presents a compelling combination of rigorous theory, detailed mechanistic analysis, and strong empirical results across multiple scales and architectures, establishing a new baseline for momentum-based optimization that is likely to influence future optimizer design.
Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy, outcome-rewarded RL over the candidate slate, cannot teach it, for a structural reason we identify and name selector credit starvation: under a broadcast, sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run's own training artifacts confirms all three properties, each worsening monotonically with horizon. SkillGate removes the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separate action-local advantage reaching exactly the skill-naming tokens, positive only when a trajectory's single read is the correct one. On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.
Primary: Unknown (Affiliations not explicitly listed in text, authors appear to be from various Chinese institutions based on names and "corrauth" markers)
All Institutions: Unknown
SkillGate introduces a novel disjoint credit channel architecture that solves "selector credit starvation," enabling 9B agents to reliably select skills from large slates, significantly outperforming outcome-only RL and demonstrating that selection is a trainable, distinct capability from execution.
The paper introduces "SkillGate," a method that addresses the specific problem of in-policy skill selection in long-horizon agentic tasks. The core technical contribution is the identification of "selector credit starvation," a structural flaw in standard outcome-rewarded RL where the tokens responsible for selecting a skill receive vanishingly small and often incorrectly signed gradients. The proposed solution partitions the token support into two disjoint credit channels: an execution channel using standard group-normalized outcome advantages, and a selection channel using an action-local advantage that rewards only the specific tokens naming the correct skill. This separation prevents the selection decision from being punished by execution failures and ensures the selection signal is not diluted by trajectory length. The methodology is theoretically sound, addressing a known bottleneck in agent training with a precise, mechanistic fix.
The authors evaluate SkillGate on five agentic benchmarks (Claw-Eval, SkillsBench, SETA, SWE, Terminal-Bench 2.0) using a 16-candidate slate. They demonstrate that SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, outperforming outcome-only RL, supervised selection, and preference learning baselines. Crucially, they show that SkillGate achieves this with fewer reads and less exposure to misleading skills, indicating genuine improvement in selection capability rather than just increased tool usage. The ablation studies effectively isolate the contribution of the selection channel, showing that coarser credit assignment (group or trajectory level) fails to improve selection. The comparison with frontier models highlights that skill selection is a distinct capability not solved by scale alone.
The paper provides detailed implementation details, including the use of Qwen3.5-9B, GRPO, specific hyperparameters (learning rate, KL coefficient), and the structure of the training tasks. The description of the "standard mixed slate" and the construction of misleading candidates is clear. The audit methodology for demonstrating credit starvation is also well-described, allowing for replication of the diagnostic findings. However, the specific code for the "clean single-oracle utility" and the exact implementation of the disjoint credit masking would be necessary for full reproducibility, which is likely in a supplementary or code release not provided here.
The method requires that the correct skill is known during training to compute the selection advantage, limiting its applicability to settings with verifiable outcomes or oracle skills. The method assumes a "single read" constraint for the selection credit to be clean; reading multiple skills dilutes the signal. The evaluation is limited to 9B and 27B models, and while it outperforms larger frontier models in selection, it does not match their overall task success, suggesting that execution capability remains a bottleneck. The audit is performed on a single run's artifacts, and while the trends are clear, broader statistical significance across seeds is not explicitly detailed in the abstract/text provided.
This work has significant implications for the development of reliable autonomous agents. By solving the credit assignment problem for skill selection, it enables agents to effectively utilize large libraries of procedural knowledge, which is critical for complex, long-horizon tasks. The insight that selection and execution require disjoint credit channels is likely to influence future agent training methodologies, moving the field away from naive outcome-only RL for tool-use scenarios. It highlights the importance of fine-grained credit assignment in multi-decision trajectories. SkillGate introduces a novel disjoint credit channel architecture that solves "selector credit starvation," enabling 9B agents to reliably select skills from large slates, significantly outperforming outcome-only RL and demonstrating that selection is a trainable, distinct capability from execution.
Programmable logic controllers (PLCs) run industrial plants, and large language models can already generate independent program organization units (POUs) for them. Whether such logic integrates into an existing PLC project and then runs correctly has been checked only in limited tests. We present SemaPLC, a project-grounded and verification-gated agent harness assembled from conventional tools but governed by a strict completion rule. Rather than stopping when the model judges its own output adequate, SemaPLC declares a task complete only when logged external checks confirm it. Those checks cover the specification, the compilation, and the behavior on a live runtime. On 117 independent-POU tasks matching existing benchmarks, it attains the highest strict verified pass rate on all seven models (72.6\% mean). On a project-context track of 65 tasks whose generated logic must compile and run inside a real project, it attains the highest mean on integrated compilation, static behavior, and dynamic behavior. Of the three layers, dynamic behavior is the most revealing. We measure it by deploying the generated and the reference logic to a live PLC runtime and comparing their executed traces. All methods fall within 10 static points of one another, whereas dynamic scores separate them sharply, from 22.4 to 31.4 for the baselines against 52.2 for SemaPLC. Overall, our verification-gated harness raises the mean at every layer and most sharply at runtime. Execution, not static scoring, is the faithful test of whether generated control logic actually works. SemaPLC is open-sourced at https://github.com/midea-ai/SemaPLC.
Primary: Midea AI
All Institutions: Midea AI
SemaPLC presents a robust, verification-gated agentic framework for PLC code generation that significantly improves runtime reliability over static baselines, establishing a new standard for evaluation in industrial LLM applications.
The paper proposes SemaPLC, an agentic harness for generating Programmable Logic Controller (PLC) code in Structured Text (ST). The core methodological contribution is not a new neural architecture, but a rigorous "verification-gated" iteration loop. It combines project-grounded generation (retrieving context from existing PLC projects) with three layers of external verification: specification auditing, compilation, and live runtime validation. The agent is forced to repair code until these external checks pass, preventing premature termination based on self-assessment. This approach effectively treats the LLM as a component within a larger, tool-using agent system, leveraging the Model Context Protocol (MCP) for tool interaction. The methodology is sound and addresses a specific, high-stakes domain (industrial automation) where static code generation metrics are insufficient.
The evaluation is comprehensive and rigorous, which is the paper's strongest asset. It introduces two tracks: a function-level track (117 tasks) and a project-context track (65 tasks). The project-context track is particularly novel, requiring integration into existing industrial plant simulations. The authors evaluate across seven state-of-the-art LLMs. The results show that SemaPLC significantly outperforms baselines (LLM4PLC, AutoPLC, Agents4PLC) on dynamic runtime behavior, a metric that static analysis fails to capture. The ablation studies clearly demonstrate the value of each verification layer. The use of a live PLC runtime for evaluation provides a level of empirical truth often missing in LLM code generation papers. The audit of the previous benchmark (Agents4PLC) to fix defective properties adds credibility to the evaluation framework.
The paper provides a GitHub repository link. The methodology describes the tools and protocols (MCP) used. The evaluation setup, including the construction of scenarios and the handling of hidden references, is detailed. The open-sourcing of the harness and the cleaned benchmark data enhances reproducibility. The use of standard LLM APIs ensures that the backbone models are accessible, although the specific versions and endpoints are noted. The detailed description of the verification skills and the agent loop allows for reasonable replication of the system.
The paper acknowledges that dynamic scoring is limited to the bounded scenario set derived from the hidden reference, meaning generalization to unseen conditions is not fully measured. The advantage narrows on the strongest models (e.g., GPT-5.5), suggesting that as models improve, the marginal gain from the complex harness may decrease. The cost in terms of API requests is higher for SemaPLC on the project track compared to some baselines, which is a practical limitation for cost-sensitive applications. The focus on IEC 61131-3 ST limits immediate applicability to other PLC languages or industrial protocols without adaptation.
This work has significant implications for the deployment of LLMs in safety-critical industrial environments. By demonstrating that verification-gated agents can reliably generate code that works on live hardware, it provides a blueprint for trustworthy AI in automation. It highlights the gap between static code metrics and actual runtime performance, encouraging the field to adopt more rigorous, execution-based evaluation standards. The open-sourcing of the tool and benchmark contributes to the community's ability to build and evaluate agentic systems for industrial tasks. SemaPLC presents a robust, verification-gated agentic framework for PLC code generation that significantly improves runtime reliability over static baselines, establishing a new standard for evaluation in industrial LLM applications.
Estimating causal effects under network interference typically assumes that the network used for training and the network used for deployment coincide. In practice, an intervention is run on one population while the question of interest concerns a different population, and the two generally differ in topology, node-covariate composition, and spillover pathways. Transporting a causal effect across networks is therefore a data-fusion problem that no existing algorithm solves. We employ a selection diagram, extended to the network setting so that covariate shift and structural network shift enter as separate selectors, and derive from it a transport formula for the direct, spillover, and total effects in the deployment population. Each formula makes explicit which interventional mechanism is assumed invariant and which observational distribution must be reweighted. We then turn the formulas into TranCE (Transported Causal Effects), a doubly-robust algorithm combining an interventional outcome model, a domain density-ratio correction, and cross-fitted inference. Extensive experiments on two semi-synthetic benchmarks derived from real-world social networks and on a fully real weather-insurance field experiment, where the transported effects are checked against held-out randomized estimates, confirm the effectiveness of our approach. Our findings have the potential to improve intervention strategies in networked systems, particularly in social networks and public health.
Primary: Unknown
All Institutions: Unknown
TranCE provides a theoretically grounded and empirically validated solution for transporting causal effects across networks under interference, addressing a critical limitation of existing methods that assume identical network structures. The extension of selection diagrams to network settings and the doubly-robust estimator offer a significant advancement in the field of causal machine learning.
The paper addresses a significant gap in causal inference: transporting causal effects across different network topologies and covariate distributions under interference. The core theoretical contribution is the extension of selection diagrams to the network setting, explicitly separating covariate shift ($S_Z$) from structural network shift ($S_G$). This allows for the derivation of transport formulas for direct, spillover, and total effects. The proposed algorithm, TranCE, implements these formulas using a doubly-robust estimator that combines an interventional outcome model (using GCNs to handle network structure) with a domain density-ratio correction (using logistic regression) and cross-fitted inference. The methodology is rigorous, leveraging standard causal inference tools (do-calculus, doubly-robust estimation) in a novel, complex setting. The separation of structural and covariate shifts is a key theoretical insight that clarifies what assumptions are required for transportability in networked systems.
The evaluation is extensive and convincing. It includes two semi-synthetic benchmarks (Twitch, Facebook-100) with rotating source-target pairs to simulate cross-network transport, and a fully real-world field experiment (weather-insurance in rural China). The semi-synthetic experiments compare TranCE against strong baselines including TARNet, IPW, IGL, Hoshino, NetEst, DANN, and IW-GCN. TranCE consistently achieves the lowest bias, particularly for spillover effects, demonstrating the necessity of both the outcome model and the density-ratio correction. The ablation studies effectively isolate the contributions of different components. The real-world validation against held-out randomized estimates provides strong external validity. The analysis of "Where Transport Is Hard" (correlation between structural gap and bias) is particularly insightful for practitioners.
The paper states that source code, data-preparation scripts, and instructions are included in a supplement. The mathematical derivations are detailed, and the algorithmic steps are clearly described. The use of standard libraries (PyTorch, GCNs) and well-defined causal estimators aids reproducibility. The semi-synthetic data generation process is described, though the specific parameters might require careful reading of the appendix. The real-world dataset (Cai insurance) is publicly available.
The method relies on the assumption of stratified interference, which limits the scope of interference to local neighborhoods. The neighbor treatment summary is discretized into a small number of levels, which might lose information. The method assumes that the interventional mechanism is invariant across domains (except for the parts captured by the selectors), which is a strong assumption in practice. The performance degrades significantly when there is a large structural gap between source and target networks, although this is expected and well-documented. The method also assumes overlap in the covariate space, which can be difficult to verify and ensure in high-dimensional settings.
This work has significant potential impact for policy evaluation in social networks, public health interventions, and online platform optimization. By enabling the transport of causal effects from controlled experiments to observational settings with different network structures, it allows for more efficient and ethical deployment of interventions. It provides a rigorous framework for understanding the limits of generalizability in networked causal inference. TranCE provides a theoretically grounded and empirically validated solution for transporting causal effects across networks under interference, addressing a critical limitation of existing methods that assume identical network structures. The extension of selection diagrams to network settings and the doubly-robust estimator offer a significant advancement in the field of causal machine learning.
Coupled-cluster theory defines the accuracy standard for molecular electronic-structure properties but scales too steeply for routine application, whereas density-functional theory is affordable yet systematically biased. We resolve this trade-off with a single equivariant network, MEHnet-MG, that predicts an effective one-electron Hamiltonian from one inexpensive B3LYP/def2-SVP calculation and derives a broad suite of properties from it (energy, optical gap, dipole, quadrupole, polarizability, Mulliken atomic charges, and Mayer bond orders) at coupled-cluster accuracy across nine main-group elements, including the under-served phosphorus, sulfur, and chlorine chemistries. The model is trained on a new in-house dataset of multi-property labels computed at the CCSD(T) level for all nine elements. On a held-out test set, it reduces the error of every property by a factor of 3.8 to 230 relative to semi-local, hybrid, and double-hybrid DFT (referenced to composite CCSD(T)/cc-pVTZ; Methods), while adding only ~25 ms wall time per molecule, delivering coupled-cluster-quality predictions at the cost of a single DFT calculation. Critically, deriving every property from a predicted Hamiltonian rather than pooling per-atom features builds the correct size-scaling into the model architecture: on pi-conjugated oligothiophenes it matches finite-field CCSD polarizability and the EOM-CCSD optical gap to ~2% at the largest sizes where those references remain affordable (44 and 37 atoms, where a single CCSD field point already costs ~500x the model's entire inference) and extrapolates the corrected trends to 58-atom chains, a regime where pooling-based architectures fail by construction. Accurate extrapolation is therefore set by the model's inductive bias rather than by the training data.
Primary: Massachusetts Institute of Technology
All Institutions: Massachusetts Institute of Technology, Honda Research Institute, U.S. Department of Energy, National Energy Research Scientific Computing Center, Texas Advanced Computing Center
This paper presents a significant advance in scientific machine learning by introducing a Hamiltonian-readout equivariant network that achieves coupled-cluster accuracy for multiple molecular properties at DFT cost, with demonstrated ability to extrapolate size-scaling trends beyond the training distribution.
The paper introduces MEHnet-MG, a novel equivariant graph neural network architecture that predicts an effective one-electron Hamiltonian correction to a cheap DFT baseline (B3LYP/def2-SVP). The core methodological innovation is the "Hamiltonian read-out" strategy: instead of pooling atomic features to predict properties directly (which enforces strict extensive/intensive scaling), the model predicts local corrections to the Fock matrix and screening matrix. Properties are then derived via exact quantum-mechanical operators (diagonalization, linear response). This architectural choice embeds the correct physical size-scaling into the model, allowing it to extrapolate to larger systems where pooling-based models fail. The approach effectively decouples the accuracy ceiling (set by the CCSD(T) labels) from the computational cost (set by the DFT baseline).
The evaluation is rigorous and comprehensive. The authors train on a new in-house dataset of ~42k main-group molecules and test on a held-out set of 959 molecules. They benchmark against three standard DFT functionals (BP86, B3LYP, DSD-PBEP86) and demonstrate significant error reductions (factors of 3.8 to 230) across seven properties (energy, gap, dipole, quadrupole, polarizability, charges, bond orders). Crucially, they perform out-of-distribution tests on oligothiophene chains up to 58 atoms, showing that MEHnet-MG correctly extrapolates super-linear scaling of polarizability and gap saturation, whereas ablated pooling-based models diverge. They also validate against experimental dipole moments from NIST CCCBDB, showing accuracy comparable to canonical CCSD(T).
The paper provides extensive details on the dataset generation, filtering, and reference calculations. Code, model weights, and data generation scripts are made publicly available on GitHub. The authors explicitly discuss the limitations of their train/test split (index-based rather than scaffold-based) and provide a roadmap for future benchmarks. The reproducibility is high, although the full training dataset labels are not publicly redistributed, requiring regeneration or request.
The primary limitation is the reliance on the quality of the training labels. The optical gap predictions are bounded by the EOM-CCSD/cc-pVDZ reference, which lacks diffuse functions and thus has a known offset from experiment. The model inherits this error. Additionally, the method is currently limited to closed-shell, neutral main-group molecules (H, C, N, O, F, Si, P, S, Cl). The train/test split is not scaffold-split, so true chemical generalization to unseen scaffolds is not fully validated. The cost is dominated by the DFT baseline, not the neural network, so hardware acceleration of the DFT step is still required for high throughput.
This work represents a significant step toward bridging the accuracy-cost gap in computational chemistry. By enabling coupled-cluster accuracy at DFT cost for a broad suite of properties across the main group, it facilitates high-throughput screening of materials and molecules that were previously inaccessible to high-accuracy methods. The architectural insight regarding Hamiltonian read-outs for size-transferability is likely to influence the design of future scientific machine learning models. This paper presents a significant advance in scientific machine learning by introducing a Hamiltonian-readout equivariant network that achieves coupled-cluster accuracy for multiple molecular properties at DFT cost, with demonstrated ability to extrapolate size-scaling trends beyond the training distribution.
OWL 2 DL ontologies, grounded in the description logic $\mathcal{SROIQ}$, express large knowledge bases in biomedicine and the Semantic Web. Neuro-symbolic (NeSy) learners over description logics either embed the ontology in a continuous space, abandoning classical entailment, or restrict to the Horn fragment $\mathcal{EL}^{++}$, which has a single canonical model. We present Baobab, which compiles a $\mathcal{SROIQ}$ ontology with a finite ABox into a Sentential Decision Diagram (SDD): it saturates a propositional core under a consequence-based calculus and instantiates the remaining $\mathcal{SROIQ}$ features (nominals, number restrictions, and the role axioms) over the active domain. The SDD's evidence-conditioned weighted model count then trains a perception network to recognize real images under partial ABox supervision: on an ontology that exercises every distinctive $\mathcal{SROIQ}$ feature, a CNN learns to read MNIST digits coupled by a successor relation and recovers latent ontology concepts that an independent perception leaves at chance. When the supervision admits several ontology-consistent completions, an independent perception collapses onto one, a reasoning shortcut: we show that a mixture indexed by the query's justifications can represent the calibrated posterior no independent perception can, and that seeding it from the circuit's enumerated completions attains the Bayes-optimal posterior on a real-image MNIST task where single-WMC and learned mixtures (the BEARS-ensemble hypothesis class) do not: to our knowledge the first to characterize and mitigate reasoning shortcuts in a non-Horn description logic. Soundness of the compiler and the representation result are machine-checked in Lean 4. Code is available at https://github.com/bio-ontology-research-group/baobab.
Primary: King Abdullah University of Science and Technology
All Institutions: King Abdullah University of Science and Technology
Baobab introduces a rigorous, formally verified compiler from OWL 2 DL to differentiable SDDs, solving the critical problem of multi-modal reasoning shortcuts in neuro-symbolic learning and enabling exact logical training on expressive ontologies.
The paper presents "Baobab," a novel neuro-symbolic framework that bridges the gap between expressive Description Logics (specifically OWL 2 DL / $\mathcal{SROIQ}$) and differentiable neural learning. The core methodological innovation is a consequence-based compiler that transforms an $\mathcal{SROIQ}$ ontology and a finite ABox into a Sentential Decision Diagram (SDD). This allows for exact, differentiable weighted model counting (WMC) over the logical constraints during the training of a perception network (CNN). Crucially, the authors address the "reasoning shortcut" problem inherent in single-mode WMC losses when supervision is partial or ambiguous. They propose "JustWMC," a mixture-of-experts approach where mixture components are seeded from the logical justifications (completions) enumerated by the circuit. This allows the model to represent multi-modal posteriors that a standard independent perception cannot. The formalization is rigorous, with soundness proofs for the compiler and representation results for the mixture model, all machine-checked in Lean 4.
The experimental evaluation is strong and well-designed. The authors use two primary benchmarks: a synthetic MNIST-based task involving successor relations and parity/primality constraints, and a real-world application using the Pizzaiolo ontology with ResNet encoders. The MNIST experiments effectively isolate the logical reasoning capabilities, demonstrating that Baobab recovers latent concepts (digit identities) with high accuracy ($0.99$) under fully determined supervision and significantly outperforms baselines (single-WMC, BEARS) in under-determined regimes by correctly modeling multi-modal posteriors. The Pizzaiolo experiment validates the approach on a real, complex ontology. The results are statistically significant (Holm-corrected). The comparison against BEARS is particularly compelling, showing that JustWMC attains the Bayes-optimal posterior where learned mixtures fail.
The paper provides a clear algorithmic description, including normalization, saturation, grounding, and SDD compilation steps. The code is explicitly made available at the provided GitHub URL. The use of standard libraries (PySDD, PyTorch) and the availability of the Lean 4 formalization add to the reproducibility and trustworthiness of the claims. The experimental setup details (seeds, metrics, hyperparameters) are sufficiently described.
The primary limitation is the computational complexity of compiling $\mathcal{SROIQ}$ ontologies to SDDs. While the paper claims polynomial bounds in certain contexts, SDD size can still be exponential in the treewidth of the underlying constraint graph. The "grounding" step over the active domain may become prohibitive for very large ABoxes, although the paper notes this is a known trade-off for exact reasoning. Additionally, the current evaluation focuses on image data; generalization to other modalities (text, graph) is not demonstrated, though the method is modality-agnostic in principle. The reliance on finite ABoxes for grounding is a standard assumption in this subfield but limits direct application to infinite domains without approximation.
This work significantly advances the field of Neuro-Symbolic AI by enabling the use of highly expressive, standard-compliant ontologies (OWL 2 DL) in differentiable learning pipelines. This has immediate implications for biomedical informatics, scientific knowledge discovery, and the Semantic Web, where logical consistency is paramount. By providing a mechanism to handle multi-modal logical uncertainty (reasoning shortcuts), it offers a more robust foundation for AI systems that must reason under ambiguity. The formal verification in Lean 4 sets a high bar for reliability in NeSy systems. Baobab introduces a rigorous, formally verified compiler from OWL 2 DL to differentiable SDDs, solving the critical problem of multi-modal reasoning shortcuts in neuro-symbolic learning and enabling exact logical training on expressive ontologies.
Most large language model services use stateless defenses, which judge only the current request, to refuse harmful tasks. Decomposition attacks exploit this limitation by splitting a harmful task into individually permissible requests and combining their answers. Defending against them therefore requires a stateful monitor that considers requests together. If it can group all requests for one attacker task, it can stop the attack. However, attackers can use unlinkable identities and combine answers elsewhere, leaving no reliable grouping signal. We ask whether decomposition attacks can still be stopped under this setting. For a fixed attack strategy without retries, we prove that the achievable security and utility tradeoff depends entirely on how benign requests for the same capabilities are grouped. Persistent, recognizable groups permit a useful defense; fresh, indistinguishable groups do not. When attackers can retry and learn from Allow/Block decisions, this useful operating point disappears: the feedback reveals what passes but not whether a block was correct. Experiments on 91 executable tasks and 11,393 capability-matched benign requests support these results. Under a 1% denial cap for these requests and a 0.5% cap for unrelated background traffic, all ten tested policies, including one privileged policy with an exact request-to-operation map, either fail to stop attacks or exceed the budget. On defense-unseen task families, attack success is at least 99% after one attempt and 100% after two. Effective defenses therefore require additional evidence or mechanisms tied to grouping, such as reliable identity linkage, costs for fresh identities, or control over answer use.
Primary: Johns Hopkins University
All Institutions: Johns Hopkins University
This paper provides a rigorous theoretical and empirical demonstration of the fundamental limits of stateful defenses against decomposition attacks in LLM services when attacker identities are unlinkable, establishing a critical benchmark for future safety research.
The paper presents a rigorous theoretical and empirical analysis of "decomposition attacks" against Large Language Model (LLM) services. The core methodological contribution is the formalization of the security-utility tradeoff in stateful defense systems where attacker identities are unlinkable. The authors derive a theoretical bound showing that without reliable grouping signals (identity linkage), effective defense is impossible under specific constraints. They complement this with extensive experiments testing ten different defense policies against executable tasks, demonstrating that current state-of-the-art defenses fail to stop attacks when identities are unlinkable and feedback is limited. The approach combines game-theoretic reasoning with practical red-teaming, providing a robust framework for understanding the limits of current LLM safety mechanisms.
The experimental evaluation is comprehensive and convincing. The authors test against 91 executable tasks and over 11,000 capability-matched benign requests. The results are stark: under strict denial caps (1% for target, 0.5% for background), all tested policies either failed to stop attacks or exceeded the budget. On defense-unseen task families, attack success rates were 99% after one attempt and 100% after two. The use of executable tasks adds a layer of practical relevance compared to purely text-based benchmarks. The control of benign request matching ensures that the observed failures are not due to poor utility but rather fundamental structural limitations of the defenses.
The paper provides a clear formulation of the attack and defense models. The dataset of 91 executable tasks and the description of the 11,393 benign requests allow for replication. The theoretical proofs are included in the supplement. The experimental setup is described in sufficient detail for other researchers to reproduce the failure modes of the tested policies.
The primary limitation is the assumption of "unlinkable identities." While realistic for many public-facing APIs, it may not hold for authenticated enterprise services where identity is known. Additionally, the theoretical results are specific to the defined threat model (fixed strategy without retries, or retry with limited feedback). The paper does not explore defenses that rely on external signals (e.g., IP reputation, device fingerprinting) as primary grouping mechanisms, though it mentions them as necessary. The scope is limited to decomposition attacks; other attack vectors are not the focus.
This paper has significant implications for the deployment of LLM services. It challenges the industry's reliance on stateless or weakly stateful defenses and highlights a critical vulnerability in the current paradigm of LLM safety. The findings suggest that effective defense requires either reliable identity linkage (which raises privacy concerns) or new architectural approaches to content moderation. This work will likely influence the development of more robust safety standards and the design of future LLM APIs, pushing the field towards more holistic security models that account for multi-turn, multi-account adversarial behavior. This paper provides a rigorous theoretical and empirical demonstration of the fundamental limits of stateful defenses against decomposition attacks in LLM services when attacker identities are unlinkable, establishing a critical benchmark for future safety research.
Browser agents perform well on short, clean demonstrations, but real deployment is fundamentally different: agents must sustain dozens of decisions on live websites while recovering from mistakes and navigating complex UIs. We argue that closing this gap requires alignment at every level of the pipeline, including execution, supervision, optimization, and evaluation, rather than scale alone. We present Wuying-Browser-Agent, a unified framework that addresses each of these levels. A structured browser harness provides stable execution primitives and decision-oriented context management. Reflection and UI-specialized Curriculum SFT (RUIC-SFT) explicitly trains on recovery trajectories and complex-UI interactions. Divergence-Aware Online GRPO (DAO-GRPO) improves long-horizon credit assignment through potential-based reward shaping and divergence-aware step weighting. Finally, we introduce BrowserBench, a bilingual real-web benchmark of 350 tasks averaging 37.9 steps, because most existing benchmarks are too short to expose long-horizon failure modes. Wuying-Browser-Agent-27B achieves 80.6\% on WebVoyager, 66.7\% on Online-Mind2Web, and 65.1\% on BrowserBench, establishing a new open-source state of the art on browser-use benchmarks. The same pipeline also transfers beyond browser use, demonstrating strong general agentic ability and reaching an average score of 73.8 on Tau2-Bench, Claw-Eval, and BFCL-v4.
Primary: Alibaba Cloud
All Institutions: Alibaba Cloud
Wuying-Browser-Agent presents a comprehensive and practically significant framework for long-horizon browser agents, combining structured execution, recovery-oriented supervised fine-tuning, and divergence-aware reinforcement learning to achieve state-of-the-art results on challenging real-world benchmarks.
The paper proposes a unified framework for long-horizon browser agents, addressing three specific structural challenges: lack of recovery supervision, dilute credit assignment in long trajectories, and insufficient bilingual evaluation. The methodology consists of three main components: (1) a structured browser harness for stable execution and context management; (2) RUIC-SFT, a curriculum-based supervised fine-tuning stage that incorporates reflection-rich recovery trajectories and specialized UI interaction data; and (3) DAO-GRPO, an online reinforcement learning algorithm that uses potential-based reward shaping and divergence-aware step weighting to improve long-horizon credit assignment. The approach is technically sound and addresses real pain points in the field of agentic LLMs. The integration of recovery data into SFT and the specific design of the divergence-aware GRPO variant are notable contributions. However, the core ideas (curriculum learning, reward shaping, divergence analysis) are incremental extensions of existing techniques rather than fundamentally new theoretical breakthroughs.
The authors evaluate Wuying-Browser-Agent on WebVoyager, Online-Mind2Web, and their own benchmark, BrowserBench. The model achieves state-of-the-art results among open-source models on these benchmarks. The introduction of BrowserBench is a significant contribution, providing a more realistic, long-horizon, and bilingual evaluation suite. The experiments demonstrate that RUIC-SFT and DAO-GRPO provide consistent gains, particularly on longer and more complex tasks. The transferability to general agentic benchmarks (Tau2-Bench, Claw-Eval, BFCL-v4) is also demonstrated, suggesting the method improves general tool-use capabilities. The evaluation is comprehensive and convincing, although the reliance on a new benchmark requires the community to adopt it for future comparisons.
The paper provides detailed descriptions of the browser harness, data construction processes, and training objectives. The structured action space and context management rules are well-defined. The authors mention releasing the model at 4B, 9B, and 27B scales, which aids reproducibility. However, the specific hyperparameters for the divergence-aware step weighting and the exact details of the LLM-based divergence estimator are somewhat abstractly described ("tuned on the validation set"), which might make exact replication challenging without access to the code or further details. The use of a proprietary sandbox (AgentBay) for evaluation also introduces some dependency, although the interface is described as lightweight and protocol-based.
The paper acknowledges that the divergence estimator is an LLM-based heuristic and not an exact oracle. The method relies on a curated set of reflection and UI-specialized data, which may not cover all possible failure modes or UI types. The performance on BrowserBench, while strong, is specific to the tasks included, and generalization to entirely new domains or website structures is not fully explored. The computational cost of online RL with grouped rollouts and divergence analysis is likely high, which may limit accessibility for smaller research groups. Additionally, the "divergence-aware" mechanism's reliance on an external LLM for step comparison adds latency and complexity to the training loop.
This work contributes to the development of more robust and reliable autonomous agents for web interaction, which has significant implications for automation, accessibility, and human-computer interaction. The introduction of BrowserBench encourages the community to focus on long-horizon, real-world, and bilingual tasks, moving beyond short, synthetic, or English-only benchmarks. The open-sourcing of the models and potentially the benchmark (if included in the release) will facilitate further research in agentic AI. However, the increased capability of browser agents also raises concerns about automated scraping, botting, and security vulnerabilities on the web, which should be considered in the deployment of such systems. Wuying-Browser-Agent presents a comprehensive and practically significant framework for long-horizon browser agents, combining structured execution, recovery-oriented supervised fine-tuning, and divergence-aware reinforcement learning to achieve state-of-the-art results on challenging real-world benchmarks.
Vision encoders are a critical component of vision-language models, and scaling their capacity effectively improves performance. However, dense scaling increases compute cost and inference latency. Mixture-of-Experts (MoE) architectures offer a compelling alternative, having enabled efficient scaling in LLMs, yet the MoE design space for CLIP-style vision encoders remains underexplored at State-of-the-Art (SOTA) levels. In this work, we systematically study MoE designs for vision encoder scaling and find that fine-grained MoE topologies yield substantial gains over both dense and standard MoE counterparts. We further propose an auxiliary-loss-free balancing variant for better expert utilization, and design a specialized MoE kernel to mitigate inference latency overhead. To enhance video capabilities while preserving image knowledge, we introduce frame-level distillation paired with a novel freezing mechanism. We pretrain a series of Mixture-of-Experts Vision Encoders (MoE-ViE) across a range of sizes, all consistently outperforming their dense counterparts. Our largest model matches the zero-shot performance of a SOTA encoder 1.7x its size at 76% of its latency. When aligned with an LLM, MoE-ViE surpasses all compared encoders on image and video benchmarks, including those with up to 5x more activated parameters. Code is available at https://github.com/facebookresearch/moe_vie.
Primary: Meta
All Institutions: Meta
MoE-ViE presents a comprehensive and effective framework for scaling vision encoders using Mixture-of-Experts, achieving state-of-the-art efficiency and accuracy through fine-grained architectures, novel load balancing, and hardware-aware kernel optimizations.
The paper proposes MoE-ViE, a Mixture-of-Experts vision encoder designed to scale capacity without linearly increasing inference latency. The core methodological contributions include: 1) Fine-grained expert topologies (smaller hidden widths per expert) compared to standard dense-to-MoE replacements; 2) A magnitude-aware, loss-free load balancing mechanism using z-score updates to router biases, avoiding auxiliary loss perturbations; 3) A specialized Triton-based MoE kernel implementing Grouped GEMM and kernel fusion to mitigate memory-bound bottlenecks; 4) A robust video finetuning strategy combining frame-level distillation and MLP freezing to prevent catastrophic forgetting of image representations. The approach is technically sound and addresses specific pain points in scaling CLIP-style models (latency vs. capacity trade-off and cross-modal forgetting).
The authors conduct extensive experiments across multiple scales (B, L, H) on zero-shot image classification (ImageNet, ImageNet-A, etc.), retrieval, and video understanding benchmarks. They demonstrate that MoE-ViE consistently outperforms dense counterparts of similar active compute budgets and matches or exceeds SOTA dense models (like SigLIP2-g-opt and PEcoreG) with significantly fewer active parameters. Latency benchmarks on H100 GPUs confirm the efficiency gains from the custom kernel. The ablation studies are thorough, covering expert granularity, balancing strategies, and finetuning components. The results are compelling and align with the claims.
The paper provides detailed architectural specifications, training schedules, and hyperparameters. The code is made available via a GitHub link. The description of the custom Triton kernel is sufficiently detailed for reproduction by practitioners familiar with low-level GPU optimization. The use of standard datasets (ImageNet, Kinetics, etc.) ensures comparability.
The paper relies heavily on proprietary data ("1.5B proprietary data") for pretraining, which limits the ability of external researchers to fully replicate the training conditions. The performance gains are significant but incremental in the context of the broader VLM landscape, where data quality and scale often dominate over architectural nuances. The "loss-free" balancing, while effective, may be sensitive to the specific z-score scaling factors chosen, though the paper argues for robustness. The focus is primarily on image/video encoding; the impact on the LLM decoder side is limited to the encoder's output quality.
This work provides a practical blueprint for scaling vision encoders efficiently, which is crucial for deploying large VLMs in resource-constrained environments. By demonstrating that MoE can outperform dense models in vision tasks when properly optimized (architecture + kernel), it challenges the assumption that dense models are always superior for vision. The video finetuning strategy is also broadly applicable to unified vision-language models. MoE-ViE presents a comprehensive and effective framework for scaling vision encoders using Mixture-of-Experts, achieving state-of-the-art efficiency and accuracy through fine-grained architectures, novel load balancing, and hardware-aware kernel optimizations.
High-performance ML systems increasingly rely on GPU kernels whose editable source is unavailable, generated, or too distant from final machine code to expose remaining optimizations. Existing LLM kernel optimizers and autotuners mainly operate on CUDA, Triton, HIP, or tensor-program source and validate against reference implementations. We study a stricter setting: optimizing an already compiled AMDGPU code object, where the deployed binary is the only behavioral oracle. We present AsmEvo, an agentic assembly-level optimizer for AMD GPU kernels. Given an AMDGPU code object K0, AsmEvo reconstructs a reassemblable representation, proposes low-level edits with a long-horizon agent, rebuilds an ABI-preserving optimized object, and accepts candidates only after differential verification against K0 under identical launches. AsmEvo combines code-object recovery, metadata-aware rebuilding, profiling-guided hot-window editing, correctness-gated timing, and conservative in-place patch fallback. We conduct extensive experiments with AsmEvo on various AMD GPU kernels. On MI308X, AsmEvo improves 29 of 30 selected KernelBench kernels, reaching 1.35x geometric-mean and 3.88x maximum speedup. On MI300X production workloads, it improves all evaluated AITer binaries and vLLM/SGLang Triton assembly kernels, reaching 1.09x/1.31x and 1.18x/1.34x geometric-mean/maximum speedups, respectively, while preserving functional equivalence.
Primary: AMD
All Institutions: AMD
AsmEvo presents a novel and technically rigorous approach to post-compilation GPU kernel optimization, effectively bridging the gap between high-level ML frameworks and low-level hardware performance by leveraging LLMs for assembly-level editing under strict functional equivalence constraints.
The paper proposes AsmEvo, a system for optimizing AMD GPU kernels at the assembly level without source code. The core methodology involves three stages: (1) recovering a reassemblable AMDGCN assembly representation from the compiled ELF/HSACO binary, (2) using a long-horizon LLM agent to propose low-level edits within "hot windows" identified by profiling, and (3) enforcing a strict "gated verification" harness that rebuilds the binary, checks for ABI preservation, and verifies functional equivalence against the original binary (the oracle) before measuring performance. The approach is technically sound and addresses a real gap in the ML systems stack: post-compilation optimization of deployed artifacts. The use of a differential oracle (comparing output of optimized vs. original binary) is a robust way to handle correctness without source-level reference implementations. The separation of the LLM agent (exploration) from deterministic gates (verification) is a good architectural choice to prevent hallucinated or incorrect optimizations from being accepted.
The experiments are conducted on AMD MI308X and MI300X hardware. The evaluation covers KernelBench (L1 and L2) and production kernels from AITer, vLLM, and SGLang. The results show significant speedups, with a geometric mean of 1.35x on KernelBench and up to 3.88x on specific kernels. The inclusion of production workloads (AITer, Triton HSACOs) is a strong point, demonstrating real-world applicability. The verification process is rigorous, ensuring that speedups are not due to semantic changes. However, the sample size for production kernels is small (8 kernels), and the KernelBench results, while impressive, are on benchmark kernels that might not fully represent the complexity of end-to-end inference workloads. The comparison is primarily internal (optimized vs. original binary), which is appropriate for the problem setting, but lacks comparison to other state-of-the-art autotuners or compiler optimizations on the same binaries, as those typically require source.
The paper provides detailed descriptions of the recovery, rebuild, and verification pipelines. The use of standard tools (ROCm, AMDGCN) and the clear definition of the verification harness enhance reproducibility. However, the reliance on a specific LLM (Claude Opus 4.8) and the proprietary nature of some production kernels (AITer) may limit full reproducibility for external researchers. The code for the recovery and rebuild tools is likely not open-sourced in the same way as typical ML models, which could hinder adoption.
The primary limitation is the scope of the optimization: it is specific to AMD GPU architecture (AMDGCN). The recovery process might fail for complex or obfuscated binaries. The "real-dispatch capture" mechanism, while powerful, requires access to the running application's memory state, which may not always be feasible or safe in all deployment environments. The speedups, while significant, are bounded by the remaining optimization headroom in the compiled binary, which is often less than in source-level optimization. The paper acknowledges that equivalence is empirical, not formal.
This work has significant implications for the ML systems community, particularly for organizations that deploy compiled ML models where source code is unavailable or optimization at the source level is no longer possible. It demonstrates that there is still value in low-level, binary-level optimization. The methodology could be extended to other architectures (e.g., NVIDIA SASS) and could inspire new tools for binary-level program analysis and optimization. It also highlights the potential of LLMs for low-level systems programming tasks when combined with rigorous verification. AsmEvo presents a novel and technically rigorous approach to post-compilation GPU kernel optimization, effectively bridging the gap between high-level ML frameworks and low-level hardware performance by leveraging LLMs for assembly-level editing under strict functional equivalence constraints.
Long-context modeling is a pivotal capability for Large Language Models, yet the quadratic complexity of attention remains a critical bottleneck, particularly during the compute-intensive prefilling phase. Our previous work, FlashPrefill, mitigates this cost through instantaneous pattern discovery and max-based dynamic thresholding; however, it remains an algorithmic prototype that is still distant from production deployment. In this paper, we present FlashPrefill V2, which evolves FlashPrefill from a prototype toward practical long-context serving along three dimensions. First, we introduce a mean correction term that effectively suppresses the approximation error, keeping performance degradation manageable even at extreme sparsity levels. Second, we redesign the sparse attention operator with PackGQA memory access, warp specialization, and pingpong pipelining, fully aligning with the latest FlashAttention-3/4 implementations and supporting FP8 inference to meet practical quantization requirements. Third, FlashPrefill V2 natively supports paged KV cache and continuous batching, allowing integration as an attention backend in modern inference frameworks such as SGLang. Extensive evaluations on NVIDIA H20 GPUs---among the most widely deployed inference accelerators---demonstrate that FlashPrefill V2 delivers up to 47.26x and 27.19x speedups over FlashAttention-2 at 128K context length under FP8 and BF16 precision, respectively, and, in FP8, still achieves a 30.49x speedup against an FA3/4-aligned dense baseline.
Primary: Tencent WeChat
All Institutions: Tencent WeChat
FlashPrefill V2 presents a substantial systems-level contribution to long-context LLM serving by transforming a sparse attention prototype into a production-ready, highly optimized backend that achieves massive speedups on widely deployed hardware through advanced kernel engineering and integration with modern inference frameworks.
The paper proposes FlashPrefill V2, an optimization for the prefilling phase of Long-Context LLM serving. It builds upon a previous prototype (FlashPrefill) by introducing three key technical components: 1) A mean correction term to reduce approximation error in block-sparse attention, allowing for higher sparsity without significant accuracy loss. 2) A highly optimized CUDA kernel implementation leveraging PackGQA memory access, warp specialization, and ping-pong pipelining, aligned with FlashAttention-3/4 paradigms and supporting FP8 inference. 3) Integration support for paged KV cache and continuous batching, making it a viable backend for modern inference engines like SGLang. The methodology is sound and addresses a critical bottleneck (prefill latency) in long-context serving. The shift from algorithmic prototype to production-ready system is a significant engineering contribution. EXPERIMENTAL_EVALUTION: The evaluation is conducted on NVIDIA H20 GPUs, which are noted as widely deployed inference accelerators. The paper reports substantial speedups: up to 47.26x over FlashAttention-2 (FA2) at 128K context under FP8, and 27.19x under BF16. It also compares against an FA3/4-aligned dense baseline, showing a 30.49x speedup in FP8. The results demonstrate clear performance gains. However, the paper lacks detailed accuracy benchmarks (e.g., perplexity drops on standard datasets) beyond the claim that error is "manageable." The focus is heavily on throughput/latency, which is appropriate for a systems paper, but a more rigorous accuracy analysis would strengthen the claim of "practical" deployment.
The paper describes specific kernel optimizations (warp specialization, ping-pong pipelining) and integration points (SGLang). While the algorithmic description is clear, the full source code is not explicitly linked in the abstract or provided in the text snippet. Reproducibility depends on the availability of the code repository, which is not extracted here. The use of standard hardware (H20) and frameworks (SGLang) aids reproducibility if code is open-sourced.
The primary limitation is the reliance on block-sparse attention, which introduces approximation error. While the mean correction helps, it does not eliminate it, potentially affecting tasks requiring precise attention mechanisms. The evaluation is limited to NVIDIA H20 GPUs; performance on other architectures (e.g., H100, A100, or AMD GPUs) is not reported. The paper does not provide extensive accuracy degradation analysis across diverse benchmarks. The "extreme sparsity levels" mentioned in the abstract are not quantified in the provided text, making it hard to assess the trade-off curve.
This work has significant potential impact on the efficiency of long-context LLM serving, reducing costs and latency for applications requiring large context windows. By aligning with modern inference frameworks and supporting FP8, it promotes energy-efficient and cost-effective deployment of large models. The focus on production-ready systems bridges the gap between academic research and industrial application. FlashPrefill V2 presents a substantial systems-level contribution to long-context LLM serving by transforming a sparse attention prototype into a production-ready, highly optimized backend that achieves massive speedups on widely deployed hardware through advanced kernel engineering and integration with modern inference frameworks.
We introduce Accelerating Dexterity via Pre-Training (ADEPT), a large-scale reinforcement learning (RL) framework for learning sim-to-real transferable dexterity across high degree-of-freedom (DoF) robot embodiments that can solve long-horizon tasks directly from raw visuo-tactile perception. ADEPT pretrains a dexterous policy on a generic object reposing task, then post-trains downstream policies with this pretrained behavior as a prior. ADEPT enables learning new behaviors that are otherwise difficult to discover from scratch on multi-fingered robots and avoids learning the same set of skills over again for every new downstream task. The pretrained policy zero-shots the reposing phase of downstream tasks, but naïve RL fine-tuning rapidly degrades this capability during transfer. We address this with a stable post-training recipe combining behavior-cloning distillation, critic warm-up, and conservative on-policy updates. To safely exploit the full kinematic dexterity, we introduce a joint-space Geometric Fabric that mediates between the RL policy and the robot. We distill post-trained teachers into perceptive students that zero-shot sim-to-real transfer on two embodiments: a 23 DoF Kuka-Allegro with two RGB cameras, and a 29 DoF Flexiv-Sharpa with two RGB cameras and five vision-based tactile sensors, and can solve long-horizon tasks from challenging initial states with dexterity at human-level speed.
Primary: University of Toronto
All Institutions: University of Toronto, Vector Institute, NVIDIA
The paper presents a significant advancement in dexterous manipulation via RL, introducing a robust pre-training and post-training framework that enables stable and efficient learning of complex manipulation skills. The combination of behavioral cloning distillation, critic warm-up, and conservative updates effectively mitigates the instability often associated with fine-tuning RL policies, while the Geometric Fabric ensures safe and feasible control. The successful sim-to-real transfer on multiple high-DoF platforms demonstrates the practical utility and generalizability of the approach, making it a valuable contribution to the field of robotics and reinforcement learning.
The paper introduces ADEPT, a framework for learning dexterous manipulation policies using Reinforcement Learning (RL). The core methodological contribution is a two-stage training pipeline: 1) Pre-training a policy on a generic "reposing" task to learn basic object handling and kinematic feasibility, and 2) Post-training (fine-tuning) on specific downstream tasks. Crucially, the authors address the issue of "catastrophic forgetting" or instability during fine-tuning by introducing a stable recipe combining behavior cloning (BC) distillation, critic warm-up, and conservative on-policy updates. They also introduce a "Geometric Fabric" in the joint space to mediate between the RL policy and the robot, ensuring safe and feasible control. The approach leverages large-scale simulation data and transfers to real-world embodiments (Kuka-Allegro and Flexiv-Sharpa). The methodology is sound and addresses a significant bottleneck in dexterous manipulation: the difficulty of discovering complex manipulation skills from scratch via RL.
The evaluation is extensive and rigorous. The authors demonstrate zero-shot sim-to-real transfer on two distinct high-DoF robotic platforms. They show that ADEPT enables the robots to solve long-horizon tasks (like object reposing and manipulation) from challenging initial states. The results indicate that the pre-training phase significantly accelerates learning and improves the stability of the post-training phase compared to training from scratch. The comparison against baselines (likely standard RL or BC-only approaches) highlights the effectiveness of the proposed post-training recipe. The ability to transfer to real hardware without further tuning is a strong empirical result.
The paper provides detailed descriptions of the simulation environment, the robot models, and the training hyperparameters. The code is made available on GitHub, and the project website includes demo videos. The use of standard simulators (Isaac Gym) and well-known robot models enhances reproducibility. The authors explicitly discuss the limitations and the specific conditions under which the method works, aiding in replication.
The method relies heavily on high-quality simulation-to-real transfer, which can still be sensitive to domain gaps not captured in simulation (e.g., friction variations, sensor noise). The "Geometric Fabric" adds a layer of complexity to the control stack. The pre-training on a generic task requires careful design to ensure the learned priors are useful for diverse downstream tasks. The computational cost of pre-training on large-scale RL data is significant.
This work contributes to the advancement of general-purpose robotic manipulation, a key step towards more autonomous and versatile robots in unstructured environments. By making dexterous manipulation more accessible via RL, it lowers the barrier for research and application in this domain. The open-source release of code and models promotes further research and development in the field. The paper presents a significant advancement in dexterous manipulation via RL, introducing a robust pre-training and post-training framework that enables stable and efficient learning of complex manipulation skills. The combination of behavioral cloning distillation, critic warm-up, and conservative updates effectively mitigates the instability often associated with fine-tuning RL policies, while the Geometric Fabric ensures safe and feasible control. The successful sim-to-real transfer on multiple high-DoF platforms demonstrates the practical utility and generalizability of the approach, making it a valuable contribution to the field of robotics and reinforcement learning.