Last 7 Days (August 19 – August 25, 2026)
Most modern optimizers form their momentum as an exponential moving average (EMA) of past gradients, forgetting every direction at one fixed rate. However, the inputs a deep network sees during training can be highly anisotropic, with a few directions queried frequently while most are seen rarely. Recent methods address this anisotropy by wrapping extra processing around this buffer, leaving the momentum update itself unchanged. We propose DeltaMomentum, which builds direction-awareness into the momentum update rule. The main observation is that the gradient of a linear layer splits into an input that acts as a key and an output-side error that acts as a value. Exploiting the key-value structure, DeltaMomentum updates the momentum buffer by the canonical delta rule, so each direction is forgotten at a rate set by how often it appears. We prove that it is a valid momentum, that it applies the input-side curvature correction without matrix inversion, and that it clears stale directions faster than EMA under both a fixed and a drifting optimum. It is a drop-in replacement for the momentum buffer of any optimizer, its coefficient transfers across widths under $μ$P, and its extra compute stays between $22.2\%$ and $25.0\%$ of a gated-MLP block's linear cost with no persistent memory. In FineWeb-Edu pretraining, AdamW with DeltaMomentum (DeltaAdamW) reaches AdamW's validation loss in up to $46.39 \pm 4.32\%$ fewer steps at 67M and $22.12 \pm 0.80\%$ at 370M over three seeds, and the gain persists at 1B on a Chinchilla-optimal budget. A Muon baseline tuned under the same protocol sits above DeltaAdamW at both language-model scales, and the gain holds for SGD, ResNet-18, and ViT-Tiny on CIFAR-10. Training-time diagnostics confirm the predicted mechanism, better gradient tracking and healthier input directions.
Primary: University of Pennsylvania
All Institutions: University of Pennsylvania, Jane Street
DeltaMomentum introduces a theoretically grounded, direction-aware momentum update rule that leverages the key-value structure of gradients to achieve faster convergence and implicit preconditioning. The paper presents a compelling combination of rigorous theory, detailed mechanistic analysis, and strong empirical results across multiple scales and architectures, establishing a new baseline for momentum-based optimization that is likely to influence future optimizer design.
The paper proposes "DeltaMomentum," a novel momentum update rule that replaces the standard Exponential Moving Average (EMA) with a delta-rule update based on the key-value structure of rank-1 gradient factorization in linear layers. The core insight is that treating the momentum buffer as an associative memory allows for direction-aware forgetting rates, where directions queried frequently are updated more aggressively. The methodology includes a theoretical derivation showing that this update implicitly performs input-side preconditioning (similar to K-FAC but without matrix inversion) and provides guarantees on convergence and tracking of non-stationary targets. The approach is theoretically sound, well-motivated by the anisotropy of deep learning landscapes, and offers a clean, drop-in replacement for existing optimizers.
The experimental evaluation is comprehensive and convincing. The authors conduct controlled ablations against AdamW (the only baseline differing in the first moment) and Muon (a strong structured optimizer baseline) on Llama-2-style language models at 67M, 370M, and 1B parameters on FineWeb-Edu. The results show significant step-efficiency gains (up to 46% fewer steps) and competitive wall-clock efficiency. Additional experiments on CIFAR-10 with ResNet-18 and ViT-Tiny demonstrate generalizability. The inclusion of mechanistic diagnostics (gradient alignment, function-space prediction error, feature covariance conditioning) strongly supports the theoretical claims. The compute overhead is quantified and shown to be manageable.
The paper provides detailed algorithmic descriptions, theoretical proofs, and mentions specific implementation details (e.g., normalized keys, $\mu$P scaling). The experimental setup is described with sufficient detail for reproduction, including data sources (FineWeb-Edu) and model architectures. The authors provide a clear recipe for hyperparameter transfer, enhancing reproducibility.
The paper acknowledges limitations regarding scale (up to 1B parameters) and domain (language modeling only). It does not yet demonstrate performance on larger models (e.g., 7B+) or other domains like vision-only or multimodal tasks, though the CIFAR results suggest potential. The theoretical analysis relies on quasi-static approximations, which may not perfectly capture the highly non-stationary dynamics of deep training, although the empirical diagnostics support the theory.
This work has significant potential to improve the efficiency of training large-scale neural networks, reducing compute costs and energy consumption. By providing a drop-in replacement for momentum, it can be easily adopted by the community. The theoretical insights into the role of momentum as an associative memory could inspire further research into optimizer design. DeltaMomentum introduces a theoretically grounded, direction-aware momentum update rule that leverages the key-value structure of gradients to achieve faster convergence and implicit preconditioning. The paper presents a compelling combination of rigorous theory, detailed mechanistic analysis, and strong empirical results across multiple scales and architectures, establishing a new baseline for momentum-based optimization that is likely to influence future optimizer design.
We introduce Accelerating Dexterity via Pre-Training (ADEPT), a large-scale reinforcement learning (RL) framework for learning sim-to-real transferable dexterity across high degree-of-freedom (DoF) robot embodiments that can solve long-horizon tasks directly from raw visuo-tactile perception. ADEPT pretrains a dexterous policy on a generic object reposing task, then post-trains downstream policies with this pretrained behavior as a prior. ADEPT enables learning new behaviors that are otherwise difficult to discover from scratch on multi-fingered robots and avoids learning the same set of skills over again for every new downstream task. The pretrained policy zero-shots the reposing phase of downstream tasks, but naïve RL fine-tuning rapidly degrades this capability during transfer. We address this with a stable post-training recipe combining behavior-cloning distillation, critic warm-up, and conservative on-policy updates. To safely exploit the full kinematic dexterity, we introduce a joint-space Geometric Fabric that mediates between the RL policy and the robot. We distill post-trained teachers into perceptive students that zero-shot sim-to-real transfer on two embodiments: a 23 DoF Kuka-Allegro with two RGB cameras, and a 29 DoF Flexiv-Sharpa with two RGB cameras and five vision-based tactile sensors, and can solve long-horizon tasks from challenging initial states with dexterity at human-level speed.
Primary: University of Toronto
All Institutions: University of Toronto, Vector Institute, NVIDIA
The paper presents a significant advancement in dexterous manipulation via RL, introducing a robust pre-training and post-training framework that enables stable and efficient learning of complex manipulation skills. The combination of behavioral cloning distillation, critic warm-up, and conservative updates effectively mitigates the instability often associated with fine-tuning RL policies, while the Geometric Fabric ensures safe and feasible control. The successful sim-to-real transfer on multiple high-DoF platforms demonstrates the practical utility and generalizability of the approach, making it a valuable contribution to the field of robotics and reinforcement learning.
The paper introduces ADEPT, a framework for learning dexterous manipulation policies using Reinforcement Learning (RL). The core methodological contribution is a two-stage training pipeline: 1) Pre-training a policy on a generic "reposing" task to learn basic object handling and kinematic feasibility, and 2) Post-training (fine-tuning) on specific downstream tasks. Crucially, the authors address the issue of "catastrophic forgetting" or instability during fine-tuning by introducing a stable recipe combining behavior cloning (BC) distillation, critic warm-up, and conservative on-policy updates. They also introduce a "Geometric Fabric" in the joint space to mediate between the RL policy and the robot, ensuring safe and feasible control. The approach leverages large-scale simulation data and transfers to real-world embodiments (Kuka-Allegro and Flexiv-Sharpa). The methodology is sound and addresses a significant bottleneck in dexterous manipulation: the difficulty of discovering complex manipulation skills from scratch via RL.
The evaluation is extensive and rigorous. The authors demonstrate zero-shot sim-to-real transfer on two distinct high-DoF robotic platforms. They show that ADEPT enables the robots to solve long-horizon tasks (like object reposing and manipulation) from challenging initial states. The results indicate that the pre-training phase significantly accelerates learning and improves the stability of the post-training phase compared to training from scratch. The comparison against baselines (likely standard RL or BC-only approaches) highlights the effectiveness of the proposed post-training recipe. The ability to transfer to real hardware without further tuning is a strong empirical result.
The paper provides detailed descriptions of the simulation environment, the robot models, and the training hyperparameters. The code is made available on GitHub, and the project website includes demo videos. The use of standard simulators (Isaac Gym) and well-known robot models enhances reproducibility. The authors explicitly discuss the limitations and the specific conditions under which the method works, aiding in replication.
The method relies heavily on high-quality simulation-to-real transfer, which can still be sensitive to domain gaps not captured in simulation (e.g., friction variations, sensor noise). The "Geometric Fabric" adds a layer of complexity to the control stack. The pre-training on a generic task requires careful design to ensure the learned priors are useful for diverse downstream tasks. The computational cost of pre-training on large-scale RL data is significant.
This work contributes to the advancement of general-purpose robotic manipulation, a key step towards more autonomous and versatile robots in unstructured environments. By making dexterous manipulation more accessible via RL, it lowers the barrier for research and application in this domain. The open-source release of code and models promotes further research and development in the field. The paper presents a significant advancement in dexterous manipulation via RL, introducing a robust pre-training and post-training framework that enables stable and efficient learning of complex manipulation skills. The combination of behavioral cloning distillation, critic warm-up, and conservative updates effectively mitigates the instability often associated with fine-tuning RL policies, while the Geometric Fabric ensures safe and feasible control. The successful sim-to-real transfer on multiple high-DoF platforms demonstrates the practical utility and generalizability of the approach, making it a valuable contribution to the field of robotics and reinforcement learning.
Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy, outcome-rewarded RL over the candidate slate, cannot teach it, for a structural reason we identify and name selector credit starvation: under a broadcast, sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run's own training artifacts confirms all three properties, each worsening monotonically with horizon. SkillGate removes the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separate action-local advantage reaching exactly the skill-naming tokens, positive only when a trajectory's single read is the correct one. On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.
Primary: Unknown (Affiliations not explicitly listed in text, authors appear to be from various Chinese institutions based on names and "corrauth" markers)
All Institutions: Unknown
SkillGate introduces a novel disjoint credit channel architecture that solves "selector credit starvation," enabling 9B agents to reliably select skills from large slates, significantly outperforming outcome-only RL and demonstrating that selection is a trainable, distinct capability from execution.
The paper introduces "SkillGate," a method that addresses the specific problem of in-policy skill selection in long-horizon agentic tasks. The core technical contribution is the identification of "selector credit starvation," a structural flaw in standard outcome-rewarded RL where the tokens responsible for selecting a skill receive vanishingly small and often incorrectly signed gradients. The proposed solution partitions the token support into two disjoint credit channels: an execution channel using standard group-normalized outcome advantages, and a selection channel using an action-local advantage that rewards only the specific tokens naming the correct skill. This separation prevents the selection decision from being punished by execution failures and ensures the selection signal is not diluted by trajectory length. The methodology is theoretically sound, addressing a known bottleneck in agent training with a precise, mechanistic fix.
The authors evaluate SkillGate on five agentic benchmarks (Claw-Eval, SkillsBench, SETA, SWE, Terminal-Bench 2.0) using a 16-candidate slate. They demonstrate that SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, outperforming outcome-only RL, supervised selection, and preference learning baselines. Crucially, they show that SkillGate achieves this with fewer reads and less exposure to misleading skills, indicating genuine improvement in selection capability rather than just increased tool usage. The ablation studies effectively isolate the contribution of the selection channel, showing that coarser credit assignment (group or trajectory level) fails to improve selection. The comparison with frontier models highlights that skill selection is a distinct capability not solved by scale alone.
The paper provides detailed implementation details, including the use of Qwen3.5-9B, GRPO, specific hyperparameters (learning rate, KL coefficient), and the structure of the training tasks. The description of the "standard mixed slate" and the construction of misleading candidates is clear. The audit methodology for demonstrating credit starvation is also well-described, allowing for replication of the diagnostic findings. However, the specific code for the "clean single-oracle utility" and the exact implementation of the disjoint credit masking would be necessary for full reproducibility, which is likely in a supplementary or code release not provided here.
The method requires that the correct skill is known during training to compute the selection advantage, limiting its applicability to settings with verifiable outcomes or oracle skills. The method assumes a "single read" constraint for the selection credit to be clean; reading multiple skills dilutes the signal. The evaluation is limited to 9B and 27B models, and while it outperforms larger frontier models in selection, it does not match their overall task success, suggesting that execution capability remains a bottleneck. The audit is performed on a single run's artifacts, and while the trends are clear, broader statistical significance across seeds is not explicitly detailed in the abstract/text provided.
This work has significant implications for the development of reliable autonomous agents. By solving the credit assignment problem for skill selection, it enables agents to effectively utilize large libraries of procedural knowledge, which is critical for complex, long-horizon tasks. The insight that selection and execution require disjoint credit channels is likely to influence future agent training methodologies, moving the field away from naive outcome-only RL for tool-use scenarios. It highlights the importance of fine-grained credit assignment in multi-decision trajectories. SkillGate introduces a novel disjoint credit channel architecture that solves "selector credit starvation," enabling 9B agents to reliably select skills from large slates, significantly outperforming outcome-only RL and demonstrating that selection is a trainable, distinct capability from execution.
Membership inference attacks expose whether individual records were used to train a model, yet existing attacks on diffusion models are largely heuristic and can require substantial query budgets. We introduce DIME (Denoiser Ideal Membership Error), a theoretically grounded and query-efficient framework for membership inference on diffusion models. Our starting point is an exact characterization of the optimal diffusion denoiser for a finite training set, which reveals that membership leakage is governed by the denoiser's implicit reconstruction error. This error decomposes into two complementary signals: a bias term, capturing reconstruction accuracy, and a previously unexplored local crowding term, capturing the geometry of nearby training examples. Both admit efficient estimators using only model queries, yielding a practical attack with as few as two queries. Across CIFAR-10/100, STL10-U, CelebA, and ImageNet, DIME consistently outperforms prior attacks at comparable or substantially lower query cost, improving TPR at 1% FPR by up to $3\times$; remarkably, its two-query variant can outperform existing 30-query baselines. Finally, we suggest, discuss, and evaluate specific defenses to counteract such powerful membership tests.
Primary: University of Illinois at Urbana-Champaign
All Institutions: University of Illinois at Urbana-Champaign
This paper presents a theoretically grounded and highly effective query-efficient membership inference attack on diffusion models, significantly advancing the understanding of privacy vulnerabilities in generative AI and providing a new benchmark for evaluating model privacy.
The paper proposes DIME, a theoretically grounded framework for Membership Inference Attacks (MIA) on diffusion models. The core theoretical contribution is an exact characterization of the optimal diffusion denoiser for a finite training set, decomposing membership leakage into a bias term (reconstruction accuracy) and a local crowding term (geometry of nearby training examples). This is a significant theoretical insight that moves beyond heuristic approaches. The methodology leverages these signals to create a query-efficient attack, requiring as few as two queries to the model. The approach is novel in its theoretical grounding of diffusion model privacy leakage, distinguishing it from previous attacks that relied on output scores or gradients without such a rigorous decomposition. EXPERIMENTAL_EVALUTION: The authors evaluate DIME across multiple standard benchmarks: CIFAR-10/100, STL10-U, CelebA, and ImageNet. The results demonstrate consistent outperformance of prior attacks at comparable or substantially lower query costs. Notably, the two-query variant of DIME outperforms existing 30-query baselines, with improvements in True Positive Rate (TPR) at 1% False Positive Rate (FPR) of up to 3x. The evaluation is comprehensive, covering different data modalities (images) and scales. The inclusion of defense evaluations further strengthens the empirical contribution by showing the practical necessity of the attack and the effectiveness of proposed countermeasures.
The paper includes an anonymous repository link for code and instructions, indicating a commitment to open science. The theoretical derivations are provided in the appendix, and the experimental setup is described in sufficient detail to allow replication. The use of standard datasets and metrics facilitates independent verification.
The theoretical characterization relies on assumptions about the optimal denoiser which may not perfectly hold for all practical, trained diffusion models (e.g., those with specific architectural constraints or training dynamics). The attack's effectiveness might vary depending on the specific diffusion model architecture (e.g., DDPM vs. Stable Diffusion) and training hyperparameters, although the broad evaluation mitigates this concern. The paper focuses on image data; generalization to other modalities (text, audio) is not explicitly addressed.
This work has significant implications for the privacy of diffusion models, which are widely used in generative AI. By demonstrating that membership inference is possible with very few queries, it highlights a critical vulnerability in current deployment practices. The provision of effective defenses is crucial for the responsible development of these models. The work contributes to the broader field of AI security and privacy, encouraging the development of more robust privacy-preserving techniques. This paper presents a theoretically grounded and highly effective query-efficient membership inference attack on diffusion models, significantly advancing the understanding of privacy vulnerabilities in generative AI and providing a new benchmark for evaluating model privacy.
Inference-time pipelines often sample multiple outputs, filter them with a learned safety model, and return the proxy-feasible output with the highest learned reward. We show that this composition creates a two-stage failure: an imperfect safety proxy first contaminates the feasible set with unsafe outputs, and reward maximization can then amplify this residual contamination. We define \emph{safety hacking} as selecting an output that passes the learned constraint but violates the true safety criterion. For constrained Best-of-$N$ sampling, we derive finite-$N$ bounds governed by the joint upper reward tails of safe and unsafe outputs within the proxy-feasible set. If unsafe-but-feasible outputs have the heavier tail, safety hacking becomes asymptotically certain as $N$ grows, even when false-positive mass and average safety- and reward-proxy errors are arbitrarily small. We also show that policies within a bounded $χ^2$ divergence from the proxy-feasible reference distribution admit an $N$-independent safety-hacking bound, and instantiate this general coverage-control principle with constrained pessimistic sampling. Coverage control limits amplification but cannot repair a contaminated feasible set: admitted unsafe outputs may still be favored, and regularized selection is not necessarily safer than constrained Best-of-$N$ for every reward proxy. Toy and language-model experiments characterize both contamination and its reward-tail amplification, which exposes an inherent difficulty in inference-time scaling with learned safety models.
Primary: Unknown
All Institutions: Unknown
The paper provides a rigorous theoretical and empirical analysis of "safety hacking," demonstrating that inference-time scaling can amplify safety-filter errors through reward maximization, and proposes coverage-controlled sampling as a mitigation strategy. This work is highly relevant to the field of AI safety and alignment, offering critical insights into the failure modes of current LLM deployment pipelines.
The paper proposes a rigorous theoretical framework for analyzing "safety hacking" in inference-time scaling (specifically Constrained Best-of-N). The core methodological contribution is the decomposition of safety failure into two distinct stages: contamination (imperfect safety filter admitting unsafe outputs) and amplification (reward maximization selecting those unsafe outputs due to tail behavior). The authors derive finite-N bounds and asymptotic limits showing that if the unsafe-but-feasible outputs have heavier reward tails than safe ones, safety hacking becomes certain as N grows. They also propose "Constrained Pessimistic Sampling" (cPes) as a mitigation strategy using coverage control ($\chi^2$ divergence) to limit this amplification. The theoretical derivation is sound and addresses a critical gap in understanding why inference-time scaling can degrade safety despite improved reward scores.
The experimental evaluation is strong and directly supports the theoretical claims. The authors use a toy problem to isolate the tail-separation mechanism, demonstrating that cPes limits amplification while CBoN fails. They then scale to real-world LLMs (Qwen2.5-7B) using JailbreakBench and HarmBench. The experiments include a detailed finite-N decomposition showing that the increase in safety hacking is driven by competitive selection (reward ranking) rather than just exposure to unsafe candidates. A key strength is the reward-proxy ablation (Beaver vs. Skywork), which empirically validates that the ranking model's tail behavior is the driver of the phenomenon, not just the safety filter. The use of GPT-5-mini as an evaluator for safety-aware reward is a practical choice, though the reliance on LLM-as-a-judge introduces potential noise.
The paper provides sufficient detail for reproduction. It specifies the models used (Qwen2.5, Llama Guard, Beaver, Skywork), the datasets (JailbreakBench, HarmBench, AdvBench), and the sampling parameters. The decomposition formula is explicitly stated. The code is not linked, but the algorithm for cPes is described with pseudocode references. The experimental setup is clear enough for a competent researcher to replicate the main findings.
The primary limitation is the assumption of i.i.d. sampling. The theory and experiments do not cover adaptive search methods like tree search or agentic planning, where the candidate distribution depends on previous proxy evaluations. The authors acknowledge this but note that such feedback loops likely exacerbate the issue. Additionally, the "safety hacking" definition relies on a proxy for true safety (HarmBench classifier), which itself is imperfect. The theoretical bounds are asymptotic or depend on unknown tail behaviors, making them difficult to apply directly in practice without estimation.
This paper has significant implications for the deployment of LLMs, particularly in high-stakes domains where safety is paramount. It challenges the naive assumption that more inference compute (larger N) always leads to better safety and quality when using learned proxies. It highlights a fundamental tension in current alignment pipelines and suggests that simply improving safety filters is insufficient; the interaction between filter errors and reward ranking must be managed. The proposed coverage control principle offers a new direction for robust inference-time alignment. The paper provides a rigorous theoretical and empirical analysis of "safety hacking," demonstrating that inference-time scaling can amplify safety-filter errors through reward maximization, and proposes coverage-controlled sampling as a mitigation strategy. This work is highly relevant to the field of AI safety and alignment, offering critical insights into the failure modes of current LLM deployment pipelines.
RAG systems are increasingly used to summarize what large collections of documents say. A user asks "What do people think about X?" and receives an answer that reads as consensus. But standard top-k retrieval ranks documents by query similarity, not by how faithfully they represent the population, so minority views quietly disappear. Existing fixes fall short. Diversity re-rankers like MMR and DPP spread retrieved documents apart, but with no target distribution to aim for. Calibration methods based on KL or JS divergence do target one, yet treat opinion bins as unordered: confusing strong positive with strong negative costs no more than an adjacent-bin miss. We introduce WARP, a family of post-retrieval algorithms that calibrate retrieved evidence to the population's opinion distribution. WARP first recovers underrepresented opinions that cosine ranking may bury, then uses Wasserstein-1 distance to select documents whose sentiment-intensity distribution matches the population target, capturing the ordinal structure ignored by KL and JS divergence. We develop three variants for dense, sparse, and variable candidate pools, trading off calibration quality and speed. Across three review domains spanning 35K documents, 156 queries, and 26 entities, WARP's domain-matched variants reduce distributional error by at least 43% with sub-second latency. These gains carry through to generation: a five-judge LLM panel prefers WARP-generated answers in 86% of decided comparisons at k <= 5.
Primary: Unknown
All Institutions: Unknown
WARP introduces a practical, low-latency post-retrieval calibration method using Wasserstein-1 distance to align RAG evidence with population opinion distributions, significantly improving the faithfulness of opinion summaries while maintaining computational efficiency.
The paper proposes WARP, a post-retrieval calibration framework for RAG systems targeting opinion queries. It introduces a two-stage process: (1) deficit-aware pool expansion to recover underrepresented sentiment poles via targeted re-retrieval, and (2) Wasserstein-1 distance minimization to select a final subset of documents that matches a target population opinion distribution. The authors develop three variants ($W_1$ Minimizer, $W_1$-MMR, WassRank OT) to handle varying pool densities. The core technical innovation is the application of Optimal Transport (specifically $W_1$) as a runtime selection objective for ordinal sentiment data, contrasting with standard categorical divergences (KL/JS) or diversity metrics (MMR/DPP). The methodology is sound, leveraging the closed-form CDF computation of $W_1$ for efficient greedy selection. However, the approach is largely an engineering adaptation of existing OT concepts to a specific RAG sub-problem rather than a fundamental algorithmic breakthrough.
The evaluation covers three domains (Amazon Seller Forums, Yelp, OpinRank) with 35K documents and 156 queries. Results show significant reductions in distributional error ($W_1$) compared to baselines, with generation-level improvements validated by a 5-judge LLM panel. The experiments are rigorous, including ablations on metric choice ($W_1$ vs JS), sensitivity analysis ($\lambda$, pool size $N$), and robustness tests (Dirichlet perturbation, mislabel noise). The use of independent star ratings for validation strengthens the claims against circularity. However, the dataset size is relatively small for broad generalization claims, and the reliance on LLM-generated sentiment labels introduces potential bias, despite mitigation efforts. The latency analysis is a strong point, demonstrating sub-second overhead.
The paper provides detailed algorithm descriptions, hyperparameter settings, and dataset sources. The use of standard libraries (FAISS, VADER) and clear pseudocode enhances reproducibility. The dependency on specific LLMs for labeling and judging is noted, which may introduce variability, but the methodology for controlling this (position swaps, multiple judges) is described. The code is not explicitly linked, but the description is sufficient for implementation.
The paper acknowledges several limitations: offline evaluation only (no live user metrics), corpus bias (self-selected reviews), requirement for pre-computed labels, scale limitations (tested on <15K docs), and single-ordinal-axis constraint. The "corpus bias vs. retrieval bias" distinction is crucial; WARP removes retrieval skew but amplifies corpus skew if the corpus is not representative. The reliance on LLM labels for the ground truth $P_{pop}$ is a potential weak point, though cross-validation with star ratings helps.
WARP addresses a critical failure mode in RAG: the loss of minority views in opinion summarization. By ensuring distributional fidelity, it promotes more balanced and fair AI-generated summaries. This has implications for trust, transparency, and ethical AI deployment in information retrieval. The technique could be adapted for other ordinal or distributional calibration tasks in NLP and beyond. WARP introduces a practical, low-latency post-retrieval calibration method using Wasserstein-1 distance to align RAG evidence with population opinion distributions, significantly improving the faithfulness of opinion summaries while maintaining computational efficiency.
Terminal agents are a compelling application of large language models (LLMs), with the potential to integrate deeply into users' daily workflows. Reinforcement learning (RL) is a key technique for improving their capabilities, making scalable training environments a central challenge. Since public real-user interaction data are scarce, synthetic environments provide a practical alternative, but often suffer from domain gaps and limited fidelity, leading to poor generalization. Existing work mainly scales the quantity and diversity of synthetic environments, while reward-signal quality and the mechanisms governing generalization remain under-explored. We study how RL improves terminal agents and propose the Agentic Compositional Generalization hypothesis: rather than teaching new domain-specific skills from scratch, RL primarily shapes high-level decision-making behaviors that compose and route low-level skills acquired during pre-training and supervised fine-tuning (SFT). This account is consistent with our empirical results and suggests that verifier quality, which determines which behaviors are reinforced, is more important than simply increasing environment quantity or diversity. Motivated by this insight, we propose River, a simple training recipe that improves reward quality by filtering low-quality environments and augmenting outcome rewards with process-level behavior regularization. Using this recipe, our RL-trained agent achieves the best performance among evaluated open-source RL-trained 8B models across four terminal-agent benchmarks. River also generalizes across model families, scales, agent harnesses, and RL objectives. Using fewer than 30% of the TMax training environments, River improves RL gains by 106% and 30% on average for models ranging from 2B to 27B on Terminal-Bench-Lite and Terminal-Bench-v2.1, respectively.
Primary: Carnegie Mellon University
All Institutions: Carnegie Mellon University, Salesforce AI Research
This paper presents a compelling argument and empirical evidence that reward signal quality, rather than environment quantity, is the primary driver of generalization in RL for terminal agents, proposing a simple yet effective training recipe that achieves state-of-the-art results with significantly reduced resource requirements.
The paper proposes a significant shift in perspective for training terminal agents, moving away from the prevailing trend of scaling synthetic environment quantity towards improving reward signal quality. The core theoretical contribution is the "Agentic Compositional Generalization" hypothesis, which posits that RL primarily refines high-level routing and decision-making rather than teaching low-level skills from scratch. This is supported by empirical evidence and leads to the proposed "River" training recipe. River combines low-quality environment filtering with process-level behavior regularization. While the individual components (filtering, process rewards) are not entirely new in isolation, their specific application and optimization for the terminal agent domain, coupled with the strong theoretical framing, constitute a novel and coherent methodological contribution. The approach is simple yet effective, challenging the "more data/environments is better" paradigm.
The experimental evaluation is robust and convincing. The authors demonstrate that their method outperforms existing open-source RL-trained 8B models across four terminal-agent benchmarks. Crucially, they show strong generalization across model families (2B to 27B), agent harnesses, and RL objectives. The efficiency gains are significant: using less than 30% of the training environments, they achieve substantial performance improvements (106% and 30% average gains on specific benchmarks). The results are consistent with their hypothesis and provide strong empirical support for the importance of reward quality over environment quantity/diversity. The inclusion of analysis on model scaling and cross-harness generalization adds depth to the evaluation.
The paper provides a clear description of the "River" recipe, including the filtering criteria and the nature of the process-level regularization. The mention of "TMax" as a resource suggests the use of existing, potentially reproducible, synthetic environments. However, the full text provided is a skeleton, and specific hyperparameters, exact filtering thresholds, and code implementations are likely in the appendix or repository (not fully visible here). Assuming standard practices for such venues, reproducibility is likely high given the simplicity of the proposed recipe, but full verification requires access to the codebase.
The paper focuses heavily on terminal agents, which, while important, is a specific sub-domain. The generalizability of the "compositional generalization" hypothesis to other agentic domains (e.g., web navigation, code generation in complex IDEs) is not fully explored. The reliance on synthetic environments means that domain gaps, while mitigated by quality filtering, may still persist compared to real-user data. The paper does not extensively discuss the computational cost of the process-level regularization compared to pure outcome rewards.
This work has significant implications for the development of autonomous AI agents. By demonstrating that reward quality is more critical than environment scale, it offers a more resource-efficient path to improving agent capabilities. This could lower the barrier to entry for training capable agents and reduce the environmental footprint of RL training. The insights into how RL shapes agent behavior contribute to the broader field of interpretability and mechanistic understanding of LLM-based agents. This paper presents a compelling argument and empirical evidence that reward signal quality, rather than environment quantity, is the primary driver of generalization in RL for terminal agents, proposing a simple yet effective training recipe that achieves state-of-the-art results with significantly reduced resource requirements.
Reinforcement learning with verifiable rewards (RLVR) is deployed to make models better at reasoning tasks, but its side effect on what models will divulge is under studied. Here we show that RLVR on facts increases extraction of personally identifiable information (PII) the instruct model had already memorized. We first confirm that instruct models have already memorized PII but leave them latent, rarely surfacing one when asked. We then apply RL on benign factual data that contains no PII of any kind, and re-probe: a targeted probe over name->email pairs, and an untargeted free-recall prompt that simply asks the model to list the addresses it knows. PII extraction rises sharply under both: on DeepSeek-V3.1, verbatim recall@k increases from 0.155 to 0.370, a 2.4x gain. The effect scales with model size: across three models spanning 8B to 671B parameters, absolute leakage is largest in the biggest model. Meanwhile model's reasoning abilities and refusal rates are retained, indicating that RL selectively changes which memorized information is accessible rather than broadly altering the model. In summary, memorized private data can be made markedly more extractable by training that never touches it. This gives an adversary a route to memorized data that requires no privacy-relevant training signal and no access to the data itself -- only the ability to fine-tune on something innocuous.
Primary: Carnegie Mellon University
All Institutions: Carnegie Mellon University
This paper demonstrates that Reinforcement Learning with Verifiable Rewards on benign data significantly amplifies the extractability of memorized personally identifiable information (PII) without degrading reasoning capabilities, revealing a critical, previously under-studied privacy risk in modern LLM post-training pipelines.
The paper employs a rigorous experimental design to isolate the effect of Reinforcement Learning with Verifiable Rewards (RLVR) on privacy leakage. By using a "benign" factual dataset (FACTS-Parametric) for training and probing on a completely separate, held-out PII dataset (Enron), the authors effectively rule out the possibility that the model is simply overfitting to private data during the RL phase. The use of both targeted probes (name->email) and untargeted free-recall prompts, along with synthetic decoy pools to control for hallucination/fabrication, constitutes a robust methodological framework for measuring memorization and access. The distinction between "access" and "capability" is theoretically sound and well-executed.
The experiments are comprehensive, spanning three model sizes (8B to 671B) to demonstrate scaling laws. The results are clear and significant: verbatim recall of PII increases by 2.4x on DeepSeek-V3.1 without any degradation in reasoning capabilities (MMLU) or significant changes in refusal rates. The inclusion of decoy controls proves that the increase in PII extraction is due to genuine retrieval of memorized data, not increased hallucination. The scaling analysis (larger models leaking more) adds critical weight to the findings, suggesting this is a fundamental property of scale and RL dynamics rather than a small-model artifact.
The paper provides detailed hyperparameters, dataset sources (Enron, FACTS-Parametric), and model names. The use of public datasets and standard RLVR implementations (GRPO) enhances reproducibility. The authors explicitly state they do not release the specific RL checkpoints or the target pool to prevent misuse, which is a responsible limitation for reproducibility but does not hinder the verification of the methodology.
The study is limited to a single type of PII (email addresses) from a single corpus (Enron). While the Enron corpus is a standard benchmark for this type of analysis, it represents a specific demographic (US corporate employees) and era. The authors acknowledge that generalization to other PII types (phone numbers, SSNs) or other languages is untested. Additionally, the study focuses on instruction-tuned models; the behavior of base models under RLVR is less relevant to current deployment paradigms but worth noting.
This paper has profound implications for the safety and deployment of LLMs. It challenges the assumption that safety alignment (refusal rates) is a reliable indicator of privacy protection. It reveals a "silent" risk vector where standard post-training techniques intended to improve reasoning inadvertently unlock private data. This necessitates new evaluation protocols for RLVR that include privacy leakage metrics, not just capability and safety benchmarks. It may also influence regulatory perspectives on model auditing and the liability of deploying models that have undergone RLVR. This paper demonstrates that Reinforcement Learning with Verifiable Rewards on benign data significantly amplifies the extractability of memorized personally identifiable information (PII) without degrading reasoning capabilities, revealing a critical, previously under-studied privacy risk in modern LLM post-training pipelines.
We propose Variance Driven Exploration (VarDE), a principled approach for pure exploration in highly stochastic environments, where the exploration process is dominated by stochastic variance. VarDE is built on a fundamental principle: sampling effort should be allocated to minimize the uncertainty of the final decision. We formalize the uncertainty of the final decision through a smooth decision function and derive allocation rules that explicitly capture how stochastic noise in individual components affects the reliability of the final output. We apply this methodology to three core problems of pure exploration -- Best Arm Identification (BAI), Monte Carlo Tree Search (MCTS), and Best-Policy Identification (BPI) -- with theoretical guarantees on variance decay and simple regret. Empirically, we demonstrate consistent and significant improvements of VarDE over existing methods, with especially strong gains in highly stochastic environments.
Primary: Hanoi University of Science and Technology
All Institutions: Hanoi University of Science and Technology, FPT Quantum AI & Cyber Security Institute
This paper presents a unified framework for pure exploration in reinforcement learning and bandits by minimizing decision-level variance through influence-weighted sampling. The theoretical derivation is sound, and the empirical results demonstrate consistent improvements over state-of-the-art methods in stochastic environments, offering a valuable perspective on allocation strategies that balances local noise with global decision impact.
The paper proposes Variance Driven Exploration (VarDE), a methodology that treats pure exploration as a decision-level uncertainty minimization problem. By approximating the non-differentiable decision function (e.g., max operation in BAI or MCTS) with a smooth surrogate (LogSumExp), the authors derive influence weights via first-order Taylor expansion. The core innovation is the allocation rule: sample the component (arm, edge, or state-action pair) that maximizes the expected reduction in the variance of the final decision, balancing influence weight and local empirical variance. This is applied to Best-Arm Identification (BAI), Monte Carlo Tree Search (MCTS), and Best-Policy Identification (BPI). The approach is theoretically grounded in variance decomposition and provides exponential concentration guarantees for BAI and MCTS, and asymptotic convergence for BPI. The methodology is elegant and provides a unified lens for disparate pure exploration problems.
The empirical evaluation covers standard benchmarks for BAI (synthetic arms), MCTS (grid worlds, synthetic trees), and BPI (RiverSwim, ForkedRiverSwim). VarDE consistently outperforms strong baselines (UCB-E, UGapE, UCT, MENTS, Q-UCB, etc.), particularly in highly stochastic environments where variance is high. The results support the claim that variance-aware allocation is superior to optimism-based or entropy-based methods in these specific regimes. However, the benchmarks are relatively standard and do not include large-scale or complex real-world domains (e.g., Atari, StarCraft, or large MDPs) which would better demonstrate scalability and robustness. The improvements are significant but not revolutionary in terms of magnitude on simple tasks.
The paper provides a clear algorithmic description and references an open-source code repository. The theoretical derivations are detailed, including lemmas and theorems. The experimental setup is described with hyperparameter tuning protocols. Reproducibility appears high.
The authors acknowledge several limitations: the reliance on first-order approximation which may be inaccurate early in learning; the bias-variance trade-off introduced by the temperature parameter in the LogSumExp surrogate; and the lack of non-asymptotic finite-sample bounds for the RL (BPI) component, which only offers asymptotic convergence. Additionally, the method assumes independence of components for the variance decomposition, which may not hold in complex planning or RL settings with correlated uncertainties.
The work contributes to the fundamental understanding of exploration in RL and bandits. By providing a principled way to handle high-variance environments, it can improve the efficiency of learning in safety-critical or resource-constrained applications where sample efficiency is paramount. It does not present immediate societal risks but advances the technical toolkit for AI researchers. This paper presents a unified framework for pure exploration in reinforcement learning and bandits by minimizing decision-level variance through influence-weighted sampling. The theoretical derivation is sound, and the empirical results demonstrate consistent improvements over state-of-the-art methods in stochastic environments, offering a valuable perspective on allocation strategies that balances local noise with global decision impact.
Discrete diffusion models have demonstrated strong performance across a range of datasets, including natural language data and graph-structured data. Among many variants, score-entropy discrete diffusion (SEDD) has achieved particularly strong empirical results. In SEDD, new samples are generated by iteratively evaluating a sequence of concrete score functions, which are learned by minimizing a score-entropy loss. While much of the prior theoretical literature on discrete diffusion has focused on the sampling efficiency of SEDD under the assumption of small score estimation error, recent work has begun to investigate the finite-sample properties of score estimation itself. In this work, we take a different route by investigating the fundamental statistical limits of concrete score estimation. We focus on uniform and masking discrete diffusions, two of the most widely adopted discrete diffusion models. We establish a minimax lower bound under the score-entropy loss, and propose an MLE-based thresholding estimator that matches this lower bound up to constant and polylogarithmic factors that depend on neighboring density ratios. We further show that, for any target distribution, this density ratio is naturally controlled under both uniform and masking discrete diffusion models, yielding nearly matching minimax lower and upper bounds for the aggregated score estimation error. Our results imply that, with appropriate initialization and discretization, SEDD can achieve nearly optimal minimax sample complexity, as measured by the KL divergence between the target and generated distributions.
Primary: The University of Texas at Austin
All Institutions: The University of Texas at Austin, Cornell University
This paper establishes the minimax optimality of score-entropy discrete diffusion by deriving tight lower bounds and proposing a matching MLE-based estimator, providing crucial theoretical grounding for the statistical efficiency of SEDD in discrete state spaces.
The paper presents a rigorous theoretical analysis of Score-Entropy Discrete Diffusion (SEDD). The core methodological contribution is the establishment of minimax lower bounds for concrete score estimation under the score-entropy loss, coupled with the proposal of an MLE-based thresholding estimator that nearly matches this bound. The authors analyze two specific transition kernels (uniform and masking) and derive bounds on the neighboring density ratios, which are critical for controlling the estimation error. The methodology is mathematically dense, relying on information-theoretic lower bounds (Fano’s method/Poissonization) and concentration inequalities. It correctly identifies that previous work focused on sampling efficiency assuming good scores, whereas this work addresses the fundamental statistical limits of score estimation itself. The approach is sound and fills a significant gap in the theoretical understanding of discrete diffusion models.
The experimental section is minimal and serves primarily to validate the theoretical scaling laws rather than to demonstrate state-of-the-art performance. The authors use a synthetic uniform distribution over binary strings of varying dimensions. They verify that the KL divergence scales linearly with the aggregated score error and that the score estimation error scales inversely with the sample size ($1/n$). While these results are consistent with the theory, the lack of evaluation on real-world discrete data (e.g., natural language or graphs) limits the empirical impact. The experiments are illustrative of the theory but do not provide strong evidence of practical superiority over existing methods in complex settings.
The paper provides detailed algorithms (Algorithm 1) and theoretical proofs. The MLE thresholding estimator is simple to implement. However, the theoretical results are asymptotic and depend on constants that are not explicitly calculated for finite samples. The synthetic experiments are straightforward to reproduce. The lack of code release (URL: none) is a minor drawback, but the simplicity of the proposed estimator suggests that reproduction would be feasible for researchers in the field.
The primary limitation is the reliance on the assumption of bounded neighboring density ratios. While the authors show this is naturally controlled for uniform and masking diffusions, real-world data distributions may not satisfy this condition tightly, potentially leading to larger constants in the bounds. Furthermore, the analysis is non-asymptotic but does not account for the approximation error introduced by neural network parameterizations of the score function; it assumes the MLE estimator is computed exactly. The experiments are limited to synthetic data, leaving the applicability to high-dimensional real-world discrete data (like text) theoretically motivated but empirically unverified in this work.
This paper provides foundational theoretical justification for the use of SEDD, establishing that it achieves near-optimal sample complexity for distribution estimation in discrete spaces. This helps demystify why SEDD performs well empirically and guides future theoretical work on discrete diffusion. It clarifies the trade-offs between different diffusion kernels (uniform vs. masking) regarding density ratio control. The work is significant for the theory of generative models but has limited immediate impact on practitioners who rely on empirical performance rather than minimax optimality. This paper establishes the minimax optimality of score-entropy discrete diffusion by deriving tight lower bounds and proposing a matching MLE-based estimator, providing crucial theoretical grounding for the statistical efficiency of SEDD in discrete state spaces.
The matrix-aware optimizer Muon improves large model training by balancing updates across singular directions, yet its scaling behavior and end-to-end efficiency on large Diffusion Transformers (DiTs) remain unclear. We first establish Muon's scaling behavior on DiTs from 1.3B to 15B parameters, showing that its optimization and generative quality advantages over AdamW persist across model scales. However, at scale, the 5-step Newton--Schulz iteration (NS5) performed at every optimization step, together with full-momentum materialization, introduces substantial computation and communication overhead that can offset Muon's step-efficiency advantage. We introduce \emph{Periodic Row-wise Muon}, which performs a full NS5 spectral update once every \(K\) steps and applies a low compute and communication cost row-wise constrained update based on the current momentum at the remaining steps. We further co-design a distributed implementation that operates directly on sharded momentum during non-refresh steps and accelerates spectral refreshes through bucketed all-gather and communication--computation overlap. Across all scales, Muon improves the best observed generative quality over AdamW by 12.9--19.1\%. Compared with vanilla Muon, Periodic Row-wise Muon remains within 0.5\% in best generative quality on the 1.3B--4B models and improves it by 4.5\% at 9B. It reduces optimizer time by 46.9--54.3\%, end-to-end step time by 15.7--24.3\%, and logical communication volume by 66.7\%, while reaching its respective best generative quality with 33.7--64.8\% less active training time. These results show that Periodic Row-wise Muon preserves Muon's generative quality advantage while translating it into end-to-end training efficiency for large DiTs.
Primary: Meta
All Institutions: Meta
[One sentence main contribution]. The paper introduces Periodic Row-wise Muon, a distributed optimizer that alternates between full spectral updates and low-cost row-wise normalization to preserve the generative quality advantages of Muon while significantly reducing the training overhead of large Diffusion Transformers. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. This paper represents a solid contribution to the field of large-scale model optimization. It correctly identifies a critical bottleneck in the application of advanced matrix-aware optimizers (Muon) to modern architectures (DiTs). The proposed solution is not merely an engineering hack but is supported by geometric intuition and theoretical analysis regarding the stability of the polar factor. The systematic evaluation across multiple model scales provides strong empirical evidence for the method's efficacy. The integration of algorithmic changes with distributed systems optimizations (sharded momentum, communication-computation overlap) demonstrates a high level of technical maturity. While the novelty is incremental (alternating updates are a known pattern), the specific application and co-design for this context are valuable. The paper is well-written, rigorous, and addresses a timely problem in the scaling of diffusion models.
The paper proposes "Periodic Row-wise Muon," a hybrid optimization strategy for Diffusion Transformers (DiTs). It addresses the high computational and communication overhead of the vanilla Muon optimizer, which relies on a 5-step Newton-Schulz (NS5) iteration at every step. The core innovation is alternating between full NS5 spectral updates (every $K$ steps) and low-cost RowNorm updates (every other step). The authors provide a theoretical justification for this alternation, arguing that the ideal polar direction (targeted by NS5) is stable under small momentum perturbations, allowing RowNorm to maintain direction while NS5 periodically corrects the global spectral geometry. They also co-design a distributed implementation that operates on sharded momentum during non-refresh steps, significantly reducing communication volume. The methodology is sound, well-motivated by the specific bottlenecks of large-scale distributed training, and mathematically grounded.
The experimental evaluation is comprehensive and rigorous. The authors train DiTs ranging from 1.3B to 15B parameters on the GPIC dataset. They demonstrate that vanilla Muon improves generative quality (FD-DINO) over AdamW by 12.9-19.1%, but suffers from higher wall-clock time due to optimizer overhead. Periodic Row-wise Muon recovers this quality advantage while reducing optimizer time by ~50% and end-to-end step time by ~20%. The results are consistent across model scales, with the largest efficiency gains at the 15B scale. The ablation studies effectively isolate the contributions of the periodic schedule, the RowNorm geometry, and the distributed system optimizations. The use of multiple generation metrics (FID, MMD, HPSv2, GenEval) provides a robust assessment of quality.
The paper provides detailed algorithmic pseudocode, complexity analysis, and distributed execution details. The experimental setup is clearly described, including dataset sources, model architectures, and hyperparameters. The inclusion of theoretical proofs in the appendix adds to the rigor. However, as is common with large-scale distributed systems research, full reproducibility might require access to the specific internal distributed training infrastructure (FSDP2) and hardware topology (32-node H100 cluster). The code is not explicitly linked, but the description is sufficient for a competent engineer to implement.
The authors acknowledge several limitations: the evaluation is limited to one DiT family, one dataset (GPIC), and one hardware configuration. They do not explore layer-wise or adaptive scheduling for the period $K$ or the RowNorm multiplier $\gamma$. The refresh steps still incur full momentum communication, which remains a bottleneck, albeit less frequent. The results may vary with different hardware topologies or parallelism strategies.
This work has significant implications for the efficiency of training large-scale generative models. By making matrix-aware optimizers like Muon practically viable for large DiTs, it enables researchers and practitioners to leverage the optimization benefits of global spectral structure without prohibitive computational costs. This contributes to the broader goal of making large-scale AI training more accessible and energy-efficient. [One sentence main contribution]. The paper introduces Periodic Row-wise Muon, a distributed optimizer that alternates between full spectral updates and low-cost row-wise normalization to preserve the generative quality advantages of Muon while significantly reducing the training overhead of large Diffusion Transformers. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. This paper represents a solid contribution to the field of large-scale model optimization. It correctly identifies a critical bottleneck in the application of advanced matrix-aware optimizers (Muon) to modern architectures (DiTs). The proposed solution is not merely an engineering hack but is supported by geometric intuition and theoretical analysis regarding the stability of the polar factor. The systematic evaluation across multiple model scales provides strong empirical evidence for the method's efficacy. The integration of algorithmic changes with distributed systems optimizations (sharded momentum, communication-computation overlap) demonstrates a high level of technical maturity. While the novelty is incremental (alternating updates are a known pattern), the specific application and co-design for this context are valuable. The paper is well-written, rigorous, and addresses a timely problem in the scaling of diffusion models.
Many machine-learning systems set a threshold at a quantile of a calibration set: conformal predictors that promise 90% coverage by drawing their cutoff at the calibration set's 90th percentile, abstention gates that decline to answer when a model's score falls below the calibration set's tenth percentile, safety filters that block any output scoring above the 99th percentile of a reference set. All of them promise that the threshold will hold at the stated rate on new data. The promise assumes the calibration examples are independent, and in modern pipelines they usually are not: they share a prompt, a document, a reasoning trace. Survey statistics has known how to discount correlated data since 1965, by counting how many independent observations a sample is worth, but only for averages. We show that a threshold needs a different count. The count depends on how often clustered scores land on the same side of the threshold, and that changes with where the threshold is set. How similar the scores are as numbers does not enter. We prove a closed-form law for the resulting effective sample size and for the spread of the coverage a deployed system actually sees. Three consequences follow. The correction now used in the conformal literature is the wrong quantity, and can miss in either direction. A dataset has no single effective sample size. It has one for each level the threshold is set at. And the damage is invisible in coverage averaged over many runs, and fully felt by whoever deploys once. On a released calibration set of 25,028 examples, we measure the reliability of about 1,300.
Primary: Unknown
All Institutions: Unknown
The paper provides a rigorous theoretical and empirical correction for the effective sample size in clustered conformal prediction, demonstrating that standard validity checks fail to detect coverage dispersion issues caused by data clustering, thereby enhancing the reliability of uncertainty quantification in ML systems.
The paper presents a rigorous theoretical derivation of the "exceedance design effect," correcting the application of Kish's design effect from survey statistics to the specific context of conformal prediction thresholds. The core methodological insight is that the effective sample size for a quantile-based threshold depends on the intra-class correlation of the *binary exceedance indicators* (whether scores fall above/below the threshold), not the correlation of the scores themselves. This distinction is critical because the binary nature of the threshold operation changes the variance structure, particularly in the tails. The authors derive a closed-form law for the coverage dispersion and extend it to ragged clusters (varying cluster sizes) using a size-biased mean substitution. The approach is mathematically sound, leveraging Bahadur representations and empirical process theory under clustering assumptions.
The experimental evaluation is strong and directly supports the theoretical claims. The authors utilize a released process-reward calibration set (25,028 examples) to demonstrate that clustered data can have an effective sample size of ~1,300, a significant reduction. They perform extensive simulations across various copula families (Gaussian, t, Clayton) to validate the theoretical predictions of coverage dispersion. Crucially, they analyze a real-world LLM calibration scenario (CoNLL-2003 sentences clustered by document) to show that standard permutation-based validity checks fail to detect the violation of exchangeability caused by clustering, leading to overconfident and miscalibrated systems. The empirical evidence effectively bridges the gap between abstract statistical theory and practical ML deployment issues.
The paper provides detailed mathematical derivations and references specific code paths (e.g., `marginal_guarantee_exact.py`) and datasets (released PRM calibration set, CoNLL-2003). The theoretical results are self-contained, and the simulation parameters are clearly defined. The use of a released dataset enhances reproducibility for the empirical portion. However, the "released artifact" mentioned in §6.1 might require specific access or processing steps not fully detailed in the text alone, though the general methodology is clear.
The theoretical results rely on asymptotic approximations (large number of clusters $b$) and specific assumptions about the dependence structure (exchangeability within clusters, continuous marginal distributions). The paper acknowledges that "informative cluster sizes" (where cluster size is correlated with the score) can introduce first-order bias that the design effect correction alone does not address, requiring additional weighting strategies. The correction is specific to the coverage dispersion and does not fully resolve bias in the mean coverage under complex sampling designs without further adjustments.
This work has significant implications for the reliability and safety of machine learning systems, particularly in conformal prediction, abstention gates, and safety filtering. By providing a correct method to estimate effective sample sizes in the presence of clustering, it helps practitioners avoid overconfident uncertainty estimates. This is crucial for high-stakes applications where miscalibration can lead to harmful decisions. The paper also highlights a systemic oversight in the ML literature regarding the distinction between shift and dependence in calibration data, encouraging more rigorous statistical validation in future work. The paper provides a rigorous theoretical and empirical correction for the effective sample size in clustered conformal prediction, demonstrating that standard validity checks fail to detect coverage dispersion issues caused by data clustering, thereby enhancing the reliability of uncertainty quantification in ML systems.
Federated learning enables privacy-preserving collaborative training, but highly heterogeneous client data remain challenging, especially in graph federated learning where clients possess structurally diverse graphs. Existing personalized federated learning (PFL) methods ignore the intrinsic geometric properties of diverse graph structures. We propose FlatLand, a novel personalized federated learning method that embeds different clients' data in tailored Lorentz space of hyperbolic geometry. Our key insight is that hyperbolic geometry naturally accommodates the intrinsic negative curvature prevalent in real-world graphs, while the time-like dimension in Lorentz space provides a principled way to encode client-specific heterogeneity. We develop a parameter decoupling strategy that separates heterogeneous information (captured in time-like parameters) from common knowledge (preserved in space-like parameters), enabling direct aggregation without requiring client similarity estimation and extra calculation modules. Empirical results on diverse federated graph learning tasks demonstrate that FlatLand achieves superior performance, particularly in low-dimensional settings.
Primary: The Chinese University of Hong Kong
All Institutions: The Chinese University of Hong Kong, Yale University, The Hong Kong University of Science and Technology (Guangzhou), Huawei Technologies Co., Ltd.
FlatLand introduces a novel geometric decoupling strategy for personalized federated graph learning, leveraging Lorentzian hyperbolic geometry to separate heterogeneous client information from common knowledge, thereby enhancing performance and privacy in structurally diverse federated settings.
The paper proposes FlatLand, a Personalized Federated Learning (PFL) framework for Graph Neural Networks (GNNs) that utilizes Lorentzian hyperbolic geometry. The core innovation lies in decoupling client-specific heterogeneity into a "time-like" dimension and common knowledge into "space-like" dimensions within the hyperbolic embedding space. This allows for aggregation of the space-like parameters while keeping time-like parameters local, theoretically addressing the challenge of structural heterogeneity in federated graph learning. The approach combines geometric insights with practical FL constraints. While the application of hyperbolic geometry to GNNs is not entirely new, its specific integration into a parameter-decoupling PFL framework for heterogeneous graphs is a distinct and well-motivated contribution. The mathematical formulation appears sound, leveraging the properties of the Lorentz model to handle negative curvature inherent in graph structures.
The authors evaluate FlatLand on diverse federated graph learning tasks. The abstract claims superior performance, particularly in low-dimensional settings. The paper includes 9 figures and 8 tables, suggesting a comprehensive empirical study. However, without access to the specific baselines and dataset details in the full text, the magnitude of improvement and statistical significance must be inferred from the abstract's claims and the venue's reputation. The focus on "low-dimensional settings" is a specific niche; performance in high-dimensional Euclidean spaces or compared to state-of-the-art Euclidean PFL methods is less clear from the summary. The inclusion of ablation studies on the decoupling strategy is implied but not explicitly detailed in the abstract.
The paper is 34 pages with extensive appendices, which typically indicates good documentation of hyperparameters and experimental setups. The methodology relies on standard hyperbolic operations and FL communication protocols, which are generally reproducible. The authors list affiliations with major tech companies and universities, suggesting access to computational resources. However, the specific implementation details of the "tailored Lorentz space" per client and the optimization dynamics are critical for reproduction and are likely contained in the full text.
A key limitation of hyperbolic embeddings is numerical instability and optimization difficulty, especially in high dimensions, although the paper claims success in low dimensions. The "tailored" aspect implies per-client geometry, which might increase memory overhead or complexity in communication if not handled carefully, although the decoupling strategy aims to mitigate this. The effectiveness of the method depends heavily on the assumption that graph heterogeneity can be effectively captured by the time-like dimension; if heterogeneity is more complex, the model might underfit. Additionally, the claim of "no client similarity estimation" is a strong advantage, but the quality of the decoupling relies on the learning dynamics, which might be sensitive to initialization or learning rates.
This work contributes to the growing field of geometric deep learning and privacy-preserving AI. By providing a principled way to handle heterogeneous data in federated settings without sharing sensitive client-specific parameters, it advances the practical applicability of FL in domains like social networks or biological graphs where data is inherently non-Euclidean and heterogeneous. The privacy-by-design aspect, avoiding the sharing of similarity matrices, is a significant benefit for sensitive applications. FlatLand introduces a novel geometric decoupling strategy for personalized federated graph learning, leveraging Lorentzian hyperbolic geometry to separate heterogeneous client information from common knowledge, thereby enhancing performance and privacy in structurally diverse federated settings.
For AI agents to be useful beyond simple chat, they must hold sensitive user context such as calendars, credentials, health records, and financial data. We study whether the mere presence of such secrets in a model's context window introduces hidden correlations into the model's benign outputs, allowing reconstruction even when the model correctly refuses direct extraction. We further study whether an adversary can actively engineer prompts that amplify this effect, using the model as a covert carrier to transmit secrets through seemingly innocuous text. In both cases, this limited leakage is exploited using a novel adaptive attack that assumes black-box access to the underlying model. In controlled experiments across eight proprietary models, we find that 2-digit in-context secrets are reconstructed with near-perfect accuracy and 4-digit secrets at 82\% exact match, all from outputs the model produces in response to ordinary, non-adversarial requests. We observe that more capable models leak more: stronger instruction-following amplifies sensitivity to in-context secrets, suggesting leakage is a byproduct of capability as opposed to a patchable bug. We show this leakage enables two practical attacks: (1) a trained classifier that infers semantic predicates about user memories (e.g., health conditions, financial events) from routine natural-language outputs, and (2) an RL-trained adversary that extracts full Social Security Numbers from a production-style agent.
Primary: University of California, San Diego
All Institutions: University of California, San Diego, University of Maryland
This paper presents a rigorous and impactful analysis of inadvertent context leakage in language models, demonstrating that sensitive information can be reconstructed from benign outputs through statistical side-channels, even when the model explicitly refuses to disclose it. The introduction of adaptive black-box decoders to exploit these channels provides a new and critical perspective on LLM privacy, with significant implications for the deployment of AI agents handling sensitive user data.
The paper introduces a novel threat model termed "inadvertent context leakage," moving beyond traditional direct extraction or prompt injection attacks. The core methodology involves training adaptive, black-box decoders (using LoRA-finetuned classifiers) to detect statistical correlations between in-context secrets and benign model outputs. The authors isolate specific leakage channels, including response length, stylistic features, and digit frequency profiles, demonstrating that these channels persist even when the model explicitly refuses to output the secret. The approach is rigorous, employing a controlled experimental setup across eight proprietary frontier models and utilizing both passive inference and active RL-optimized prompt injection to engineer covert channels.
The experimental evaluation is comprehensive and convincing. The authors test on eight leading proprietary models (Claude, Gemini, GPT, Grok) using a variety of secret lengths (1-8 digits) and semantic predicates. Key findings include near-perfect reconstruction of 2-digit secrets and 82% exact match for 4-digit secrets on Claude Opus 4.6. The paper provides strong ablation studies, including the "suppression" hypothesis (that stronger instructions to hide data paradoxically increase leakage by distorting the output distribution), cross-model transferability of decoders, and the effectiveness of the attack against production-style agents (SSN extraction). The results are statistically significant and robust across different prompt templates and context sizes.
The paper provides detailed descriptions of the attack pipelines, decoder architectures (Qwen-2.5-1.5B backbone with LoRA), and hyperparameters. The use of public APIs for proprietary models limits exact reproducibility of the victim models' internal states, but the attack methodology is clearly defined. The authors provide prompt templates and describe the GRPO optimization process for the active attack. The data collection process (1,000 secrets, 10 responses per pair) is well-documented, allowing other researchers to replicate the study on accessible models or verify the API interactions.
The study is limited to numeric secrets and binary semantic predicates; generalization to open-vocabulary, free-text secrets is not addressed due to the combinatorial explosion of the value space. The experiments are conducted in a black-box setting via APIs, which may not capture all side-channels available in local deployments. The "resistant" tier of models (Grok, GPT-nano) suggests that leakage is model-dependent, and the mechanisms for resistance are not fully understood or mitigated. The study assumes the adversary can query the model, which may not always be the case in all deployment scenarios.
This work has significant implications for the privacy of AI agents and the security of LLM deployments. It challenges the assumption that instruction-following and refusal mechanisms are sufficient for data protection. The findings suggest that privacy in LLMs requires fundamental architectural changes or information-theoretic guarantees rather than just prompt engineering or output filtering. This could drive new research into privacy-preserving model training, secure inference protocols, and standardized privacy benchmarks for AI agents. This paper presents a rigorous and impactful analysis of inadvertent context leakage in language models, demonstrating that sensitive information can be reconstructed from benign outputs through statistical side-channels, even when the model explicitly refuses to disclose it. The introduction of adaptive black-box decoders to exploit these channels provides a new and critical perspective on LLM privacy, with significant implications for the deployment of AI agents handling sensitive user data.
Hidden chain-of-thought (CoT) traces, especially those from frontier proprietary large reasoning models (LRMs), are valuable model assets. Yet whether these hidden CoTs can be directly extracted from black-box models remains largely unexplored. In this work, we systematically study whether hidden CoTs can be extracted near-verbatim from black-box LRMs through API interactions. We identify a previously overlooked reasoning replay surface between tool calls and develop EchoCoT, a multi-step attack that iteratively extracts hidden CoTs using API-returned fidelity signals. We further develop an LLM-based optimization framework that automatically searches for an effective universal injection trajectory across various datasets. We evaluate EchoCoT on three open-source and five frontier proprietary LRMs. On open-source LRMs, EchoCoT achieves up to 66.4\% near-verbatim extraction success, with the extracted trace length within 10\% of the target and at least 90\% of tokens exactly matching the target CoT. The same injection trajectory also generalizes to unseen datasets, achieving up to 80\% extraction success under the same criterion. For tested frontier proprietary LRMs, a substantial fraction of extracted CoTs closely align with provider-reported reasoning lengths and available CoT summaries. EchoCoT can also extract very long CoTs: on Gemini-2.5, it extracts 33,463 tokens from a 32,948-token target. These results establish hidden-CoT extraction as a practical security risk and highlight the need to better protect hidden CoT assets.
Primary: CISPA Helmholtz Center for Information Security
All Institutions: CISPA Helmholtz Center for Information Security
EchoCoT establishes a practical and effective method for extracting hidden Chain-of-Thought traces from black-box Large Reasoning Models, exposing a significant security vulnerability in current API designs and setting a new benchmark for CoT privacy research.
The paper proposes EchoCoT, a sophisticated black-box attack that exploits the "reasoning replay surface" inherent in tool-calling mechanisms of Large Reasoning Models (LRMs). The core insight is that while standard multi-turn conversations discard hidden CoTs, tool calls preserve them within the context window. By iteratively injecting instructions via a scratchpad tool, the authors force the model to reproduce its hidden reasoning. The novelty lies in the automated optimization framework (LLM-based Inject-Reflect-Distill) that searches for universal injection trajectories using only API-provided proxy signals (token counts and summaries) as fidelity feedback. This is a clever adaptation of prompt optimization techniques to the specific constraints of black-box CoT extraction. The methodology is technically sound and demonstrates a deep understanding of LRM internal states and API behaviors.
The evaluation is extensive and rigorous. The authors test EchoCoT on three open-source LRMs (DeepSeek, Qwen, GLM) with ground-truth CoTs available for precise measurement, and five frontier proprietary models (Gemini, Claude) where they use proxy metrics and qualitative analysis. The results are striking: on open-source models, they achieve near-verbatim extraction (Token-EM > 0.90) for a significant fraction of samples. The cross-dataset transferability is also strong, with optimized trajectories working well on unseen datasets. The ability to extract very long CoTs (33k+ tokens) is a significant empirical finding. The baselines (Direct Prompting, CoT Synthesis, REP) are appropriate, and EchoCoT vastly outperforms them.
The paper provides a detailed description of the attack pipeline, optimization framework, and evaluation metrics. The code is available on GitHub. The use of public APIs for proprietary models ensures that the attack can be reproduced by others, provided they have access to the same models. The open-source model evaluations are fully reproducible.
The attack relies on the availability of tool-calling interfaces and specific API signals (reasoning token counts, summaries). Models that disable tool calls or hide these signals would be resistant. The attack is also computationally expensive due to the iterative nature and the need for multiple LLM calls for optimization. Furthermore, the success rate, while high, is not 100%, and failures often stem from length mismatches or refusals. The paper notes that defenses are possible but currently insufficient.
This work has profound implications for the security and privacy of frontier LRM providers. It demonstrates that hidden CoTs, considered valuable intellectual property and safety assets, are not secure against determined black-box attackers. This could lead to industrial espionage, leakage of sensitive training data, or bypassing of safety filters embedded in the reasoning process. It highlights a critical gap in the security design of LRM APIs and necessitates new defense mechanisms, such as obfuscating reasoning steps, limiting tool-call context, or detecting adversarial tool usage. EchoCoT establishes a practical and effective method for extracting hidden Chain-of-Thought traces from black-box Large Reasoning Models, exposing a significant security vulnerability in current API designs and setting a new benchmark for CoT privacy research.
Multi-agent LLM systems split work across models, so answering often requires knowledge that sits in another agent's context: a Sharer has encoded information that a Receiver needs to complete its task. They usually communicate by exchanging text, which puts autoregressive decoding on the critical path and reduces the exchange to a discrete message written without sight of the receiver's state. Recent latent protocols instead translate the sharer's key-value (KV) cache into the receiver's: C2C supports heterogeneous models but requires both to read the same input, while LCF-X removes this shared-context requirement through position-free sharer-cache pooling. Three restrictions remain: LCF-X compresses the sharer alone, supplies the same layer-local summary to every receiver position with no joint cross-layer memory to retrieve from, and assumes matched layer count and KV geometry. We introduce XKV, which lifts all three: learned-query attention pools both caches; self-attention over receiver-aligned layer tokens, with a learned layer map reconciling different depths, mixes the pooled summaries into a compact joint memory; and a shared position decoder lets every raw receiver cache position retrieve its own per-head-gated residual in the receiver's native KV geometry. Both models stay frozen and may differ in family, depth, KV-head count, head dimension, and tokenizer; only the translator is trained. Across 45 dataset-model-pair settings (six heterogeneous and three same-model ordered pairings, five datasets), XKV attains the highest macro score and best average rank, improving on LCF-X on every dataset (by 4.6 exact-match and 4.2 F1 points on ROPES) and surpassing text communication on four of the five, while training 76% fewer parameters and translating a cache pair 10.3x faster (5.8 vs. 59.9 ms); end to end, XKV is 26% faster than LCF-X and 6.8x faster than text communication.
Primary: Amazon
All Institutions: Amazon
[One sentence main contribution]. XKV introduces a dual-cache latent communication protocol that pools both sharer and receiver KV caches into a joint memory, enabling efficient, receiver-aware, and geometry-agnostic knowledge transfer between heterogeneous frozen LLMs. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper makes a solid technical contribution to the emerging area of latent communication in multi-agent systems. By addressing the limitations of prior work (LCF-X) through dual-cache pooling and position-specific retrieval, it achieves state-of-the-art accuracy and efficiency on split-evidence reasoning tasks. The rigorous evaluation across heterogeneous model pairs and the clear demonstration of efficiency-accuracy trade-offs make this a valuable addition to the literature. The method is well-motivated, technically detailed, and empirically validated.
The paper proposes XKV, a "dual-cache" latent communication protocol for heterogeneous frozen LLMs. The core innovation lies in pooling both the sharer's and receiver's KV caches using learned-query attention to create a joint memory, rather than just compressing the sharer's cache as in prior work (LCF-X). This joint memory is then queried by each receiver position to retrieve position-specific, receiver-native KV residuals. The method includes a learned layer map to handle depth mismatches and a shared position decoder. The approach is technically sound, leveraging standard attention mechanisms in a novel configuration for cross-model state transfer. It addresses specific bottlenecks in previous latent communication methods: receiver-independence, layer-local single-summary bottlenecks, and matched-geometry assumptions.
The evaluation is comprehensive, covering 45 dataset-model-pair settings across five benchmarks (ROPES, MuSiQue, QASC, StrategyQA, HotpotQA-bridge) and nine ordered pairs of Qwen, Gemma, and Llama models. XKV consistently outperforms text-to-text (T2T) and the latent baseline LCF-X in accuracy/F1 scores. Crucially, it demonstrates significant efficiency gains: 10.3x faster communication latency than LCF-X and 6.8x faster end-to-end than T2T, with 76% fewer parameters. The ablation studies effectively isolate the contribution of the dual-cache pooling and position-specific retrieval. The results are robust and clearly presented.
The paper provides detailed mathematical formulations for the pooling, layer alignment, and decoding steps. It specifies the model families, dataset splits, and evaluation metrics. The claim that both base models are frozen and only the translator is trained simplifies the reproduction of the inference pipeline. The latency measurements are specific and comparable. While code is not explicitly linked in the text provided, the methodological description is sufficient for reproduction by a competent researcher.
The method assumes a "cross-context" setting where evidence is split between two models. It does not address multi-turn conversations or many-agent systems with complex topologies, though this is noted as future work. The performance gain over LCF-X, while consistent, is moderate in absolute terms (e.g., +4.6 EM on ROPES). The reliance on a learned translator introduces a small overhead, albeit minimal compared to the savings. The evaluation is limited to small-to-medium models (0.6B-3B); scaling to larger models (7B+) is not demonstrated.
This work advances the field of multi-agent LLM systems by enabling more efficient and accurate latent communication. By removing the need for text decoding/encoding cycles and supporting heterogeneous models, it lowers the barrier to building distributed LLM systems. The efficiency gains could lead to reduced latency and cost in deployed agent frameworks. The approach of joint state pooling is a generalizable technique that could inspire similar methods in other domains requiring cross-model information transfer. [One sentence main contribution]. XKV introduces a dual-cache latent communication protocol that pools both sharer and receiver KV caches into a joint memory, enabling efficient, receiver-aware, and geometry-agnostic knowledge transfer between heterogeneous frozen LLMs. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper makes a solid technical contribution to the emerging area of latent communication in multi-agent systems. By addressing the limitations of prior work (LCF-X) through dual-cache pooling and position-specific retrieval, it achieves state-of-the-art accuracy and efficiency on split-evidence reasoning tasks. The rigorous evaluation across heterogeneous model pairs and the clear demonstration of efficiency-accuracy trade-offs make this a valuable addition to the literature. The method is well-motivated, technically detailed, and empirically validated.
Self-normalized concentration inequalities are standard tools in bandit and reinforcement-learning analyses. A widely used weighted extension claims an analogous time-uniform guarantee for discounted least-squares estimators in non-stationary problems. A simple scalar Gaussian counterexample with a fixed parameter shows that the claimed bounded radius is crossed with probability one. For fixed discount and regularization parameters, we further show that, when $δ\leq1/2$ and $T/δ$ is sufficiently large, any deterministic anytime boundary valid uniformly over the stated conditionally sub-Gaussian model class must be at least of order $R\sqrt{\log(T/δ)}$ at some time by horizon $T$; for nondecreasing boundaries, this order is required at time $T$. We identify the proof error: different terminal times use different Gaussian mixing distributions, so the fixed-time mixtures do not form one supermartingale, and the stopping-time argument does not repair this failure. Finally, we show that the weighted inequality remains valid at each fixed deterministic time, give valid finite- and infinite-horizon corrections, and discuss consequences for downstream analyses.
Primary: Academia Sinica
All Institutions: Academia Sinica, National Tsing Hua University
This paper provides a crucial correction to the theoretical analysis of discounted least-squares in non-stationary bandits, identifying a fundamental flaw in the proof of a widely used time-uniform concentration inequality and establishing the necessity of logarithmic confidence costs for valid anytime bounds.
The paper employs a rigorous theoretical analysis to identify a fundamental flaw in the proof of a widely used self-normalized concentration inequality for discounted least-squares estimators. The methodology involves constructing a scalar Gaussian counterexample to disprove the time-uniform bound claimed in prior work (RVC19weightedLB) and deriving matching lower bounds for any valid anytime boundary. The authors correctly identify the logical error in the original proof: the use of different Gaussian mixing distributions at different times prevents the construction of a single supermartingale required for Ville's inequality, and the proposed "stopping-time repair" is invalid because it conditions on a sequence of directions rather than a fixed one. This is a classic example of "e-hacking" or pseudo-maximization in sequential analysis. The correction provided involves explicit confidence allocation (union bound) which restores validity but at the cost of a logarithmic factor in the horizon.
As a theoretical note, the paper does not present empirical experiments in the traditional sense (e.g., training models on datasets). Instead, the "experiments" are the mathematical proofs of the counterexample and the lower bounds. The scalar Gaussian example serves as a synthetic proof-of-concept that the theoretical bounds fail. The evaluation is purely analytical, relying on probability theory and concentration inequalities.
The mathematical derivations, proofs, and counterexamples are fully detailed in the text and appendices. The logic is self-contained and reproducible by any researcher with a background in stochastic processes and bandit theory. The identification of the error in prior work is precise and verifiable.
The primary limitation is that this is a "negative" result paper; it identifies an error and provides a correction, but does not propose a new, more efficient algorithm or a significantly tighter bound than the corrected union-bound approach. The correction (union bound) is standard but may be loose in practice compared to what a correct time-uniform bound might achieve if one existed (though the lower bound suggests the logarithmic factor is necessary for non-decreasing boundaries). The scope is limited to the specific discounted least-squares setting and does not address all possible variations of non-stationary bandit algorithms.
This paper has significant impact on the theoretical foundations of non-stationary bandit and reinforcement learning literature. By correcting a widely cited inequality, it invalidates the theoretical guarantees of several downstream papers (RCG20-NSGLB, KT20, TV20, etc.) that relied on the flawed anytime bound. It forces the community to re-evaluate the regret bounds of these algorithms, potentially requiring them to pay an additional logarithmic factor or rely on fixed-time analysis with union bounds. It serves as an important cautionary tale regarding the application of Ville's inequality and supermartingale arguments in time-varying settings. This paper provides a crucial correction to the theoretical analysis of discounted least-squares in non-stationary bandits, identifying a fundamental flaw in the proof of a widely used time-uniform concentration inequality and establishing the necessity of logarithmic confidence costs for valid anytime bounds.
Most modern optimizers form their momentum as an exponential moving average (EMA) of past gradients, forgetting every direction at one fixed rate. However, the inputs a deep network sees during training can be highly anisotropic, with a few directions queried frequently while most are seen rarely. Recent methods address this anisotropy by wrapping extra processing around this buffer, leaving the momentum update itself unchanged. We propose DeltaMomentum, which builds direction-awareness into the momentum update rule. The main observation is that the gradient of a linear layer splits into an input that acts as a key and an output-side error that acts as a value. Exploiting the key-value structure, DeltaMomentum updates the momentum buffer by the canonical delta rule, so each direction is forgotten at a rate set by how often it appears. We prove that it is a valid momentum, that it applies the input-side curvature correction without matrix inversion, and that it clears stale directions faster than EMA under both a fixed and a drifting optimum. It is a drop-in replacement for the momentum buffer of any optimizer, its coefficient transfers across widths under $μ$P, and its extra compute stays between $22.2\%$ and $25.0\%$ of a gated-MLP block's linear cost with no persistent memory. In FineWeb-Edu pretraining, AdamW with DeltaMomentum (DeltaAdamW) reaches AdamW's validation loss in up to $46.39 \pm 4.32\%$ fewer steps at 67M and $22.12 \pm 0.80\%$ at 370M over three seeds, and the gain persists at 1B on a Chinchilla-optimal budget. A Muon baseline tuned under the same protocol sits above DeltaAdamW at both language-model scales, and the gain holds for SGD, ResNet-18, and ViT-Tiny on CIFAR-10. Training-time diagnostics confirm the predicted mechanism, better gradient tracking and healthier input directions.
Primary: University of Pennsylvania
All Institutions: University of Pennsylvania, Jane Street
DeltaMomentum introduces a theoretically grounded, direction-aware momentum update rule that leverages the key-value structure of gradients to achieve faster convergence and implicit preconditioning. The paper presents a compelling combination of rigorous theory, detailed mechanistic analysis, and strong empirical results across multiple scales and architectures, establishing a new baseline for momentum-based optimization that is likely to influence future optimizer design.
The paper proposes "DeltaMomentum," a novel momentum update rule that replaces the standard Exponential Moving Average (EMA) with a delta-rule update based on the key-value structure of rank-1 gradient factorization in linear layers. The core insight is that treating the momentum buffer as an associative memory allows for direction-aware forgetting rates, where directions queried frequently are updated more aggressively. The methodology includes a theoretical derivation showing that this update implicitly performs input-side preconditioning (similar to K-FAC but without matrix inversion) and provides guarantees on convergence and tracking of non-stationary targets. The approach is theoretically sound, well-motivated by the anisotropy of deep learning landscapes, and offers a clean, drop-in replacement for existing optimizers.
The experimental evaluation is comprehensive and convincing. The authors conduct controlled ablations against AdamW (the only baseline differing in the first moment) and Muon (a strong structured optimizer baseline) on Llama-2-style language models at 67M, 370M, and 1B parameters on FineWeb-Edu. The results show significant step-efficiency gains (up to 46% fewer steps) and competitive wall-clock efficiency. Additional experiments on CIFAR-10 with ResNet-18 and ViT-Tiny demonstrate generalizability. The inclusion of mechanistic diagnostics (gradient alignment, function-space prediction error, feature covariance conditioning) strongly supports the theoretical claims. The compute overhead is quantified and shown to be manageable.
The paper provides detailed algorithmic descriptions, theoretical proofs, and mentions specific implementation details (e.g., normalized keys, $\mu$P scaling). The experimental setup is described with sufficient detail for reproduction, including data sources (FineWeb-Edu) and model architectures. The authors provide a clear recipe for hyperparameter transfer, enhancing reproducibility.
The paper acknowledges limitations regarding scale (up to 1B parameters) and domain (language modeling only). It does not yet demonstrate performance on larger models (e.g., 7B+) or other domains like vision-only or multimodal tasks, though the CIFAR results suggest potential. The theoretical analysis relies on quasi-static approximations, which may not perfectly capture the highly non-stationary dynamics of deep training, although the empirical diagnostics support the theory.
This work has significant potential to improve the efficiency of training large-scale neural networks, reducing compute costs and energy consumption. By providing a drop-in replacement for momentum, it can be easily adopted by the community. The theoretical insights into the role of momentum as an associative memory could inspire further research into optimizer design. DeltaMomentum introduces a theoretically grounded, direction-aware momentum update rule that leverages the key-value structure of gradients to achieve faster convergence and implicit preconditioning. The paper presents a compelling combination of rigorous theory, detailed mechanistic analysis, and strong empirical results across multiple scales and architectures, establishing a new baseline for momentum-based optimization that is likely to influence future optimizer design.
Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy, outcome-rewarded RL over the candidate slate, cannot teach it, for a structural reason we identify and name selector credit starvation: under a broadcast, sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run's own training artifacts confirms all three properties, each worsening monotonically with horizon. SkillGate removes the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separate action-local advantage reaching exactly the skill-naming tokens, positive only when a trajectory's single read is the correct one. On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.
Primary: Unknown (Affiliations not explicitly listed in text, authors appear to be from various Chinese institutions based on names and "corrauth" markers)
All Institutions: Unknown
SkillGate introduces a novel disjoint credit channel architecture that solves "selector credit starvation," enabling 9B agents to reliably select skills from large slates, significantly outperforming outcome-only RL and demonstrating that selection is a trainable, distinct capability from execution.
The paper introduces "SkillGate," a method that addresses the specific problem of in-policy skill selection in long-horizon agentic tasks. The core technical contribution is the identification of "selector credit starvation," a structural flaw in standard outcome-rewarded RL where the tokens responsible for selecting a skill receive vanishingly small and often incorrectly signed gradients. The proposed solution partitions the token support into two disjoint credit channels: an execution channel using standard group-normalized outcome advantages, and a selection channel using an action-local advantage that rewards only the specific tokens naming the correct skill. This separation prevents the selection decision from being punished by execution failures and ensures the selection signal is not diluted by trajectory length. The methodology is theoretically sound, addressing a known bottleneck in agent training with a precise, mechanistic fix.
The authors evaluate SkillGate on five agentic benchmarks (Claw-Eval, SkillsBench, SETA, SWE, Terminal-Bench 2.0) using a 16-candidate slate. They demonstrate that SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, outperforming outcome-only RL, supervised selection, and preference learning baselines. Crucially, they show that SkillGate achieves this with fewer reads and less exposure to misleading skills, indicating genuine improvement in selection capability rather than just increased tool usage. The ablation studies effectively isolate the contribution of the selection channel, showing that coarser credit assignment (group or trajectory level) fails to improve selection. The comparison with frontier models highlights that skill selection is a distinct capability not solved by scale alone.
The paper provides detailed implementation details, including the use of Qwen3.5-9B, GRPO, specific hyperparameters (learning rate, KL coefficient), and the structure of the training tasks. The description of the "standard mixed slate" and the construction of misleading candidates is clear. The audit methodology for demonstrating credit starvation is also well-described, allowing for replication of the diagnostic findings. However, the specific code for the "clean single-oracle utility" and the exact implementation of the disjoint credit masking would be necessary for full reproducibility, which is likely in a supplementary or code release not provided here.
The method requires that the correct skill is known during training to compute the selection advantage, limiting its applicability to settings with verifiable outcomes or oracle skills. The method assumes a "single read" constraint for the selection credit to be clean; reading multiple skills dilutes the signal. The evaluation is limited to 9B and 27B models, and while it outperforms larger frontier models in selection, it does not match their overall task success, suggesting that execution capability remains a bottleneck. The audit is performed on a single run's artifacts, and while the trends are clear, broader statistical significance across seeds is not explicitly detailed in the abstract/text provided.
This work has significant implications for the development of reliable autonomous agents. By solving the credit assignment problem for skill selection, it enables agents to effectively utilize large libraries of procedural knowledge, which is critical for complex, long-horizon tasks. The insight that selection and execution require disjoint credit channels is likely to influence future agent training methodologies, moving the field away from naive outcome-only RL for tool-use scenarios. It highlights the importance of fine-grained credit assignment in multi-decision trajectories. SkillGate introduces a novel disjoint credit channel architecture that solves "selector credit starvation," enabling 9B agents to reliably select skills from large slates, significantly outperforming outcome-only RL and demonstrating that selection is a trainable, distinct capability from execution.
Programmable logic controllers (PLCs) run industrial plants, and large language models can already generate independent program organization units (POUs) for them. Whether such logic integrates into an existing PLC project and then runs correctly has been checked only in limited tests. We present SemaPLC, a project-grounded and verification-gated agent harness assembled from conventional tools but governed by a strict completion rule. Rather than stopping when the model judges its own output adequate, SemaPLC declares a task complete only when logged external checks confirm it. Those checks cover the specification, the compilation, and the behavior on a live runtime. On 117 independent-POU tasks matching existing benchmarks, it attains the highest strict verified pass rate on all seven models (72.6\% mean). On a project-context track of 65 tasks whose generated logic must compile and run inside a real project, it attains the highest mean on integrated compilation, static behavior, and dynamic behavior. Of the three layers, dynamic behavior is the most revealing. We measure it by deploying the generated and the reference logic to a live PLC runtime and comparing their executed traces. All methods fall within 10 static points of one another, whereas dynamic scores separate them sharply, from 22.4 to 31.4 for the baselines against 52.2 for SemaPLC. Overall, our verification-gated harness raises the mean at every layer and most sharply at runtime. Execution, not static scoring, is the faithful test of whether generated control logic actually works. SemaPLC is open-sourced at https://github.com/midea-ai/SemaPLC.
Primary: Midea AI
All Institutions: Midea AI
SemaPLC presents a robust, verification-gated agentic framework for PLC code generation that significantly improves runtime reliability over static baselines, establishing a new standard for evaluation in industrial LLM applications.
The paper proposes SemaPLC, an agentic harness for generating Programmable Logic Controller (PLC) code in Structured Text (ST). The core methodological contribution is not a new neural architecture, but a rigorous "verification-gated" iteration loop. It combines project-grounded generation (retrieving context from existing PLC projects) with three layers of external verification: specification auditing, compilation, and live runtime validation. The agent is forced to repair code until these external checks pass, preventing premature termination based on self-assessment. This approach effectively treats the LLM as a component within a larger, tool-using agent system, leveraging the Model Context Protocol (MCP) for tool interaction. The methodology is sound and addresses a specific, high-stakes domain (industrial automation) where static code generation metrics are insufficient.
The evaluation is comprehensive and rigorous, which is the paper's strongest asset. It introduces two tracks: a function-level track (117 tasks) and a project-context track (65 tasks). The project-context track is particularly novel, requiring integration into existing industrial plant simulations. The authors evaluate across seven state-of-the-art LLMs. The results show that SemaPLC significantly outperforms baselines (LLM4PLC, AutoPLC, Agents4PLC) on dynamic runtime behavior, a metric that static analysis fails to capture. The ablation studies clearly demonstrate the value of each verification layer. The use of a live PLC runtime for evaluation provides a level of empirical truth often missing in LLM code generation papers. The audit of the previous benchmark (Agents4PLC) to fix defective properties adds credibility to the evaluation framework.
The paper provides a GitHub repository link. The methodology describes the tools and protocols (MCP) used. The evaluation setup, including the construction of scenarios and the handling of hidden references, is detailed. The open-sourcing of the harness and the cleaned benchmark data enhances reproducibility. The use of standard LLM APIs ensures that the backbone models are accessible, although the specific versions and endpoints are noted. The detailed description of the verification skills and the agent loop allows for reasonable replication of the system.
The paper acknowledges that dynamic scoring is limited to the bounded scenario set derived from the hidden reference, meaning generalization to unseen conditions is not fully measured. The advantage narrows on the strongest models (e.g., GPT-5.5), suggesting that as models improve, the marginal gain from the complex harness may decrease. The cost in terms of API requests is higher for SemaPLC on the project track compared to some baselines, which is a practical limitation for cost-sensitive applications. The focus on IEC 61131-3 ST limits immediate applicability to other PLC languages or industrial protocols without adaptation.
This work has significant implications for the deployment of LLMs in safety-critical industrial environments. By demonstrating that verification-gated agents can reliably generate code that works on live hardware, it provides a blueprint for trustworthy AI in automation. It highlights the gap between static code metrics and actual runtime performance, encouraging the field to adopt more rigorous, execution-based evaluation standards. The open-sourcing of the tool and benchmark contributes to the community's ability to build and evaluate agentic systems for industrial tasks. SemaPLC presents a robust, verification-gated agentic framework for PLC code generation that significantly improves runtime reliability over static baselines, establishing a new standard for evaluation in industrial LLM applications.
Estimating causal effects under network interference typically assumes that the network used for training and the network used for deployment coincide. In practice, an intervention is run on one population while the question of interest concerns a different population, and the two generally differ in topology, node-covariate composition, and spillover pathways. Transporting a causal effect across networks is therefore a data-fusion problem that no existing algorithm solves. We employ a selection diagram, extended to the network setting so that covariate shift and structural network shift enter as separate selectors, and derive from it a transport formula for the direct, spillover, and total effects in the deployment population. Each formula makes explicit which interventional mechanism is assumed invariant and which observational distribution must be reweighted. We then turn the formulas into TranCE (Transported Causal Effects), a doubly-robust algorithm combining an interventional outcome model, a domain density-ratio correction, and cross-fitted inference. Extensive experiments on two semi-synthetic benchmarks derived from real-world social networks and on a fully real weather-insurance field experiment, where the transported effects are checked against held-out randomized estimates, confirm the effectiveness of our approach. Our findings have the potential to improve intervention strategies in networked systems, particularly in social networks and public health.
Primary: Unknown
All Institutions: Unknown
TranCE provides a theoretically grounded and empirically validated solution for transporting causal effects across networks under interference, addressing a critical limitation of existing methods that assume identical network structures. The extension of selection diagrams to network settings and the doubly-robust estimator offer a significant advancement in the field of causal machine learning.
The paper addresses a significant gap in causal inference: transporting causal effects across different network topologies and covariate distributions under interference. The core theoretical contribution is the extension of selection diagrams to the network setting, explicitly separating covariate shift ($S_Z$) from structural network shift ($S_G$). This allows for the derivation of transport formulas for direct, spillover, and total effects. The proposed algorithm, TranCE, implements these formulas using a doubly-robust estimator that combines an interventional outcome model (using GCNs to handle network structure) with a domain density-ratio correction (using logistic regression) and cross-fitted inference. The methodology is rigorous, leveraging standard causal inference tools (do-calculus, doubly-robust estimation) in a novel, complex setting. The separation of structural and covariate shifts is a key theoretical insight that clarifies what assumptions are required for transportability in networked systems.
The evaluation is extensive and convincing. It includes two semi-synthetic benchmarks (Twitch, Facebook-100) with rotating source-target pairs to simulate cross-network transport, and a fully real-world field experiment (weather-insurance in rural China). The semi-synthetic experiments compare TranCE against strong baselines including TARNet, IPW, IGL, Hoshino, NetEst, DANN, and IW-GCN. TranCE consistently achieves the lowest bias, particularly for spillover effects, demonstrating the necessity of both the outcome model and the density-ratio correction. The ablation studies effectively isolate the contributions of different components. The real-world validation against held-out randomized estimates provides strong external validity. The analysis of "Where Transport Is Hard" (correlation between structural gap and bias) is particularly insightful for practitioners.
The paper states that source code, data-preparation scripts, and instructions are included in a supplement. The mathematical derivations are detailed, and the algorithmic steps are clearly described. The use of standard libraries (PyTorch, GCNs) and well-defined causal estimators aids reproducibility. The semi-synthetic data generation process is described, though the specific parameters might require careful reading of the appendix. The real-world dataset (Cai insurance) is publicly available.
The method relies on the assumption of stratified interference, which limits the scope of interference to local neighborhoods. The neighbor treatment summary is discretized into a small number of levels, which might lose information. The method assumes that the interventional mechanism is invariant across domains (except for the parts captured by the selectors), which is a strong assumption in practice. The performance degrades significantly when there is a large structural gap between source and target networks, although this is expected and well-documented. The method also assumes overlap in the covariate space, which can be difficult to verify and ensure in high-dimensional settings.
This work has significant potential impact for policy evaluation in social networks, public health interventions, and online platform optimization. By enabling the transport of causal effects from controlled experiments to observational settings with different network structures, it allows for more efficient and ethical deployment of interventions. It provides a rigorous framework for understanding the limits of generalizability in networked causal inference. TranCE provides a theoretically grounded and empirically validated solution for transporting causal effects across networks under interference, addressing a critical limitation of existing methods that assume identical network structures. The extension of selection diagrams to network settings and the doubly-robust estimator offer a significant advancement in the field of causal machine learning.
Visual autoregressive (VAR) models have emerged as a fast, high-quality alternative to diffusion for text-to-image generation, but like diffusion models they exhibit persistent compositional failures, producing images that violate the attribute bindings and spatial relations specified in the prompt. While a rich line of test-time alignment methods has developed for diffusion, no comparable approach exists for next-scale VAR generation, whose stateful, discrete, multi-resolution sampling process makes existing techniques inapplicable. We close this gap with \textbf{VISTA} (\textbf{Vi}sual Autoregressive \textbf{S}emantic \textbf{T}est-time \textbf{A}lignment), the first gradient-based test-time alignment framework for next-scale autoregressive image generation. Built on Infinity, VISTA intervenes directly in the generation process, optimizing intermediate representations through the frozen transformer to steer visual predictions toward compositional constraints, without modifying model parameters or requiring additional training. VISTA introduces the mechanisms needed to make such optimization stable across scales, together with an extensible objective space that any differentiable constraint on cross-attention can plug into. Across two benchmarks and two model scales, VISTA improves every targeted compositional category, raising the mean targeted score by nearly 20\% on a 2B backbone and almost 6\% on an 8B backbone, with the largest gains on spatial relations. Image quality is preserved: an independent preference model VISTA never optimizes scores its outputs nearly 20\% higher. Notably, the 2B model with VISTA surpasses a backbone four times its size, indicating that a substantial part of the compositional gap between model scales is recoverable at test time.
Primary: Unknown
All Institutions: Unknown
VISTA introduces the first gradient-based test-time alignment framework for visual autoregressive models, effectively bridging the gap between diffusion-based alignment techniques and the unique constraints of autoregressive generation, thereby significantly enhancing compositional fidelity in image generation without model retraining.
The paper proposes VISTA, a gradient-based test-time alignment framework for Visual Autoregressive (VAR) models. The core innovation lies in adapting test-time optimization techniques, typically used for diffusion models, to the discrete, multi-resolution, and stateful sampling process of VAR models. The method intervenes in the generation process by optimizing intermediate representations through the frozen transformer to steer visual predictions toward compositional constraints. It introduces mechanisms to stabilize this optimization across scales and an extensible objective space. The approach is theoretically sound, addressing a specific gap in VAR literature where existing diffusion-aligned methods fail due to the architectural differences.
The evaluation is conducted on two benchmarks and two model scales (2B and 8B). Results indicate significant improvements in compositional categories, with mean targeted scores rising by nearly 20% on the 2B backbone and almost 6% on the 8B backbone. The largest gains are observed in spatial relations. Crucially, the paper demonstrates that image quality is preserved, with an independent preference model scoring outputs nearly 20% higher. A compelling finding is that a 2B model with VISTA surpasses a backbone four times its size in compositional fidelity, suggesting that test-time alignment can recover a substantial part of the compositional gap between model scales.
The paper describes the method in detail, including the optimization mechanisms and objective space. However, as an arXiv preprint without an explicit code release link in the provided text, reproducibility relies on the clarity of the method description. The use of standard benchmarks and frozen backbones (Infinity) aids in reproducibility.
The paper acknowledges limitations in Section 5, though the specific details are not fully expanded in the abstract. Potential limitations likely include the computational cost of gradient-based optimization at test time, which may be prohibitive for real-time applications, and the potential for over-optimization or mode collapse if constraints are not carefully balanced. The effectiveness might also vary depending on the specific prompt complexity and the base model's inherent capabilities.
VISTA addresses a persistent failure mode in generative AI (compositional alignment) without requiring retraining. This has significant implications for making smaller, more efficient models competitive with larger ones in terms of prompt adherence. It opens up new avenues for test-time optimization in autoregressive visual generation, potentially influencing future research in efficient and controllable generative models. VISTA introduces the first gradient-based test-time alignment framework for visual autoregressive models, effectively bridging the gap between diffusion-based alignment techniques and the unique constraints of autoregressive generation, thereby significantly enhancing compositional fidelity in image generation without model retraining.
Reasoning-augmented text-to-image models such as GoT-R1 emit an explicit textual plan - object names, attributes, and bounding boxes - before generating image tokens. When such a model fails a compositional prompt, is the plan wrong, or is the plan right and the decoder unfaithful? Because the plan is machine-readable it can be edited before decoding, which makes the two separable. We first validate the ruler. Swapping the two bounding boxes inside the model's own chain demonstrably flips the generated layout: detector-based accuracy falls 0.75 -> 0.48 (p<1e-3), while a widely used VQA-based spatial metric rises. A five-rater human study agrees with the detector on 81% of items and with the VQA judge on 57%. All spatial results therefore use geometric scoring. Under sound measurement the decoder is a faithful executor: 94% of generated layouts realize the planned relation, and object-box binding survives reordering of the plan's object segments. The planner is the bottleneck. It writes wrong relations for phrasing-dependent reasons - 98% accuracy on "left" against 54% on "right" for semantically identical layouts, a raster-order bias we isolate with a mention-order control - and cluttered geometry that the decoder faithfully reproduces. Editing the plan therefore fixes the image without retraining: symbolic verification with resampling gives +5.0 points (p<1e-3), minimal in-place repair +6.0 (p=.02), rewriting only box geometry +10.7 (p<1e-4), and replacing the plan outright +13.3 (p=1e-4). Gains are indifferent to the plan's prose style and to its likelihood under the planner, but not to its geometry. Modular planner-decoder designs are therefore viable, provided the plan is internally consistent: box-text contradictions induce object duplication and identity fusion. We release the plan-fidelity evaluation protocol, all plans, and 12k generated images.
Primary: Carnegie Mellon University
All Institutions: Carnegie Mellon University
The paper makes a significant contribution by rigorously diagnosing the source of compositional failures in reasoning-augmented text-to-image models, demonstrating that the decoder is a faithful executor and the planner is the bottleneck, thereby validating inference-time plan repair as a highly effective, training-free intervention.
The paper employs a rigorous perturbation-based methodology to diagnose failure modes in reasoning-augmented text-to-image models (specifically GoT-R1). By treating the textual plan as an editable intermediate representation, the authors decouple planning errors from execution errors. The core methodological contribution is the "box swap" intervention and the "oracle dial" experiment, which systematically vary plan geometry and text style to determine what the decoder actually conditions on. This approach is methodologically sound, leveraging controlled interventions to isolate causal factors in complex generative pipelines. The use of paired permutation tests and hardware-matched controls adds significant statistical rigor compared to standard benchmark reporting.
The experimental evaluation is comprehensive and convincing. The authors validate their metric choice by showing that VQA-based spatial metrics are blind to layout inversions, while detector-based geometric scoring correctly identifies them. Their findings—that the decoder is a faithful executor (94% fidelity) and the planner is the bottleneck—are supported by extensive ablations, including seed robustness checks and scaling to a 7B model. The "oracle dial" results are particularly strong, demonstrating that clean geometric plans outperform the model's own plans regardless of text style or likelihood, directly challenging the co-adaptation hypothesis. The statistical significance of all reported gains is established.
The paper provides a high level of reproducibility. The authors explicitly state that they release the plan-fidelity evaluation protocol, all plans, 12k generated images, and analysis code. The experimental setup is well-documented, including specific model versions (GoT-R1-1B/7B), decoding parameters (CFG, temperature, seeds), and the exact nature of the interventions (verify, repair, replace). The inclusion of a reproducibility statement and the release of raw data and scripts significantly lowers the barrier for replication.
The primary limitation is the scope of the evaluation. The study focuses exclusively on the GoT-R1 model family (1B and 7B). While the authors acknowledge this and suggest generalizing to other architectures (T2I-R1, BAGEL), the findings regarding the decoder's faithfulness and the planner's specific biases (raster-order) may not transfer universally to all reasoning-augmented T2I models. Additionally, the evaluation is limited to validation subsets of T2I-CompBench++, and while the metrics are validated, the detector-based scoring inherits the limitations of the underlying open-vocabulary detector (OWLv2).
This work has significant implications for the design of multimodal generative systems. By proving that modular planner-decoder designs are viable and that the planner is the primary bottleneck, it shifts the research focus from end-to-end training to inference-time plan verification and repair. This could lead to more robust and controllable image generation systems. The ethical statement appropriately notes the dual-use nature of improved compositional control and the potential for amplifying biases if not audited. The paper makes a significant contribution by rigorously diagnosing the source of compositional failures in reasoning-augmented text-to-image models, demonstrating that the decoder is a faithful executor and the planner is the bottleneck, thereby validating inference-time plan repair as a highly effective, training-free intervention.
Large Language Models generate text autoregressively, but open-ended generation is prone to verbatim looping, in which models repeat spans already present in context. Standard defenses such as repetition, presence, and frequency penalties and n-gram blocking act on token recurrence rather than the sequential structure of a loop, and often suppress looping only at strengths that also degrade formatting or fluency. We propose Don't Repeat Yourself (DRY), a sampling-time logit adjustment that penalizes a candidate token only when generating it would extend the current suffix into an exact continuation of a span seen earlier in the context. Sequence breakers protect chat templates and formatting tokens. Across models from 1.5B to 120B parameters, nine prompt families, and a 600-pair human study, DRY reduces suffix-extension rate by 47% while improving lexical diversity. An intervention-matched placebo produces no comparable reduction, identifying suffix matching as the operative mechanism. On AWQ-quantized 70B and 120B models, DRY reduces loop rate by roughly half while preserving MT-Bench, MMLU, and GSM8K performance, whereas standard alternatives lose measurable ground. DRY has been adopted by popular open-source LLM inference frameworks including llama.cpp, ExLlamaV2, and text-generation-webui, highlighting its practical impact on text generation.
Primary: Unknown (Paper text redacts affiliations; mentions "All" for author contributions)
All Institutions: Unknown
The paper presents a highly effective, practically significant, and well-evaluated method for mitigating verbatim looping in LLMs, demonstrating clear improvements over existing baselines with minimal overhead and broad applicability across model scales.
The paper proposes "Don't Repeat Yourself" (DRY), a sampling-time logit adjustment method designed to mitigate verbatim looping in autoregressive language models. Unlike standard repetition penalties that penalize token recurrence globally, DRY specifically targets the extension of suffixes that have already appeared in the context. It employs a penalty that grows exponentially with the length of the matching suffix and utilizes "sequence breakers" (e.g., newlines, colons) to prevent penalizing structurally necessary repetitions like list items or dialogue tags. The methodology is technically sound, leveraging a reverse-search algorithm for efficient suffix matching. The approach is conceptually simple but effectively addresses a specific, persistent failure mode in LLM generation that standard controls handle poorly.
The evaluation is comprehensive and rigorous. It spans models from 1.5B to 120B parameters, covering multiple tokenizer families and decoding regimes. The authors employ a robust set of metrics, including Suffix-Extension Rate (SER), MAUVE, Distinct-4, and capability benchmarks (MT-Bench, MMLU, GSM8K). Crucially, they include an intervention-matched placebo control to isolate the effect of suffix-matching from generic logit perturbation, and a 600-pair human evaluation to validate automated metrics. The results show a 47% reduction in loop rates with minimal impact on fluency or capability, outperforming standard penalties and n-gram blocking. The inclusion of frontier-scale models (70B, 120B) and quantized deployments adds significant practical relevance.
The paper claims full reproducibility, providing code, prompts, and evaluation scripts. The experimental setup is well-documented, including hyperparameter tuning protocols and ablation studies. The use of standard frameworks (Hugging Face Transformers, llama.cpp) and widely available models enhances reproducibility. The human evaluation protocol is also described, though the specific MTurk dataset is not linked in the text (likely due to anonymity constraints in the provided text).
The primary limitation is that DRY addresses surface-form verbatim loops but does not handle semantic repetition or discourse-level looping. The effectiveness is dependent on the quality of the "sequence breaker" set, which is tokenizer-specific and may require tuning for different domains. Additionally, the human evaluation was limited to one model family (Qwen) and specific prompt types, which may not generalize to all use cases. The paper is a preprint, so peer review status is pending, though the technical content is strong.
DRY has immediate practical impact, as evidenced by its adoption in popular open-source inference frameworks (llama.cpp, ExLlamaV2). By providing a simple, effective, and low-overhead solution to verbatim looping, it improves the user experience for a vast number of LLM applications, particularly in chat and long-context scenarios. It shifts the paradigm from global token penalties to targeted structural interventions, offering a template for future decoding-time controls. The paper presents a highly effective, practically significant, and well-evaluated method for mitigating verbatim looping in LLMs, demonstrating clear improvements over existing baselines with minimal overhead and broad applicability across model scales.
Policy optimization (PO) for Large Language Models faces a stability--exploration trade-off, currently mediated by an action-side Policy-KL regularizer. This puts practitioners in a double bind: keeping Policy-KL constrains response behavior and consumes the action-side exploration budget, while dropping it leaves the optimization without an explicit drift control. We argue for an alternative that breaks the dilemma by moving regularization to the input side. As training progresses, the distribution over training queries induced by the current policy drifts unchecked from its pre-RL reference distribution. Concretely, Environment-Regularized Policy Optimization (ERPO) introduces a Query-KL (QKL) term that bounds this query distribution shift, together with a dataset-static reference-derived per-query weight that biases each per-query update toward queries typical under the reference. The QKL gradient flows strictly through the query likelihood; the response score function used by policy-gradient estimators does not appear in the QKL term, so QKL exerts no direct gradient pressure on the response distribution---exploration is preserved. ERPO plugs into GRPO/PPO/REINFORCE-style pipelines without additional forward passes. On six mathematical reasoning benchmarks, ERPO replaces the standard Policy-KL regularizer while achieving effective control over query distribution drift, delivering stronger accuracy and substantially more stable behavior under high-temperature decoding and long-horizon training.Our source code are available at https://github.com/alibaba/ERPO
Primary: Alibaba Group
All Institutions: Alibaba Group
The paper introduces ERPO, a novel regularization technique for LLM policy optimization that shifts KL constraints to the input space to preserve response exploration, demonstrating improved accuracy and stability in mathematical reasoning tasks.
The paper proposes Environment-Regularized Policy Optimization (ERPO), a method designed to address the stability-exploration trade-off in LLM policy optimization (specifically RLHF/RLAIF). The core innovation is shifting the KL regularization from the action space (response) to the input space (query). The authors argue that standard Policy-KL constrains the model's ability to explore diverse responses. ERPO introduces a Query-KL (QKL) term to bound the drift of the query distribution induced by the policy from a reference distribution, alongside a static per-query weight derived from the reference. The authors claim this decouples parameter regularization from the optimization objective, preserving exploration in the response space while controlling distributional shift in the input space. The method is presented as a plug-in replacement for standard KL penalties in PPO/GRPO pipelines.
The evaluation is conducted on six mathematical reasoning benchmarks using Qwen-family models. The results indicate that ERPO achieves comparable KL divergence control to standard Policy-KL but delivers superior accuracy and more stable training behavior, particularly under high-temperature decoding and long-horizon training. The experiments focus on measuring accuracy improvements and training stability metrics. While the results are promising within the specified domain, the evaluation is limited to mathematical reasoning, which is a specific subset of LLM capabilities.
The authors provide source code via a GitHub repository (https://github.com/alibaba/ERPO), which significantly aids reproducibility. The paper includes details on the method's integration into existing pipelines (GRPO/PPO/REINFORCE). However, the limitations section notes that the quality and computational cost of estimating query-level likelihoods may vary, and an exhaustive hyperparameter sweep was not conducted, which might affect reproducibility across different model scales or data distributions.
The primary limitation is the narrow scope of evaluation, focusing exclusively on mathematical reasoning benchmarks and Qwen-family models. The transferability to instruction-following, dialogue, code generation, and multilingual settings is unverified. Additionally, the reliance on estimating query-level likelihoods introduces potential computational overhead and sensitivity to data-selection mechanisms, which is not fully quantified. The lack of a comprehensive hyperparameter analysis for the regularization coefficient is also noted.
This work offers a novel perspective on regularization in LLM training, potentially influencing how researchers approach the stability-exploration dilemma in RLHF. By decoupling input and output regularization, it may enable more robust training procedures for complex reasoning tasks. However, the impact is currently constrained by the specific domain of mathematical reasoning. The paper introduces ERPO, a novel regularization technique for LLM policy optimization that shifts KL constraints to the input space to preserve response exploration, demonstrating improved accuracy and stability in mathematical reasoning tasks.
Multi-agent systems built from large language models are deployed widely, yet how much performance is lost when two LLMs must coordinate rather than act alone remains unclear. We formulate the collaboration tax as the team-decentralisation loss of a two-player cooperative game with private information, with two propositions characterising its sign and its equivalence to a max-superadditivity violation. We operationalise this definition on 32 solo-tractable tasks grouped by source of grounding friction and measure it on 11 models from 7 providers. The tax is structured along two no-exception axes: a category ordering across every model and a monotonic decrease with capability. The proximate mechanism is not a reasoning deficit but a four-stage conversational cascade in which agents make ungrounded claims, fail to query the partner, skip integrating both views, and accept the answer without re-derivation. The tax is mechanically predictable from conversation features and partly tractable: a prompt intervention targeting all four stages closes a substantial fraction of the gap, with the dominant bottleneck differing across categories. In heterogeneous pairs the tax is pulled toward the stronger partner rather than the additive midpoint, empirically realising the max-superadditivity violation predicted by our framework. Together these results recast collaboration in LLM systems as a measurable, predictable, and partly tractable cost.
Primary: University of Notre Dame
All Institutions: University of Notre Dame, Meta Superintelligence Labs, Simon Fraser University
The paper presents a novel and rigorous framework for quantifying and analyzing the coordination costs in LLM multi-agent systems, offering significant theoretical and practical insights into the mechanisms of failure and potential remedies.
The paper introduces a rigorous theoretical framework for quantifying "collaboration tax" in LLM multi-agent systems, grounded in cooperative game theory (Shapley values, superadditivity). The methodology is sound, defining a clear metric (solo vs. paired performance gap) and operationalizing it through a 32-task suite designed to isolate coordination failures from problem-solving capacity. The mechanistic analysis using a four-stage conversational cascade (Grounding, Querying, Integration, Re-derivation) is a significant methodological contribution, providing a granular diagnostic tool for multi-agent failures. The use of prompt interventions as ablation studies to validate these mechanisms is clever and effective.
The experimental setup is comprehensive, evaluating 11 models from 7 providers across homogeneous and heterogeneous pairs. The results are robust, showing consistent patterns across models and task types. The finding that the tax is predictable from conversation features and that prompt interventions can recover a substantial fraction of the loss is empirically strong. The heterogeneous pair analysis, while limited by sample size (only two pairs), provides compelling qualitative evidence for the theoretical predictions regarding asymmetric information and Shapley value allocation. The use of deterministic graders and continuous scoring adds rigor to the evaluation.
The paper provides a detailed description of the task suite, protocols, and metrics. The code and data are made available via GitHub and a project website, which significantly enhances reproducibility. The deterministic nature of the tasks and graders further supports reproducibility. The only potential hurdle is the reliance on API access to specific model versions, but the prompt-level interventions are fully specified.
The primary limitation is the scope of the evaluation. The tasks are synthetic puzzles (spatial, relational, CSP) and may not fully capture the complexity of real-world multi-agent applications like code generation or long-horizon planning. The analysis is restricted to dyadic pairs, and the generalization to larger multi-agent teams is unstudied. The heterogeneous pair analysis is limited to two specific pairs, which restricts the statistical power of those findings. Additionally, the "collaboration tax" metric, while useful, is sensitive to the specific solo baseline used (full information vs. split view), and the interpretation of negative tax values requires careful handling.
This work has significant implications for the design and deployment of LLM-based multi-agent systems. By quantifying the cost of coordination and identifying specific mechanistic failure points, it provides actionable insights for developers to improve system robustness through prompt engineering and protocol design. The theoretical framing connects LLM collaboration to established game-theoretic concepts, potentially opening new avenues for research in multi-agent reasoning and coordination. The findings suggest that much of the "intelligence" in multi-agent systems may be wasted on coordination overhead, a critical consideration for scaling these systems. The paper presents a novel and rigorous framework for quantifying and analyzing the coordination costs in LLM multi-agent systems, offering significant theoretical and practical insights into the mechanisms of failure and potential remedies.
Flow-matching language models refine all token positions in parallel and can trade sampling steps for latency, yet generation quality still degrades sharply with few sampling steps. We trace a source of this degradation to a train--inference mismatch in previous-prediction self-conditioning: during training, the self-conditioning input is computed from the current noisy state with no intervening solver step; during sampling, the solver folds the previous prediction into the latent before that same prediction reappears as the explicit self-conditioning input. This coupling, absent during training, creates redundancy that grows with step width. We show that the mismatch degrades both the self-conditioning input and the solver update, and derive a correction for each from the model's own structure. From the frozen projection weights we identify directions along which the self-conditioning input is redundant with the latent and dampen them; from the solver's integration structure we derive that a step-average prediction is needed and approximate it from prediction history, with scale set by offline trajectory statistics. The resulting sampler, Untied Self-Conditioning, requires no retraining and uses one evaluation per step. At 8 sampling steps on LangFlow, it reduces OpenWebText generative perplexity from $531$ to~$62$ ($8.6\times$); under an adapted Arena-Hard-Auto~v2 protocol, its outputs are preferred in $96\%$ of pairwise comparisons. On ELF-B it reduces generative perplexity from $71$ to~$43$. Improvements hold from 8 to 256 sampling steps.
Primary: Unknown
All Institutions: Unknown
The paper presents a theoretically grounded and empirically powerful correction to self-conditioning in Flow Matching for language models, offering a retraining-free path to high-efficiency, high-quality few-step generation.
The paper addresses a critical and subtle issue in Flow Matching (FM) for discrete sequence generation (language models): the train-inference mismatch in self-conditioning. The authors correctly identify that standard self-conditioning (using previous predictions as input) creates a dependency loop that is handled differently during training (using ground truth or clean noisy states) versus inference (using autoregressive or parallel predictions). They propose "Untied Self-Conditioning" (USC), which decouples this dependency by projecting out redundant components from the self-conditioning input and approximating a step-average prediction using history. This is a methodologically sound derivation based on the linear structure of the projection layers and the solver's integration steps. The approach is elegant because it requires no retraining, leveraging existing models.
The experimental results are exceptionally strong, bordering on suspiciously so for an abstract-only summary. The reduction of perplexity from 531 to 62 on OpenWebText at 8 steps is a massive improvement (order of magnitude). Similarly, the ELF-B result (71 to 43) is significant. The preference rate of 96% on Arena-Hard-Auto v2 is also extremely high. While the magnitude of improvement suggests a very effective correction, such dramatic gains in a mature field often warrant scrutiny regarding the baseline's validity or potential overfitting to the specific evaluation protocol. However, assuming the baselines are standard (e.g., standard Rectified Flow or Consistency Models for language), this would represent a state-of-the-art result for few-step generation. The evaluation covers both likelihood (perplexity) and quality (human preference), providing a comprehensive view.
The paper claims the method requires no retraining and uses one evaluation per step, which enhances reproducibility and accessibility. The description of the correction terms (dampening redundant directions, approximating step-average) suggests clear algorithmic steps. However, the "Unknown" institution and lack of explicit code links in the prompt text mean external verification of the specific implementation details (e.g., how "offline trajectory statistics" are computed) is currently limited. The appendix mentions proofs and details, which is a positive sign for reproducibility if the full text is available.
The primary limitation is the reliance on the specific structure of the self-conditioning mechanism in current FM language models. If the model architecture changes significantly (e.g., different attention mechanisms or non-linear projections), the specific projection-based correction might need adaptation. Additionally, the approximation of the step-average prediction using history might introduce noise if the history is short or noisy, potentially limiting performance at very low step counts (< 4 steps). The "96% preference" metric might also be sensitive to the specific prompt distribution in Arena-Hard-Auto.
This work has significant potential impact on the efficiency of language models. By enabling high-quality generation with very few sampling steps (8 steps), it drastically reduces inference latency, making powerful language models more viable for real-time applications and reducing computational costs. This aligns with the broader trend towards efficient AI. The paper presents a theoretically grounded and empirically powerful correction to self-conditioning in Flow Matching for language models, offering a retraining-free path to high-efficiency, high-quality few-step generation.
Large Language Models (LLMs) achieve strong reasoning performance, but their robustness to realistic lexical corruption remains poorly understood. We evaluate four open-weight instruction-tuned models and frontier models across four reasoning benchmarks under keyboard noise, character swaps, and filler insertion. Character-level perturbations substantially degrade accuracy, especially on multi-step reasoning tasks, while filler insertion has little effect. We trace this asymmetry to Attention Diversion: lexical corruption fragments subword tokenization, and the resulting fragments attract disproportionate attention mass, concentrated in middle and final transformer layers. Length-matched controls confirm that fragmentation, not prompt length, drives the loss. A factorial intervention then shows why the damage is hard to undo: fragmentation corrupts token content and attention allocation together, and the two are coupled. Restoring clean attention while the content remains corrupted is actively harmful, restoring content alone is insufficient, and only restoring both recovers a substantial share of the gap. This coupling explains why inference-time strategies, including chain-of-thought prompting, spell-checking, self-repair, and stronger repair models, fail to consistently recover performance: each addresses one channel at a time. Code and data are available at https://github.com/Jiaqian-Janelle/Attention-Diversion
Primary: Missouri University of Science and Technology
All Institutions: Missouri University of Science and Technology, University of North Texas
The paper provides a rigorous mechanistic explanation for LLM fragility to lexical perturbations, identifying "Attention Diversion" caused by subword fragmentation as the primary driver of reasoning failure, and demonstrating through factorial interventions that token content and attention allocation are coupled in a way that renders single-channel repairs ineffective.
The paper employs a rigorous mechanistic interpretability approach to diagnose LLM robustness. It moves beyond simple accuracy metrics by introducing "Attention Diversion" as a causal mechanism, supported by factorial interventions (decoupling token content from attention allocation) and length-matched controls. The methodology is sound, well-controlled, and effectively isolates the specific failure mode of subword fragmentation. However, the core observation—that typos break tokenization and hurt performance—is a known phenomenon in NLP; the novelty lies in the detailed mechanistic tracing and the specific "coupling" finding, which is insightful but not paradigm-shifting.
The experimental setup is comprehensive, covering multiple model families (Llama, Mistral, Qwen, Gemma), scales (7B-70B), and frontier models (GPT-4o, GPT-5.4). The use of multiple reasoning benchmarks (BoolQ, PIQA, HellaSwag, GSM8K) and perturbation types (keyboard, swap, filler) provides strong empirical grounding. The results are consistent and statistically significant. The factorial intervention is the strongest part of the evaluation, providing clear causal evidence for the coupling hypothesis. The inclusion of Chinese/CMath data adds valuable cross-lingual generalization evidence.
The paper provides a GitHub repository link and describes the experimental protocols in sufficient detail for replication. The use of standard benchmarks and open-weight models enhances reproducibility. The specific interventions (embedding replacement, attention suppression) are described with enough technical detail to be implemented.
The paper acknowledges limitations, including the focus on English/QWERTY (though mitigated by Chinese data), synthetic perturbations, and the single-task nature of the factorial intervention. A key limitation is that the "repair" strategies tested are somewhat basic or ill-suited to the specific coupling problem identified, which limits the practical immediate impact of the mitigation section. The venue date (2026) is futuristic but treated as given.
This work has significant implications for the reliability of LLMs in real-world scenarios involving noisy input (e.g., voice-to-text, fast typing). It challenges the assumption that semantic preservation is sufficient for robustness and highlights the fragility of subword tokenization. It suggests that future robustness improvements must address tokenization-level defenses or training-time alignments rather than just inference-time prompting. The paper provides a rigorous mechanistic explanation for LLM fragility to lexical perturbations, identifying "Attention Diversion" caused by subword fragmentation as the primary driver of reasoning failure, and demonstrating through factorial interventions that token content and attention allocation are coupled in a way that renders single-channel repairs ineffective.
High-performance ML systems increasingly rely on GPU kernels whose editable source is unavailable, generated, or too distant from final machine code to expose remaining optimizations. Existing LLM kernel optimizers and autotuners mainly operate on CUDA, Triton, HIP, or tensor-program source and validate against reference implementations. We study a stricter setting: optimizing an already compiled AMDGPU code object, where the deployed binary is the only behavioral oracle. We present AsmEvo, an agentic assembly-level optimizer for AMD GPU kernels. Given an AMDGPU code object K0, AsmEvo reconstructs a reassemblable representation, proposes low-level edits with a long-horizon agent, rebuilds an ABI-preserving optimized object, and accepts candidates only after differential verification against K0 under identical launches. AsmEvo combines code-object recovery, metadata-aware rebuilding, profiling-guided hot-window editing, correctness-gated timing, and conservative in-place patch fallback. We conduct extensive experiments with AsmEvo on various AMD GPU kernels. On MI308X, AsmEvo improves 29 of 30 selected KernelBench kernels, reaching 1.35x geometric-mean and 3.88x maximum speedup. On MI300X production workloads, it improves all evaluated AITer binaries and vLLM/SGLang Triton assembly kernels, reaching 1.09x/1.31x and 1.18x/1.34x geometric-mean/maximum speedups, respectively, while preserving functional equivalence.
Primary: AMD
All Institutions: AMD
AsmEvo presents a novel and technically rigorous approach to post-compilation GPU kernel optimization, effectively bridging the gap between high-level ML frameworks and low-level hardware performance by leveraging LLMs for assembly-level editing under strict functional equivalence constraints.
The paper proposes AsmEvo, a system for optimizing AMD GPU kernels at the assembly level without source code. The core methodology involves three stages: (1) recovering a reassemblable AMDGCN assembly representation from the compiled ELF/HSACO binary, (2) using a long-horizon LLM agent to propose low-level edits within "hot windows" identified by profiling, and (3) enforcing a strict "gated verification" harness that rebuilds the binary, checks for ABI preservation, and verifies functional equivalence against the original binary (the oracle) before measuring performance. The approach is technically sound and addresses a real gap in the ML systems stack: post-compilation optimization of deployed artifacts. The use of a differential oracle (comparing output of optimized vs. original binary) is a robust way to handle correctness without source-level reference implementations. The separation of the LLM agent (exploration) from deterministic gates (verification) is a good architectural choice to prevent hallucinated or incorrect optimizations from being accepted.
The experiments are conducted on AMD MI308X and MI300X hardware. The evaluation covers KernelBench (L1 and L2) and production kernels from AITer, vLLM, and SGLang. The results show significant speedups, with a geometric mean of 1.35x on KernelBench and up to 3.88x on specific kernels. The inclusion of production workloads (AITer, Triton HSACOs) is a strong point, demonstrating real-world applicability. The verification process is rigorous, ensuring that speedups are not due to semantic changes. However, the sample size for production kernels is small (8 kernels), and the KernelBench results, while impressive, are on benchmark kernels that might not fully represent the complexity of end-to-end inference workloads. The comparison is primarily internal (optimized vs. original binary), which is appropriate for the problem setting, but lacks comparison to other state-of-the-art autotuners or compiler optimizations on the same binaries, as those typically require source.
The paper provides detailed descriptions of the recovery, rebuild, and verification pipelines. The use of standard tools (ROCm, AMDGCN) and the clear definition of the verification harness enhance reproducibility. However, the reliance on a specific LLM (Claude Opus 4.8) and the proprietary nature of some production kernels (AITer) may limit full reproducibility for external researchers. The code for the recovery and rebuild tools is likely not open-sourced in the same way as typical ML models, which could hinder adoption.
The primary limitation is the scope of the optimization: it is specific to AMD GPU architecture (AMDGCN). The recovery process might fail for complex or obfuscated binaries. The "real-dispatch capture" mechanism, while powerful, requires access to the running application's memory state, which may not always be feasible or safe in all deployment environments. The speedups, while significant, are bounded by the remaining optimization headroom in the compiled binary, which is often less than in source-level optimization. The paper acknowledges that equivalence is empirical, not formal.
This work has significant implications for the ML systems community, particularly for organizations that deploy compiled ML models where source code is unavailable or optimization at the source level is no longer possible. It demonstrates that there is still value in low-level, binary-level optimization. The methodology could be extended to other architectures (e.g., NVIDIA SASS) and could inspire new tools for binary-level program analysis and optimization. It also highlights the potential of LLMs for low-level systems programming tasks when combined with rigorous verification. AsmEvo presents a novel and technically rigorous approach to post-compilation GPU kernel optimization, effectively bridging the gap between high-level ML frameworks and low-level hardware performance by leveraging LLMs for assembly-level editing under strict functional equivalence constraints.
Long-context modeling is a pivotal capability for Large Language Models, yet the quadratic complexity of attention remains a critical bottleneck, particularly during the compute-intensive prefilling phase. Our previous work, FlashPrefill, mitigates this cost through instantaneous pattern discovery and max-based dynamic thresholding; however, it remains an algorithmic prototype that is still distant from production deployment. In this paper, we present FlashPrefill V2, which evolves FlashPrefill from a prototype toward practical long-context serving along three dimensions. First, we introduce a mean correction term that effectively suppresses the approximation error, keeping performance degradation manageable even at extreme sparsity levels. Second, we redesign the sparse attention operator with PackGQA memory access, warp specialization, and pingpong pipelining, fully aligning with the latest FlashAttention-3/4 implementations and supporting FP8 inference to meet practical quantization requirements. Third, FlashPrefill V2 natively supports paged KV cache and continuous batching, allowing integration as an attention backend in modern inference frameworks such as SGLang. Extensive evaluations on NVIDIA H20 GPUs---among the most widely deployed inference accelerators---demonstrate that FlashPrefill V2 delivers up to 47.26x and 27.19x speedups over FlashAttention-2 at 128K context length under FP8 and BF16 precision, respectively, and, in FP8, still achieves a 30.49x speedup against an FA3/4-aligned dense baseline.
Primary: Tencent WeChat
All Institutions: Tencent WeChat
FlashPrefill V2 presents a substantial systems-level contribution to long-context LLM serving by transforming a sparse attention prototype into a production-ready, highly optimized backend that achieves massive speedups on widely deployed hardware through advanced kernel engineering and integration with modern inference frameworks.
The paper proposes FlashPrefill V2, an optimization for the prefilling phase of Long-Context LLM serving. It builds upon a previous prototype (FlashPrefill) by introducing three key technical components: 1) A mean correction term to reduce approximation error in block-sparse attention, allowing for higher sparsity without significant accuracy loss. 2) A highly optimized CUDA kernel implementation leveraging PackGQA memory access, warp specialization, and ping-pong pipelining, aligned with FlashAttention-3/4 paradigms and supporting FP8 inference. 3) Integration support for paged KV cache and continuous batching, making it a viable backend for modern inference engines like SGLang. The methodology is sound and addresses a critical bottleneck (prefill latency) in long-context serving. The shift from algorithmic prototype to production-ready system is a significant engineering contribution. EXPERIMENTAL_EVALUTION: The evaluation is conducted on NVIDIA H20 GPUs, which are noted as widely deployed inference accelerators. The paper reports substantial speedups: up to 47.26x over FlashAttention-2 (FA2) at 128K context under FP8, and 27.19x under BF16. It also compares against an FA3/4-aligned dense baseline, showing a 30.49x speedup in FP8. The results demonstrate clear performance gains. However, the paper lacks detailed accuracy benchmarks (e.g., perplexity drops on standard datasets) beyond the claim that error is "manageable." The focus is heavily on throughput/latency, which is appropriate for a systems paper, but a more rigorous accuracy analysis would strengthen the claim of "practical" deployment.
The paper describes specific kernel optimizations (warp specialization, ping-pong pipelining) and integration points (SGLang). While the algorithmic description is clear, the full source code is not explicitly linked in the abstract or provided in the text snippet. Reproducibility depends on the availability of the code repository, which is not extracted here. The use of standard hardware (H20) and frameworks (SGLang) aids reproducibility if code is open-sourced.
The primary limitation is the reliance on block-sparse attention, which introduces approximation error. While the mean correction helps, it does not eliminate it, potentially affecting tasks requiring precise attention mechanisms. The evaluation is limited to NVIDIA H20 GPUs; performance on other architectures (e.g., H100, A100, or AMD GPUs) is not reported. The paper does not provide extensive accuracy degradation analysis across diverse benchmarks. The "extreme sparsity levels" mentioned in the abstract are not quantified in the provided text, making it hard to assess the trade-off curve.
This work has significant potential impact on the efficiency of long-context LLM serving, reducing costs and latency for applications requiring large context windows. By aligning with modern inference frameworks and supporting FP8, it promotes energy-efficient and cost-effective deployment of large models. The focus on production-ready systems bridges the gap between academic research and industrial application. FlashPrefill V2 presents a substantial systems-level contribution to long-context LLM serving by transforming a sparse attention prototype into a production-ready, highly optimized backend that achieves massive speedups on widely deployed hardware through advanced kernel engineering and integration with modern inference frameworks.
Humans imitate at the level of intent: given a demonstration, we infer its goal and carry it out with whatever tools, objects, and layouts are at hand. Current robot policies instead learn observation-to-action mappings from visual inputs and language instructions, without explicitly inferring the demonstrated task. Learning from human video thus remains largely trajectory-level: models can replay motions in near-identical scenes, but still struggle to imitate what the demonstrator intends rather than merely what they do. We introduce The Imitator Game, a four-level benchmark (L0-L3) that progressively widens the gap between the human demonstration and the robot's own scene, isolating where trajectory replay ceases to suffice and task understanding becomes necessary. We pair it with IG-10K, the largest environment-aligned paired human-robot dataset to date and the only one instantiated across all four levels in both real and simulated settings (20,000+ paired episodes, 50+ tasks, 6 domains), and Imitator Arena, an open platform for blind A/B human evaluation. Across nine state-of-the-art models, performance is stable from L0 to L2 but collapses at L3, identifying functional substitution - achieving the same intent through a different object affordance - as the decisive barrier to intent-level imitation. Human-video-conditioned models outperform caption-conditioned ones, yet every model falls below 13% zero-shot success on unseen tasks; fine-tuning IG-10K-pretrained models with only $10$ paired human-robot demonstrations yields large gains that grow with pretraining scale. The project website and access to Imitator Arena are available at https://imitator-game.github.io.
Primary: Zhejiang University
All Institutions: Zhejiang University
The paper presents a significant contribution to robotics by introducing a challenging benchmark and large-scale dataset that isolates the critical challenge of intent-level imitation, providing the field with a rigorous tool to measure and improve upon current limitations in robot learning.
The paper introduces "The Imitator Game," a structured benchmark designed to evaluate robot imitation beyond simple trajectory replay. It defines four levels of difficulty (L0-L3) that progressively increase the discrepancy between the human demonstration and the robot's environment, specifically targeting "functional substitution" at L3. The authors also release IG-10K, a large-scale paired human-robot dataset spanning 50+ tasks across 6 domains. The methodology involves training state-of-the-art vision-language-action models and evaluating their zero-shot and few-shot performance. The core methodological contribution is the rigorous isolation of intent understanding versus motor mimicry through this hierarchical benchmarking framework.
The experimental section is comprehensive, evaluating nine state-of-the-art models. Key findings include stable performance from L0 to L2 but a collapse at L3, identifying functional substitution as the primary barrier to intent-level imitation. The paper demonstrates that human-video-conditioned models outperform caption-conditioned ones. A significant result is the finding that fine-tuning with only 10 paired demonstrations yields large gains, suggesting that pre-training on large-scale data (like IG-10K) provides a strong prior for intent understanding. The use of "Imitator Arena" for blind A/B human evaluation adds a layer of subjective quality assessment that complements objective success rates.
The paper provides a project website and claims to release the IG-10K dataset and the benchmark environment. The dataset size (20,000+ episodes) and diversity (50+ tasks, 6 domains) suggest a substantial resource for the community. The availability of the benchmark and dataset significantly enhances reproducibility and allows for future comparative studies. The paper cites the specific URL for access, facilitating verification.
The primary limitation is the current performance ceiling; no model achieves high success on L3 tasks in zero-shot settings (below 13%). This indicates that while the benchmark is effective at exposing current limitations, it may be too challenging for existing architectures to solve without significant domain-specific tuning or architectural changes. Additionally, the reliance on human-video conditioning might limit applicability in scenarios where video demonstrations are not available, although the paper addresses this by comparing with caption-conditioned models. The generalization to unseen tasks remains poor, highlighting a gap between benchmark performance and real-world robustness.
This work has significant implications for the field of robotics and embodied AI. By clearly defining the gap between trajectory replay and intent understanding, it provides a clear roadmap for future research. The release of IG-10K and the benchmark framework will likely become standard tools for evaluating imitation learning capabilities. It shifts the focus from mere motion matching to semantic understanding of tasks, which is crucial for deploying robots in unstructured human environments. The findings suggest that current models are still largely "parrots" rather than "understanders," urging the community to develop more robust intent inference mechanisms. The paper presents a significant contribution to robotics by introducing a challenging benchmark and large-scale dataset that isolates the critical challenge of intent-level imitation, providing the field with a rigorous tool to measure and improve upon current limitations in robot learning.
Vision-Language-Action (VLA) models are typically trained with behavior cloning (BC) on expert demonstrations. However, BC provides only positive supervision for expert actions, without explicit negative supervision indicating which actions are instruction-inconsistent or otherwise inappropriate. Reinforcement learning (RL) can provide such corrective signals, but often relies on externally specified rewards or curated non-expert data, both of which are costly to obtain in robotics. We show that offline RL for VLA models need not rely on curated non-expert trajectories: successful expert demonstrations alone can be transformed into dense corrective supervision through instruction relabeling. Specifically, by pairing expert actions with mismatched alternative instructions, we synthesize counterfactual instruction-observation-action tuples from the dataset and combine them with adversarial discriminator training to learn an instruction-grounded reward model for offline RL, without collecting additional rollouts or annotations. On the robustness-focused LIBERO-PRO benchmark, our method improves robustness to object position and task perturbations over a strong state-of-the-art baseline. It also outperforms competitive baselines in real-robot experiments on the TX-G2 (compatible with AGIBot G2). More broadly, our results suggest that, for data-constrained VLA learning, extracting denser supervision from each demonstration can complement collecting additional data.
Primary: Unknown
All Institutions: Unknown
[One sentence main contribution]. CounterAlign introduces a counterfactual supervision mechanism for VLA models that synthesizes negative training samples from expert demonstrations via instruction relabeling, enabling robust offline RL without additional data collection. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a novel approach to improving the robustness of Vision-Language-Action models by addressing the lack of negative supervision in Behavior Cloning. By leveraging instruction relabeling to create counterfactual negative samples, the authors enable the training of an adversarial reward model that guides offline RL. This method is significant because it offers a data-efficient alternative to collecting non-expert trajectories or designing complex reward functions. The empirical results on LIBERO-PRO and real-robot experiments demonstrate improved robustness to perturbations, validating the effectiveness of the approach. While the novelty is moderate (as adversarial training and data augmentation are well-known concepts), the specific application to VLA models and the simplicity of the instruction-swapping mechanism make it a valuable contribution. The technical impact is solid, offering a practical improvement for VLA training pipelines. However, the incremental nature of the improvement and the computational costs prevent it from reaching the highest tiers of impact. The paper is well-written and addresses a relevant problem in the field, making it a strong candidate for publication in a top-tier robotics or ML venue.
The paper proposes "CounterAlign," a method to enhance Vision-Language-Action (VLA) models by generating negative supervision from positive expert demonstrations. The core idea involves "instruction relabeling," where expert action-observation tuples are paired with mismatched (counterfactual) instructions to create negative samples. These are used to train an adversarial discriminator that serves as a reward signal for offline Reinforcement Learning (RL). This approach aims to solve the lack of explicit negative feedback in Behavior Cloning (BC) without requiring costly non-expert data collection or external reward models. The methodology is theoretically sound, leveraging existing offline RL frameworks (like CQL or similar implicit Q-learning variants) but adapting the reward structure through a learned discriminator. The novelty lies in the specific mechanism of synthesizing negative data via instruction swapping rather than collecting it, which is a clever data-augmentation strategy for VLA training.
The evaluation is conducted on the LIBERO-PRO benchmark, which is a standard and rigorous testbed for VLA robustness, focusing on perturbations in object positions and task definitions. The paper reports improvements over a strong state-of-the-art baseline (likely RT-2 or similar BC-based VLA). Additionally, real-robot experiments on the TX-G2 platform are presented. The results show improved robustness to perturbations, which is the primary claim. However, the abstract-only score was 60, and the full text analysis suggests the gains, while positive, may be incremental. The comparison against other RL-based baselines or more recent SOTA VLAs is crucial; if the baseline is weak, the improvement might be overstated. The real-robot results add significant weight, as sim-to-real gaps are a major hurdle. The evaluation appears comprehensive for the scope, but the magnitude of improvement needs to be weighed against the computational cost of training the discriminator and the offline RL loop.
The paper describes the method in detail, including the instruction relabeling process and the adversarial training setup. However, as is common with arXiv submissions, the code is not explicitly linked in the provided text (URL extraction returned "none"). Reproducibility would depend on the availability of the code, which is not guaranteed. The use of standard benchmarks (LIBERO-PRO) aids in reproducibility of the evaluation metrics. The specific hyperparameters for the adversarial training and the offline RL algorithm are likely detailed in the full text, but without access to the code or supplementary material, exact replication might be challenging.
The primary limitation is the reliance on the quality of the instruction relabeling. If the language model or heuristic used to generate "mismatched" instructions is not sufficiently diverse or semantically plausible, the negative samples may not provide strong enough gradients for the discriminator. Furthermore, the method assumes that expert demonstrations are available, which is the same constraint as BC. The computational overhead of training the discriminator and running the offline RL loop is higher than pure BC. The paper also notes that the method has not been tested on larger, more diverse datasets, limiting the generalizability of the claims.
This work contributes to the field of embodied AI by providing a more data-efficient training paradigm for VLA models. By extracting denser supervision from existing data, it reduces the need for expensive data collection, which is a major bottleneck in robotics. This could accelerate the development of robust robotic systems. However, the reliance on expert data means that biases in the expert demonstrations could be amplified if not carefully handled. The method also highlights the importance of negative supervision in learning robust policies, a insight that may extend beyond VLA models to other sequential decision-making tasks. [One sentence main contribution]. CounterAlign introduces a counterfactual supervision mechanism for VLA models that synthesizes negative training samples from expert demonstrations via instruction relabeling, enabling robust offline RL without additional data collection. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a novel approach to improving the robustness of Vision-Language-Action models by addressing the lack of negative supervision in Behavior Cloning. By leveraging instruction relabeling to create counterfactual negative samples, the authors enable the training of an adversarial reward model that guides offline RL. This method is significant because it offers a data-efficient alternative to collecting non-expert trajectories or designing complex reward functions. The empirical results on LIBERO-PRO and real-robot experiments demonstrate improved robustness to perturbations, validating the effectiveness of the approach. While the novelty is moderate (as adversarial training and data augmentation are well-known concepts), the specific application to VLA models and the simplicity of the instruction-swapping mechanism make it a valuable contribution. The technical impact is solid, offering a practical improvement for VLA training pipelines. However, the incremental nature of the improvement and the computational costs prevent it from reaching the highest tiers of impact. The paper is well-written and addresses a relevant problem in the field, making it a strong candidate for publication in a top-tier robotics or ML venue.
We introduce Accelerating Dexterity via Pre-Training (ADEPT), a large-scale reinforcement learning (RL) framework for learning sim-to-real transferable dexterity across high degree-of-freedom (DoF) robot embodiments that can solve long-horizon tasks directly from raw visuo-tactile perception. ADEPT pretrains a dexterous policy on a generic object reposing task, then post-trains downstream policies with this pretrained behavior as a prior. ADEPT enables learning new behaviors that are otherwise difficult to discover from scratch on multi-fingered robots and avoids learning the same set of skills over again for every new downstream task. The pretrained policy zero-shots the reposing phase of downstream tasks, but naïve RL fine-tuning rapidly degrades this capability during transfer. We address this with a stable post-training recipe combining behavior-cloning distillation, critic warm-up, and conservative on-policy updates. To safely exploit the full kinematic dexterity, we introduce a joint-space Geometric Fabric that mediates between the RL policy and the robot. We distill post-trained teachers into perceptive students that zero-shot sim-to-real transfer on two embodiments: a 23 DoF Kuka-Allegro with two RGB cameras, and a 29 DoF Flexiv-Sharpa with two RGB cameras and five vision-based tactile sensors, and can solve long-horizon tasks from challenging initial states with dexterity at human-level speed.
Primary: University of Toronto
All Institutions: University of Toronto, Vector Institute, NVIDIA
The paper presents a significant advancement in dexterous manipulation via RL, introducing a robust pre-training and post-training framework that enables stable and efficient learning of complex manipulation skills. The combination of behavioral cloning distillation, critic warm-up, and conservative updates effectively mitigates the instability often associated with fine-tuning RL policies, while the Geometric Fabric ensures safe and feasible control. The successful sim-to-real transfer on multiple high-DoF platforms demonstrates the practical utility and generalizability of the approach, making it a valuable contribution to the field of robotics and reinforcement learning.
The paper introduces ADEPT, a framework for learning dexterous manipulation policies using Reinforcement Learning (RL). The core methodological contribution is a two-stage training pipeline: 1) Pre-training a policy on a generic "reposing" task to learn basic object handling and kinematic feasibility, and 2) Post-training (fine-tuning) on specific downstream tasks. Crucially, the authors address the issue of "catastrophic forgetting" or instability during fine-tuning by introducing a stable recipe combining behavior cloning (BC) distillation, critic warm-up, and conservative on-policy updates. They also introduce a "Geometric Fabric" in the joint space to mediate between the RL policy and the robot, ensuring safe and feasible control. The approach leverages large-scale simulation data and transfers to real-world embodiments (Kuka-Allegro and Flexiv-Sharpa). The methodology is sound and addresses a significant bottleneck in dexterous manipulation: the difficulty of discovering complex manipulation skills from scratch via RL.
The evaluation is extensive and rigorous. The authors demonstrate zero-shot sim-to-real transfer on two distinct high-DoF robotic platforms. They show that ADEPT enables the robots to solve long-horizon tasks (like object reposing and manipulation) from challenging initial states. The results indicate that the pre-training phase significantly accelerates learning and improves the stability of the post-training phase compared to training from scratch. The comparison against baselines (likely standard RL or BC-only approaches) highlights the effectiveness of the proposed post-training recipe. The ability to transfer to real hardware without further tuning is a strong empirical result.
The paper provides detailed descriptions of the simulation environment, the robot models, and the training hyperparameters. The code is made available on GitHub, and the project website includes demo videos. The use of standard simulators (Isaac Gym) and well-known robot models enhances reproducibility. The authors explicitly discuss the limitations and the specific conditions under which the method works, aiding in replication.
The method relies heavily on high-quality simulation-to-real transfer, which can still be sensitive to domain gaps not captured in simulation (e.g., friction variations, sensor noise). The "Geometric Fabric" adds a layer of complexity to the control stack. The pre-training on a generic task requires careful design to ensure the learned priors are useful for diverse downstream tasks. The computational cost of pre-training on large-scale RL data is significant.
This work contributes to the advancement of general-purpose robotic manipulation, a key step towards more autonomous and versatile robots in unstructured environments. By making dexterous manipulation more accessible via RL, it lowers the barrier for research and application in this domain. The open-source release of code and models promotes further research and development in the field. The paper presents a significant advancement in dexterous manipulation via RL, introducing a robust pre-training and post-training framework that enables stable and efficient learning of complex manipulation skills. The combination of behavioral cloning distillation, critic warm-up, and conservative updates effectively mitigates the instability often associated with fine-tuning RL policies, while the Geometric Fabric ensures safe and feasible control. The successful sim-to-real transfer on multiple high-DoF platforms demonstrates the practical utility and generalizability of the approach, making it a valuable contribution to the field of robotics and reinforcement learning.