Last 7 Days (August 20 – August 26, 2026)
We prove a depth hierarchy for ReLU neural networks in which every additional ReLU layer can save exponentially many neurons. For every $\ell\geq 3$, a globally $[0,1]$-valued, $1$-Lipschitz function is realized by a depth-$\ell$ network of width $\mathcal{O}(d^4)$, whereas every depth-$(\ell-1)$ network with unrestricted weights and width at most $2^d/[2d(\ell-2)]$ has squared $L_2$ error at least $1/24$ under an absolutely continuous distribution. To the best of our knowledge, this is the first exponential separation for ReLU networks between two fixed depths whose shallower depth is at least $3$, and the first exponential hierarchy across all adjacent fixed depths. The lower bound also immediately yields the corresponding hierarchy for exact computation. Moreover, the case $\ell=3$ gives a compactly supported depth-$3$-versus-depth-$2$ separation with unrestricted shallow-network weights, answering a question raised by Safran, Eldan, and Shamir (2019, Sec. 2.3). The corresponding distribution nevertheless has all its mass at exponential radius, so the construction falls outside the regularity regime in which such a separation would imply major threshold-circuit lower bounds. We also prove an exact separation for a more benign function. It is computed by a polynomial-width depth-$4$ network, whereas every depth-$3$ network agreeing with it on the unit hypercube requires exponentially many neurons in its first hidden layer, again without any restriction on the weights. The function is globally $[0,1]$-valued and $\mathcal{O}(\sqrt d)$-Lipschitz, and maps the unit hypercube onto $[0,1]$.
Primary: Weizmann Institute of Science
All Institutions: Weizmann Institute of Science
[One sentence main contribution]. This paper proves the first exponential depth hierarchy for ReLU networks between any two adjacent fixed depths $k \ge 3$, demonstrating that every additional layer can save exponentially many neurons. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The work is a landmark result in the theoretical foundations of deep learning. It resolves a longstanding open problem by establishing that the exponential advantage of depth is not limited to the specific case of depth 3 vs 2, but holds for all adjacent depths. The methodology, involving the recursive copying of hard functions via geometric perturbations, is elegant and powerful. While the exponential radius requirement limits its immediate implication for circuit complexity, it provides a complete characterization of the expressive power of ReLU networks with respect to depth in the $L_2$ approximation setting. The exact separation result for depth 4 vs 3 on the unit hypercube is also significant, offering a more "benign" setting for future investigations. This paper will serve as a definitive reference for researchers studying the expressivity and complexity of neural networks.
The paper presents a rigorous theoretical contribution to the understanding of ReLU neural network expressivity. The core methodology involves constructing specific families of continuous piecewise-linear (CPWL) functions that exhibit exponential depth hierarchies. The authors employ a recursive construction technique: they define a "copier" mechanism that creates exponentially many translated and rescaled copies of a hard base function. By perturbing the Boolean hypercube vertices into quantitative affine general position, they ensure that any shallow network (depth $k-1$) must use an exponential number of neurons to capture the geometry of these copies. The proof relies on geometric arguments regarding the intersection of affine hyperplanes with these perturbed structures. The approach is mathematically sophisticated, leveraging tools from approximation theory, geometry of numbers, and circuit complexity lower bounds.
This is a theoretical paper with no empirical experiments, datasets, or code. The "evaluation" consists of formal mathematical proofs of lower bounds. The validity of the paper rests entirely on the correctness of the theorems and their proofs, which appear rigorous and well-structured.
As a theoretical work, reproducibility refers to the verifiability of the proofs. The paper provides detailed constructions (e.g., the perturbation map, the copier function) and lemmas that support the main theorems. The appendices contain full proofs. While not "reproducible" in the empirical sense, the logical flow is transparent and self-contained.
The primary limitation, acknowledged by the authors, is the normalization scale. The $L_2$ separation result requires the target distribution to have support at an exponential radius from the origin. This places the result outside the "regularity regime" (polynomial radius, bounded Lipschitz constant) where the Vardi-Shamir reduction to threshold circuit lower bounds applies. Consequently, while it resolves the fixed-depth gap for ReLU networks, it does not immediately resolve the corresponding open problems in Boolean circuit complexity. The exact computation separation is more benign (polynomial radius) but only applies to depths 4 vs 3, not the full adjacent-depth hierarchy.
This paper addresses a fundamental question in deep learning theory: the necessity of depth. By proving that every additional layer can save exponentially many neurons for ReLU networks (at least for depths $\ge 3$), it provides strong theoretical justification for the use of deep architectures in practice. It bridges the gap between neural network approximation theory and classical circuit complexity, highlighting both the similarities and the specific barriers (like the natural proofs barrier) that prevent direct translation of results. It clarifies the landscape of what is known about depth hierarchies and identifies the precise conditions under which exponential separations can and cannot be proven. [One sentence main contribution]. This paper proves the first exponential depth hierarchy for ReLU networks between any two adjacent fixed depths $k \ge 3$, demonstrating that every additional layer can save exponentially many neurons. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The work is a landmark result in the theoretical foundations of deep learning. It resolves a longstanding open problem by establishing that the exponential advantage of depth is not limited to the specific case of depth 3 vs 2, but holds for all adjacent depths. The methodology, involving the recursive copying of hard functions via geometric perturbations, is elegant and powerful. While the exponential radius requirement limits its immediate implication for circuit complexity, it provides a complete characterization of the expressive power of ReLU networks with respect to depth in the $L_2$ approximation setting. The exact separation result for depth 4 vs 3 on the unit hypercube is also significant, offering a more "benign" setting for future investigations. This paper will serve as a definitive reference for researchers studying the expressivity and complexity of neural networks.
Search-augmented reasoning remains difficult for small language models. On-policy distillation (OPD) from trained teachers offers a promising direction, but suffers from two issues: (1) high-quality multi-turn search trajectories depend on dynamic retriever responses, making SFT data prohibitively expensive to collect at scale; (2) task-specifically trained teachers incur substantial training cost, while directly applying OPD with an off-the-shelf teacher without task-specific fine-tuning constrains the student to the teacher's performance ceiling and suffers from severe training instability. We propose OPDSearch+, the first distillation paradigm that requires no teacher fine-tuning for search-augmented reasoning. We investigate the role of a frozen off-the-shelf instruct model as the teacher in on-policy distillation, and reveal a key insight: the teacher reshapes the student's policy distribution so that subsequent RL converges to a superior solution that RL alone cannot reach. In stage one, the student interacts with a live search engine and is distilled via a per-position forward KL objective, transferring reasoning decomposition and evidence integration skills without any task-specific teacher training. In stage two, RL refines the distilled student from a richer behavioral foundation, achieving performance that RL alone cannot reach from scratch. Across seven QA benchmarks, OPDSearch+ with a 3B model consistently outperforms all prior 3B RL baselines, achieving gains of 13.1% on HotpotQA and 8.5% on 2WikiMultihopQA.
Primary: University of Chinese Academy of Sciences
All Institutions: University of Chinese Academy of Sciences, Institute of Computing Technology, Chinese Academy of Sciences, Institute of Automation, Chinese Academy of Sciences, University of Macau, Wuhan University, Georgia Institute of Technology, Northwestern Polytechnical University, University of Hong Kong
[One sentence main contribution]. OPDSearch+ introduces a novel two-stage on-policy distillation and RL refinement framework that leverages a frozen off-the-shelf teacher to stabilize training and enhance performance in search-augmented reasoning for small language models. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a significant advancement in the field of search-augmented reasoning for small language models. By addressing the critical issues of data collection costs and training instability associated with traditional on-policy distillation, OPDSearch+ offers a practical and effective solution. The theoretical analysis of forward-KL divergence in this context is a valuable contribution, providing insights into gradient variance and entropy collapse. The empirical results demonstrate clear improvements over state-of-the-art baselines, particularly in multi-hop reasoning tasks. The work is well-motivated, rigorously evaluated, and has the potential to influence future research in agent-based learning and model distillation.
The paper proposes OPDSearch+, a two-stage framework for search-augmented reasoning in small language models (SLMs). Stage 1 uses On-Policy Distillation (OPD) with a frozen off-the-shelf teacher (Qwen2.5-14B-Instruct) to distill token-level distributions onto student-generated trajectories using a forward-KL objective. Stage 2 refines this distilled policy using Reinforcement Learning (RL) with Group Relative Policy Optimization (GRPO). The core theoretical contribution is the analysis of forward-KL vs. reverse-KL in the context of on-policy distillation for interactive agents, arguing that forward-KL provides better gradient variance control and prevents entropy collapse. The methodology is sound, addressing the distribution shift problem inherent in offline SFT for dynamic retrieval environments. The use of a frozen generalist teacher to provide implicit supervision for retrieval quality is a clever and cost-effective design choice.
The experimental evaluation is comprehensive, covering seven QA benchmarks (NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultihopQA, MuSiQue, Bamboogle). The results show that OPDSearch+ (3B) outperforms all prior 3B RL baselines, including Search-R1 and GiGPO, with significant gains on multi-hop tasks (e.g., +13.1% on HotpotQA). The ablation studies are thorough, comparing against offline SFT, pure RL, pure OPD, and joint optimization, confirming the superiority of the two-stage approach. The analysis of training dynamics and gradient stability supports the theoretical claims. The comparison with reverse-KL variants demonstrates the practical necessity of the proposed forward-KL objective.
The paper provides detailed hyperparameters, model versions (Qwen2.5-3B/14B), and training configurations (veRL framework, 4 H200 GPUs). The description of the loss functions and masking strategies is clear. The code is not explicitly linked in the text provided, but the method relies on standard components (GRPO, KL divergence) and open-source models, making reproduction feasible for researchers with similar resources. The specific clipping parameters and learning rates are mentioned, aiding reproducibility.
The primary limitation is the reliance on a significantly larger teacher model (14B) for distillation, which incurs inference costs during the training phase, although no fine-tuning of the teacher is required. The method is evaluated only on QA benchmarks; its performance on other reasoning tasks (e.g., mathematical reasoning, code generation) is not explored. The "frozen" teacher assumption might limit the transfer of highly task-specific nuances that a fine-tuned teacher could provide, although the authors argue this is mitigated by the subsequent RL stage. The computational cost of generating on-policy trajectories with a live search engine is non-trivial.
This work contributes to the democratization of advanced reasoning capabilities by enabling small models to perform complex search-augmented tasks efficiently. It reduces the dependency on expensive, task-specific teacher training, lowering the barrier to entry for developing capable search agents. The insights into KL divergence directions for stability in interactive settings are broadly applicable to other agent-based learning scenarios. [One sentence main contribution]. OPDSearch+ introduces a novel two-stage on-policy distillation and RL refinement framework that leverages a frozen off-the-shelf teacher to stabilize training and enhance performance in search-augmented reasoning for small language models. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a significant advancement in the field of search-augmented reasoning for small language models. By addressing the critical issues of data collection costs and training instability associated with traditional on-policy distillation, OPDSearch+ offers a practical and effective solution. The theoretical analysis of forward-KL divergence in this context is a valuable contribution, providing insights into gradient variance and entropy collapse. The empirical results demonstrate clear improvements over state-of-the-art baselines, particularly in multi-hop reasoning tasks. The work is well-motivated, rigorously evaluated, and has the potential to influence future research in agent-based learning and model distillation.
We settle the minimax-optimal alternating regret, a regret notion motivated by alternating learning dynamics in games, for both online linear optimization (OLO) and online convex optimization (OCO). For OLO over the probability simplex $Δ_d$, we give an algorithm with $O(\log d)$ alternating regret that remains a constant for any time horizon $T$, and a matching lower bound. Our constant regret bound significantly improves previous results with $O(\log ^{2/3}d \cdot T^{1/3})$ regret [Cevher, Cutkosky, Kavis, Piliouras, Skoulakis, Viano, NeurIPS 2023, Hait, Li, Luo, Zhang, COLT 2025]. As a result, we obtain alternating learning dynamics with $O(\log d /T)$ convergence to Nash equilibria in two-player zero-sum games and $O(\log d /T)$ convergence to coarse correlated equilibria in two-player general-sum games. This is the first uncoupled learning dynamics with $O(1/T)$ convergence to CCE in two-player general-sum games, while all prior works suffer additional $\log T$ factors. For general OCO over a $d$-dimensional compact convex set, we give an algorithm with $O(d\log (1+T/d))$ alternating regret, improving the previous best of $\widetilde{O}(d^{2/3}T^{1/3})$. We also prove a matching lower bound of $Ω(d\log (1+T/d))$, showing that the $Ω(\log T)$ factor is unavoidable.
Primary: Shanghai University of Finance and Economics
All Institutions: Shanghai University of Finance and Economics, Yale University
The paper makes a significant theoretical contribution by establishing the minimax-optimal alternating regret for OLO and OCO, providing tight upper and lower bounds that improve upon previous state-of-the-art results and enabling faster convergence rates in uncoupled learning dynamics for games.
The paper proposes a novel online learning algorithm, "Alternation-Aware Hedge" (AA-Hedge), specifically designed to minimize alternating regret in Online Linear Optimization (OLO) and Online Convex Optimization (OCO). The core methodological innovation lies in exploiting the structure of alternating updates in two-player games, where the learner observes the opponent's previous move before making their own. The authors derive a potential function analysis that yields a constant $O(d)$ alternating regret for OLO over the simplex, a significant improvement over previous $O(T^{1/3})$ bounds. For general OCO, they propose a continuous variant ("Continuous AA-Hedge") achieving $O(d(1+T/d))$ regret. The theoretical framework is rigorous, involving detailed proofs of feasibility, potential monotonicity, and matching lower bounds for both settings. The extension from discrete simplex to general convex bodies via measure-theoretic arguments is technically sophisticated.
The paper is primarily theoretical. It does not include empirical experiments on standard game benchmarks or large-scale simulations. The "evaluation" consists of theoretical lower bound constructions and asymptotic complexity analysis. While the theoretical results are strong, the lack of empirical validation on practical game scenarios (e.g., poker variants, large-scale matrix games) limits the immediate assessment of practical performance compared to the theoretical bounds. However, in theoretical ML research, rigorous bounds often suffice for high-impact publication if the problem is well-posed and the bounds are tight.
The algorithms (AA-Hedge and Continuous AA-Hedge) are fully described with pseudocode and parameter settings. The proofs are provided in the appendix. The work is highly reproducible from a theoretical standpoint. The authors acknowledge the use of AI assistants for proof generation, which is a transparency note but does not invalidate the mathematical content provided it is verified (which the text claims was done).
The primary limitation is the lack of empirical validation. The theoretical bounds are asymptotic; finite-sample performance or behavior on non-convex/non-smooth variants of games is not addressed. Additionally, the "alternating" setting assumes a specific turn-based structure which, while common in some game-playing contexts (like CFR variants), is less standard than simultaneous move dynamics in many general ML applications. The $O(d)$ constant might be large in practice for high-dimensional $d$, though it is independent of $T$.
This work settles a long-standing open question regarding the minimax-optimal alternating regret. It provides a theoretical foundation for faster convergence in uncoupled learning dynamics for general-sum games, specifically achieving the first $O(1/T)$ convergence to Coarse Correlated Equilibria (CCE) without logarithmic factors. This has significant implications for the design of efficient, decentralized learning algorithms in multi-agent systems and game theory. The paper makes a significant theoretical contribution by establishing the minimax-optimal alternating regret for OLO and OCO, providing tight upper and lower bounds that improve upon previous state-of-the-art results and enabling faster convergence rates in uncoupled learning dynamics for games.
Search-augmented reasoning remains difficult for small language models. On-policy distillation (OPD) from trained teachers offers a promising direction, but suffers from two issues: (1) high-quality multi-turn search trajectories depend on dynamic retriever responses, making SFT data prohibitively expensive to collect at scale; (2) task-specifically trained teachers incur substantial training cost, while directly applying OPD with an off-the-shelf teacher without task-specific fine-tuning constrains the student to the teacher's performance ceiling and suffers from severe training instability. We propose OPDSearch+, the first distillation paradigm that requires no teacher fine-tuning for search-augmented reasoning. We investigate the role of a frozen off-the-shelf instruct model as the teacher in on-policy distillation, and reveal a key insight: the teacher reshapes the student's policy distribution so that subsequent RL converges to a superior solution that RL alone cannot reach. In stage one, the student interacts with a live search engine and is distilled via a per-position forward KL objective, transferring reasoning decomposition and evidence integration skills without any task-specific teacher training. In stage two, RL refines the distilled student from a richer behavioral foundation, achieving performance that RL alone cannot reach from scratch. Across seven QA benchmarks, OPDSearch+ with a 3B model consistently outperforms all prior 3B RL baselines, achieving gains of 13.1% on HotpotQA and 8.5% on 2WikiMultihopQA.
Primary: University of Chinese Academy of Sciences
All Institutions: University of Chinese Academy of Sciences, Institute of Computing Technology, Chinese Academy of Sciences, Institute of Automation, Chinese Academy of Sciences, University of Macau, Wuhan University, Georgia Institute of Technology, Northwestern Polytechnical University, University of Hong Kong
[One sentence main contribution]. OPDSearch+ introduces a novel two-stage on-policy distillation and RL refinement framework that leverages a frozen off-the-shelf teacher to stabilize training and enhance performance in search-augmented reasoning for small language models. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a significant advancement in the field of search-augmented reasoning for small language models. By addressing the critical issues of data collection costs and training instability associated with traditional on-policy distillation, OPDSearch+ offers a practical and effective solution. The theoretical analysis of forward-KL divergence in this context is a valuable contribution, providing insights into gradient variance and entropy collapse. The empirical results demonstrate clear improvements over state-of-the-art baselines, particularly in multi-hop reasoning tasks. The work is well-motivated, rigorously evaluated, and has the potential to influence future research in agent-based learning and model distillation.
The paper proposes OPDSearch+, a two-stage framework for search-augmented reasoning in small language models (SLMs). Stage 1 uses On-Policy Distillation (OPD) with a frozen off-the-shelf teacher (Qwen2.5-14B-Instruct) to distill token-level distributions onto student-generated trajectories using a forward-KL objective. Stage 2 refines this distilled policy using Reinforcement Learning (RL) with Group Relative Policy Optimization (GRPO). The core theoretical contribution is the analysis of forward-KL vs. reverse-KL in the context of on-policy distillation for interactive agents, arguing that forward-KL provides better gradient variance control and prevents entropy collapse. The methodology is sound, addressing the distribution shift problem inherent in offline SFT for dynamic retrieval environments. The use of a frozen generalist teacher to provide implicit supervision for retrieval quality is a clever and cost-effective design choice.
The experimental evaluation is comprehensive, covering seven QA benchmarks (NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultihopQA, MuSiQue, Bamboogle). The results show that OPDSearch+ (3B) outperforms all prior 3B RL baselines, including Search-R1 and GiGPO, with significant gains on multi-hop tasks (e.g., +13.1% on HotpotQA). The ablation studies are thorough, comparing against offline SFT, pure RL, pure OPD, and joint optimization, confirming the superiority of the two-stage approach. The analysis of training dynamics and gradient stability supports the theoretical claims. The comparison with reverse-KL variants demonstrates the practical necessity of the proposed forward-KL objective.
The paper provides detailed hyperparameters, model versions (Qwen2.5-3B/14B), and training configurations (veRL framework, 4 H200 GPUs). The description of the loss functions and masking strategies is clear. The code is not explicitly linked in the text provided, but the method relies on standard components (GRPO, KL divergence) and open-source models, making reproduction feasible for researchers with similar resources. The specific clipping parameters and learning rates are mentioned, aiding reproducibility.
The primary limitation is the reliance on a significantly larger teacher model (14B) for distillation, which incurs inference costs during the training phase, although no fine-tuning of the teacher is required. The method is evaluated only on QA benchmarks; its performance on other reasoning tasks (e.g., mathematical reasoning, code generation) is not explored. The "frozen" teacher assumption might limit the transfer of highly task-specific nuances that a fine-tuned teacher could provide, although the authors argue this is mitigated by the subsequent RL stage. The computational cost of generating on-policy trajectories with a live search engine is non-trivial.
This work contributes to the democratization of advanced reasoning capabilities by enabling small models to perform complex search-augmented tasks efficiently. It reduces the dependency on expensive, task-specific teacher training, lowering the barrier to entry for developing capable search agents. The insights into KL divergence directions for stability in interactive settings are broadly applicable to other agent-based learning scenarios. [One sentence main contribution]. OPDSearch+ introduces a novel two-stage on-policy distillation and RL refinement framework that leverages a frozen off-the-shelf teacher to stabilize training and enhance performance in search-augmented reasoning for small language models. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a significant advancement in the field of search-augmented reasoning for small language models. By addressing the critical issues of data collection costs and training instability associated with traditional on-policy distillation, OPDSearch+ offers a practical and effective solution. The theoretical analysis of forward-KL divergence in this context is a valuable contribution, providing insights into gradient variance and entropy collapse. The empirical results demonstrate clear improvements over state-of-the-art baselines, particularly in multi-hop reasoning tasks. The work is well-motivated, rigorously evaluated, and has the potential to influence future research in agent-based learning and model distillation.
We settle the minimax-optimal alternating regret, a regret notion motivated by alternating learning dynamics in games, for both online linear optimization (OLO) and online convex optimization (OCO). For OLO over the probability simplex $Δ_d$, we give an algorithm with $O(\log d)$ alternating regret that remains a constant for any time horizon $T$, and a matching lower bound. Our constant regret bound significantly improves previous results with $O(\log ^{2/3}d \cdot T^{1/3})$ regret [Cevher, Cutkosky, Kavis, Piliouras, Skoulakis, Viano, NeurIPS 2023, Hait, Li, Luo, Zhang, COLT 2025]. As a result, we obtain alternating learning dynamics with $O(\log d /T)$ convergence to Nash equilibria in two-player zero-sum games and $O(\log d /T)$ convergence to coarse correlated equilibria in two-player general-sum games. This is the first uncoupled learning dynamics with $O(1/T)$ convergence to CCE in two-player general-sum games, while all prior works suffer additional $\log T$ factors. For general OCO over a $d$-dimensional compact convex set, we give an algorithm with $O(d\log (1+T/d))$ alternating regret, improving the previous best of $\widetilde{O}(d^{2/3}T^{1/3})$. We also prove a matching lower bound of $Ω(d\log (1+T/d))$, showing that the $Ω(\log T)$ factor is unavoidable.
Primary: Shanghai University of Finance and Economics
All Institutions: Shanghai University of Finance and Economics, Yale University
The paper makes a significant theoretical contribution by establishing the minimax-optimal alternating regret for OLO and OCO, providing tight upper and lower bounds that improve upon previous state-of-the-art results and enabling faster convergence rates in uncoupled learning dynamics for games.
The paper proposes a novel online learning algorithm, "Alternation-Aware Hedge" (AA-Hedge), specifically designed to minimize alternating regret in Online Linear Optimization (OLO) and Online Convex Optimization (OCO). The core methodological innovation lies in exploiting the structure of alternating updates in two-player games, where the learner observes the opponent's previous move before making their own. The authors derive a potential function analysis that yields a constant $O(d)$ alternating regret for OLO over the simplex, a significant improvement over previous $O(T^{1/3})$ bounds. For general OCO, they propose a continuous variant ("Continuous AA-Hedge") achieving $O(d(1+T/d))$ regret. The theoretical framework is rigorous, involving detailed proofs of feasibility, potential monotonicity, and matching lower bounds for both settings. The extension from discrete simplex to general convex bodies via measure-theoretic arguments is technically sophisticated.
The paper is primarily theoretical. It does not include empirical experiments on standard game benchmarks or large-scale simulations. The "evaluation" consists of theoretical lower bound constructions and asymptotic complexity analysis. While the theoretical results are strong, the lack of empirical validation on practical game scenarios (e.g., poker variants, large-scale matrix games) limits the immediate assessment of practical performance compared to the theoretical bounds. However, in theoretical ML research, rigorous bounds often suffice for high-impact publication if the problem is well-posed and the bounds are tight.
The algorithms (AA-Hedge and Continuous AA-Hedge) are fully described with pseudocode and parameter settings. The proofs are provided in the appendix. The work is highly reproducible from a theoretical standpoint. The authors acknowledge the use of AI assistants for proof generation, which is a transparency note but does not invalidate the mathematical content provided it is verified (which the text claims was done).
The primary limitation is the lack of empirical validation. The theoretical bounds are asymptotic; finite-sample performance or behavior on non-convex/non-smooth variants of games is not addressed. Additionally, the "alternating" setting assumes a specific turn-based structure which, while common in some game-playing contexts (like CFR variants), is less standard than simultaneous move dynamics in many general ML applications. The $O(d)$ constant might be large in practice for high-dimensional $d$, though it is independent of $T$.
This work settles a long-standing open question regarding the minimax-optimal alternating regret. It provides a theoretical foundation for faster convergence in uncoupled learning dynamics for general-sum games, specifically achieving the first $O(1/T)$ convergence to Coarse Correlated Equilibria (CCE) without logarithmic factors. This has significant implications for the design of efficient, decentralized learning algorithms in multi-agent systems and game theory. The paper makes a significant theoretical contribution by establishing the minimax-optimal alternating regret for OLO and OCO, providing tight upper and lower bounds that improve upon previous state-of-the-art results and enabling faster convergence rates in uncoupled learning dynamics for games.
Tensor-valued prediction is fundamental to geometric deep learning, yet uncertainty quantification (UQ) for such outputs remains an open challenge. While E(3)-equivariant neural networks excel at point estimates, they lack rigorous confidence measures. We focus on symmetric rank-2 tensor prediction, where the target has six Kelvin--Mandel coordinates and full uncertainty is represented by a $6\times6$ covariance matrix. We introduce a framework for E(3)-equivariant UQ, modeling the full predictive distribution where both mean and covariance preserve rotational symmetry. Our approach decomposes the covariance into irreducible representations $\mathrm{Sym}^2(ρ_c) \cong 2\times(l=0) \oplus 2\times(l=2) \oplus 1\times(l=4)$. By mapping from the flat Lie algebra $\mathfrak{sym}(6)$ to the curved SPD manifold via matrix exponentiation, we strictly ensure positive-definite covariances while maintaining exact equivariance. Furthermore, we formulate a Log-Euclidean Equivariant Scoring Objective (LE-ESO)---a robust surrogate loss based on the Multivariate Laplace distribution---providing robustness to heavy-tailed errors and stable optimization. Validation on ModelNet40 inertia tensors and Materials Project dielectric tensors demonstrates that our method achieves competitive performance and provides physically consistent, symmetry-preserving uncertainty estimates with useful risk and OOD sensitivity.
Primary: Fudan University
All Institutions: Fudan University, ByteDance, East China University of Science and Technology
This paper presents a rigorous and mathematically elegant framework for equivariant uncertainty quantification in tensor-valued geometric learning, addressing a critical gap in the field by combining representation theory with Riemannian geometry to ensure physically consistent, positive-definite, and rotationally equivariant covariance estimates.
The paper addresses a significant gap in geometric deep learning: the lack of rigorous uncertainty quantification (UQ) for tensor-valued predictions that respects E(3) equivariance. The core methodological contribution is the construction of an equivariant covariance matrix for symmetric rank-2 tensors. The authors correctly identify that standard Cholesky decomposition breaks equivariance in the Kelvin-Mandel representation, while direct regression fails to guarantee positive-definiteness. Their solution—parameterizing the covariance via the matrix exponential of an equivariant operator in the Lie algebra $\mathfrak{sym}(6)$—is mathematically sound and elegant. The decomposition of the covariance into irreducible representations ($l=0, 2, 4$) provides a structured, physically motivated parameterization. The introduction of the Log-Euclidean Equivariant Scoring Objective (LE-ESO) based on the Multivariate Laplace distribution is a practical innovation for robust training. The approach is theoretically rigorous, leveraging representation theory to hard-constrain the network's output space, which is a superior approach to soft regularization.
The experimental validation is conducted on two datasets: ModelNet40 (inertia tensors) and Materials Project (dielectric tensors). The results demonstrate that the proposed method achieves competitive point prediction accuracy while providing physically consistent uncertainty estimates. The visualization of uncertainty ellipsoids on ModelNet40 effectively illustrates the anisotropic nature of the predicted uncertainty. The comparison against deterministic baselines and diagonal UQ models highlights the necessity of full-covariance modeling. However, the paper lacks comparison against other probabilistic geometric deep learning methods (if any exist for this specific task) or Bayesian ensembles, which would strengthen the claim of superiority. The evaluation of "OOD sensitivity" is mentioned but the specific metrics and results are not fully detailed in the provided text snippet, relying on appendices. The performance on Materials Project (MAE 1.55) is reported, but without context on the baseline MAEs, it is difficult to assess the magnitude of improvement in uncertainty calibration.
The paper provides sufficient detail regarding the architecture (using e3nn), the decomposition of irreps, and the loss function. The use of standard libraries (e3nn) and well-defined mathematical operations (matrix exponential, Clebsch-Gordan coefficients) enhances reproducibility. The authors mention an appendix with detailed derivations and hyperparameter sweeps, which is good practice. The code is not explicitly linked in the text provided, but the methodological clarity is high.
The method is currently restricted to symmetric rank-2 tensors. While the authors mention potential extension to rank-4 tensors (elasticity), the computational complexity of the Clebsch-Gordan coefficients and the size of the resulting Lie algebra operators may pose significant challenges. The reliance on the Log-Euclidean metric, while stable, is an approximation of the affine-invariant metric on SPD manifolds; the impact of this choice on the quality of uncertainty estimates is not deeply analyzed. Furthermore, the method assumes that the uncertainty can be fully captured by a Gaussian-like distribution in the Log-Euclidean space, which may not hold for all complex physical systems with multi-modal uncertainties.
This work has significant implications for scientific machine learning, particularly in materials science and computational physics, where reliable uncertainty estimates are crucial for experimental design and risk assessment. By providing a framework for equivariant UQ, it enables safer deployment of geometric neural networks in high-stakes applications. The rigorous treatment of SPD constraints and equivariance sets a new standard for probabilistic geometric deep learning. This paper presents a rigorous and mathematically elegant framework for equivariant uncertainty quantification in tensor-valued geometric learning, addressing a critical gap in the field by combining representation theory with Riemannian geometry to ensure physically consistent, positive-definite, and rotationally equivariant covariance estimates.
Clinicians read chain-of-thought (CoT) rationales as evidence of medical reasoning, but whether the visible chain plays that role is rarely tested. General-domain CoT-faithfulness probes ignore clinical cost, and medical LLM evaluations treat the chain as a black box. We close this gap with a medical perturbation audit: a 30-operator battery edits both the chain and the question with clinically motivated operators (severity reversal, negation flip, demographic swap, evidence ablation), paired with a chain-update times answer-flip joint analysis that classifies each model by its failure mode. Applied to 14 LLMs on four medical QA benchmarks, three independent tests converge: the Chain-Decoupling Rate (CDR; chain does not register the edit and the answer does not flip) is 72.9% panel-wide on clinically meaningful destructive edits, chain corruption leaves accuracy unchanged, and removing CoT prompting does not reduce accuracy. Two board-certified clinicians re-annotate N=197 perturbed questions; 98.5% leave the gold defensible. The pattern holds across medical and reasoning fine-tuning and scale; on the closed-source tier, where the chain text is unavailable, the answer-side signals are consistent with the same decoupling. Our framework and CDR provide a reusable yardstick for auditing whether medical CoT is faithful or merely documentation.
Primary: Eindhoven University of Technology
All Institutions: Eindhoven University of Technology, Dana-Farber Cancer Institute
This paper presents a comprehensive and empirically robust audit of medical Chain-of-Thought reasoning, revealing a high rate of decoupling between generated rationales and model decisions, thereby providing a critical safety assessment for the deployment of LLMs in clinical settings.
The paper introduces a rigorous perturbation audit framework specifically designed for medical Chain-of-Thought (CoT) reasoning. The core methodological contribution is the "Chain-Decoupling Rate" (CDR), which jointly analyzes whether the generated reasoning chain registers clinically meaningful edits to the question and whether the final answer flips accordingly. The authors define a 30-operator battery of perturbations, split into chain-level (structural/surface edits to the CoT) and question-level (clinical content edits like severity reversal, negation, demographic swap). This dual-path design allows for a fine-grained taxonomy of failure modes (e.g., faithful/robust vs. decoupled/narrative). The approach is methodologically sound, moving beyond simple accuracy metrics to probe the causal link between reasoning text and decision output.
The evaluation is extensive, covering 14 LLMs (open-weight, reasoning-tuned, and closed-source) across four major medical benchmarks (MedQA, MedMCQA, PubMedQA, MMLU-med). The results are striking and consistent: a panel-wide CDR of 72.9%, indicating that in nearly three-quarters of cases with clinically meaningful destructive edits, the model's chain does not register the change and the answer does not flip. The paper provides robust ablation studies, including clinician validation of the perturbations (98.5% of edits left the gold answer defensible, confirming the edits were clinically valid but non-disruptive to the ground truth), sensitivity analyses, and checks for contamination (option-shuffle tests). The convergence of evidence across open and closed-source models strengthens the validity of the findings.
The paper provides detailed descriptions of the perturbation operators, including regex patterns and seed-driven variants, which aids reproducibility. The authors mention that the reference Python implementation is available from the corresponding author, though a public GitHub link is not explicitly in the text provided. The experimental setup, including model versions, decoding parameters, and prompt templates, is clearly documented. The use of standard benchmarks ensures that other researchers can replicate the baseline comparisons.
The authors acknowledge several limitations. The study is restricted to multiple-choice questions, which may not fully capture the complexity of free-text clinical reasoning. The "chain-update" metric relies on token matching or an LLM-judge, which might miss semantic paraphrasing of the evidence. The closed-source models were evaluated only on answer-side signals due to API restrictions, limiting the direct measurement of CDR for these models. Additionally, the demographic scope is limited to binary gender and age, and the datasets are English-centric.
This paper has significant implications for the deployment of LLMs in healthcare. By demonstrating that CoT rationales are often "decorative" rather than causally load-bearing, it challenges the assumption that clinicians can rely on these rationales for auditability or trust. It highlights a critical safety gap: models can be accurate while being unfaithful, potentially hiding incorrect reasoning. The proposed framework provides a reusable yardstick for auditing faithfulness, which is crucial for developing safer and more reliable medical AI systems. This paper presents a comprehensive and empirically robust audit of medical Chain-of-Thought reasoning, revealing a high rate of decoupling between generated rationales and model decisions, thereby providing a critical safety assessment for the deployment of LLMs in clinical settings.
We prove a depth hierarchy for ReLU neural networks in which every additional ReLU layer can save exponentially many neurons. For every $\ell\geq 3$, a globally $[0,1]$-valued, $1$-Lipschitz function is realized by a depth-$\ell$ network of width $\mathcal{O}(d^4)$, whereas every depth-$(\ell-1)$ network with unrestricted weights and width at most $2^d/[2d(\ell-2)]$ has squared $L_2$ error at least $1/24$ under an absolutely continuous distribution. To the best of our knowledge, this is the first exponential separation for ReLU networks between two fixed depths whose shallower depth is at least $3$, and the first exponential hierarchy across all adjacent fixed depths. The lower bound also immediately yields the corresponding hierarchy for exact computation. Moreover, the case $\ell=3$ gives a compactly supported depth-$3$-versus-depth-$2$ separation with unrestricted shallow-network weights, answering a question raised by Safran, Eldan, and Shamir (2019, Sec. 2.3). The corresponding distribution nevertheless has all its mass at exponential radius, so the construction falls outside the regularity regime in which such a separation would imply major threshold-circuit lower bounds. We also prove an exact separation for a more benign function. It is computed by a polynomial-width depth-$4$ network, whereas every depth-$3$ network agreeing with it on the unit hypercube requires exponentially many neurons in its first hidden layer, again without any restriction on the weights. The function is globally $[0,1]$-valued and $\mathcal{O}(\sqrt d)$-Lipschitz, and maps the unit hypercube onto $[0,1]$.
Primary: Weizmann Institute of Science
All Institutions: Weizmann Institute of Science
[One sentence main contribution]. This paper proves the first exponential depth hierarchy for ReLU networks between any two adjacent fixed depths $k \ge 3$, demonstrating that every additional layer can save exponentially many neurons. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The work is a landmark result in the theoretical foundations of deep learning. It resolves a longstanding open problem by establishing that the exponential advantage of depth is not limited to the specific case of depth 3 vs 2, but holds for all adjacent depths. The methodology, involving the recursive copying of hard functions via geometric perturbations, is elegant and powerful. While the exponential radius requirement limits its immediate implication for circuit complexity, it provides a complete characterization of the expressive power of ReLU networks with respect to depth in the $L_2$ approximation setting. The exact separation result for depth 4 vs 3 on the unit hypercube is also significant, offering a more "benign" setting for future investigations. This paper will serve as a definitive reference for researchers studying the expressivity and complexity of neural networks.
The paper presents a rigorous theoretical contribution to the understanding of ReLU neural network expressivity. The core methodology involves constructing specific families of continuous piecewise-linear (CPWL) functions that exhibit exponential depth hierarchies. The authors employ a recursive construction technique: they define a "copier" mechanism that creates exponentially many translated and rescaled copies of a hard base function. By perturbing the Boolean hypercube vertices into quantitative affine general position, they ensure that any shallow network (depth $k-1$) must use an exponential number of neurons to capture the geometry of these copies. The proof relies on geometric arguments regarding the intersection of affine hyperplanes with these perturbed structures. The approach is mathematically sophisticated, leveraging tools from approximation theory, geometry of numbers, and circuit complexity lower bounds.
This is a theoretical paper with no empirical experiments, datasets, or code. The "evaluation" consists of formal mathematical proofs of lower bounds. The validity of the paper rests entirely on the correctness of the theorems and their proofs, which appear rigorous and well-structured.
As a theoretical work, reproducibility refers to the verifiability of the proofs. The paper provides detailed constructions (e.g., the perturbation map, the copier function) and lemmas that support the main theorems. The appendices contain full proofs. While not "reproducible" in the empirical sense, the logical flow is transparent and self-contained.
The primary limitation, acknowledged by the authors, is the normalization scale. The $L_2$ separation result requires the target distribution to have support at an exponential radius from the origin. This places the result outside the "regularity regime" (polynomial radius, bounded Lipschitz constant) where the Vardi-Shamir reduction to threshold circuit lower bounds applies. Consequently, while it resolves the fixed-depth gap for ReLU networks, it does not immediately resolve the corresponding open problems in Boolean circuit complexity. The exact computation separation is more benign (polynomial radius) but only applies to depths 4 vs 3, not the full adjacent-depth hierarchy.
This paper addresses a fundamental question in deep learning theory: the necessity of depth. By proving that every additional layer can save exponentially many neurons for ReLU networks (at least for depths $\ge 3$), it provides strong theoretical justification for the use of deep architectures in practice. It bridges the gap between neural network approximation theory and classical circuit complexity, highlighting both the similarities and the specific barriers (like the natural proofs barrier) that prevent direct translation of results. It clarifies the landscape of what is known about depth hierarchies and identifies the precise conditions under which exponential separations can and cannot be proven. [One sentence main contribution]. This paper proves the first exponential depth hierarchy for ReLU networks between any two adjacent fixed depths $k \ge 3$, demonstrating that every additional layer can save exponentially many neurons. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The work is a landmark result in the theoretical foundations of deep learning. It resolves a longstanding open problem by establishing that the exponential advantage of depth is not limited to the specific case of depth 3 vs 2, but holds for all adjacent depths. The methodology, involving the recursive copying of hard functions via geometric perturbations, is elegant and powerful. While the exponential radius requirement limits its immediate implication for circuit complexity, it provides a complete characterization of the expressive power of ReLU networks with respect to depth in the $L_2$ approximation setting. The exact separation result for depth 4 vs 3 on the unit hypercube is also significant, offering a more "benign" setting for future investigations. This paper will serve as a definitive reference for researchers studying the expressivity and complexity of neural networks.
Membership inference attacks expose whether individual records were used to train a model, yet existing attacks on diffusion models are largely heuristic and can require substantial query budgets. We introduce DIME (Denoiser Ideal Membership Error), a theoretically grounded and query-efficient framework for membership inference on diffusion models. Our starting point is an exact characterization of the optimal diffusion denoiser for a finite training set, which reveals that membership leakage is governed by the denoiser's implicit reconstruction error. This error decomposes into two complementary signals: a bias term, capturing reconstruction accuracy, and a previously unexplored local crowding term, capturing the geometry of nearby training examples. Both admit efficient estimators using only model queries, yielding a practical attack with as few as two queries. Across CIFAR-10/100, STL10-U, CelebA, and ImageNet, DIME consistently outperforms prior attacks at comparable or substantially lower query cost, improving TPR at 1% FPR by up to $3\times$; remarkably, its two-query variant can outperform existing 30-query baselines. Finally, we suggest, discuss, and evaluate specific defenses to counteract such powerful membership tests.
Primary: University of Illinois at Urbana-Champaign
All Institutions: University of Illinois at Urbana-Champaign
This paper presents a theoretically grounded and highly effective query-efficient membership inference attack on diffusion models, significantly advancing the understanding of privacy vulnerabilities in generative AI and providing a new benchmark for evaluating model privacy.
The paper proposes DIME, a theoretically grounded framework for Membership Inference Attacks (MIA) on diffusion models. The core theoretical contribution is an exact characterization of the optimal diffusion denoiser for a finite training set, decomposing membership leakage into a bias term (reconstruction accuracy) and a local crowding term (geometry of nearby training examples). This is a significant theoretical insight that moves beyond heuristic approaches. The methodology leverages these signals to create a query-efficient attack, requiring as few as two queries to the model. The approach is novel in its theoretical grounding of diffusion model privacy leakage, distinguishing it from previous attacks that relied on output scores or gradients without such a rigorous decomposition. EXPERIMENTAL_EVALUTION: The authors evaluate DIME across multiple standard benchmarks: CIFAR-10/100, STL10-U, CelebA, and ImageNet. The results demonstrate consistent outperformance of prior attacks at comparable or substantially lower query costs. Notably, the two-query variant of DIME outperforms existing 30-query baselines, with improvements in True Positive Rate (TPR) at 1% False Positive Rate (FPR) of up to 3x. The evaluation is comprehensive, covering different data modalities (images) and scales. The inclusion of defense evaluations further strengthens the empirical contribution by showing the practical necessity of the attack and the effectiveness of proposed countermeasures.
The paper includes an anonymous repository link for code and instructions, indicating a commitment to open science. The theoretical derivations are provided in the appendix, and the experimental setup is described in sufficient detail to allow replication. The use of standard datasets and metrics facilitates independent verification.
The theoretical characterization relies on assumptions about the optimal denoiser which may not perfectly hold for all practical, trained diffusion models (e.g., those with specific architectural constraints or training dynamics). The attack's effectiveness might vary depending on the specific diffusion model architecture (e.g., DDPM vs. Stable Diffusion) and training hyperparameters, although the broad evaluation mitigates this concern. The paper focuses on image data; generalization to other modalities (text, audio) is not explicitly addressed.
This work has significant implications for the privacy of diffusion models, which are widely used in generative AI. By demonstrating that membership inference is possible with very few queries, it highlights a critical vulnerability in current deployment practices. The provision of effective defenses is crucial for the responsible development of these models. The work contributes to the broader field of AI security and privacy, encouraging the development of more robust privacy-preserving techniques. This paper presents a theoretically grounded and highly effective query-efficient membership inference attack on diffusion models, significantly advancing the understanding of privacy vulnerabilities in generative AI and providing a new benchmark for evaluating model privacy.
We study autonomous mathematical discovery in the Station, an open-world multi-agent environment in which AI agents from different model families pursue a shared research goal without a central coordinator or scripted pipeline. Agents choose their own research directions, conduct experiments, collaborate, and build a shared scientific literature. Across 12 construction problems from the AlphaEvolve catalogue and two additional case studies, the Station obtained results novel relative to the prior literature on five problems: a new infinite family of finite-field Kakeya sets, new exact 604-point kissing configurations in dimension 11, new records for the discretized Kakeya needle and sign uncertainty problems, and a substantially improved lower bound for Erdős's minimum-overlap problem. Agents also discovered novel infinite families for Book Ramsey numbers. Importantly, the agents produced not only numerical constructions but also theorems and analyses explaining how those constructions work, making the results more interpretable and easier for mathematicians to build upon. We release all raw agent dialogues, proofs, and verification code, providing a transparent record of how these discoveries emerged.
Primary: DualverseAI
All Institutions: DualverseAI, University of Cambridge, University of Hong Kong, University of California San Diego
The paper presents a significant advancement in autonomous scientific discovery by introducing The Station, an open-world multi-agent environment that enables heterogeneous AI agents to collaboratively conduct mathematical research, resulting in novel, interpretable proofs and constructions that outperform previous state-of-the-art systems like AlphaEvolve on several open problems.
The paper introduces "The Station," an open-world multi-agent environment designed for autonomous mathematical discovery. Unlike previous systems that rely on rigid, scripted pipelines or centralized coordinators (e.g., AlphaEvolve), The Station simulates a scientific ecosystem where heterogeneous AI agents (from different model families like GPT-4.5, Claude Opus, Gemini Pro) operate with high autonomy. Agents choose their own research directions, conduct experiments, communicate, and publish papers to a shared archive. This architecture allows for the accumulation of scientific knowledge across generations of agents. The methodology shifts the paradigm from AI as a tool within a pipeline to AI as an independent researcher within a community. The approach leverages the emergent behaviors of multi-agent systems to explore the solution space more broadly and interpretably than single-agent or tightly coupled multi-agent systems.
The authors evaluate The Station on 12 construction problems from the AlphaEvolve catalogue and two additional case studies (Book Ramsey numbers and the Jacobian Conjecture). The results are significant: The Station produced novel results relative to prior literature on five problems, including a new infinite family of finite-field Kakeya sets, exact 604-point kissing configurations in dimension 11, and improved bounds for the discretized Kakeya needle and sign uncertainty problems. Notably, it outperformed AlphaEvolve on several tasks by finding more interpretable, algebraic constructions rather than just numerical optimizations. It also successfully reconstructed a counterexample to the Jacobian Conjecture. The evaluation demonstrates that the system can handle complex, open-ended mathematical tasks without human intervention in the discovery loop, only verifying the outputs.
The paper provides a high degree of reproducibility. The authors release all raw agent dialogues, proofs, verification code, and the complete source code for The Station. This transparency allows other researchers to audit the discovery process, understand how conclusions were reached, and potentially replicate or extend the findings. The release of raw data is particularly valuable for studying the dynamics of multi-agent scientific collaboration.
The paper acknowledges several limitations. First, the agents lack "expert intuition," often deprioritizing promising directions or getting stuck in "attractor traps" (e.g., optimizing trivial details). Second, there is a lack of diverse research tastes among agents from the same model family, potentially narrowing the exploration scope. Third, the system relies on in-context learning, which does not update model weights, limiting the long-term retention of complex knowledge across very long runs. Finally, the current setup is computationally expensive and requires significant human effort to set up the initial environment and verify the final mathematical proofs, although the discovery process itself is autonomous.
This work has profound implications for the future of AI-assisted scientific discovery. By demonstrating that autonomous agents can produce novel, interpretable mathematical results, it suggests a path toward fully automated research assistants. The emphasis on interpretable outputs (theorems and proofs) addresses a major bottleneck in AI science: the "black box" nature of many AI discoveries. This approach could be extended to other fields like computational biology and materials science, potentially accelerating the pace of scientific discovery. It also raises important questions about the role of human researchers in an era of autonomous AI scientists. The paper presents a significant advancement in autonomous scientific discovery by introducing The Station, an open-world multi-agent environment that enables heterogeneous AI agents to collaboratively conduct mathematical research, resulting in novel, interpretable proofs and constructions that outperform previous state-of-the-art systems like AlphaEvolve on several open problems.
Two analysts who calibrate the same predictive model on independent samples will deploy different prediction sets every time, because the calibration threshold inherits the randomness of the data. Wherever deployments must be audited, cached, or approved across sites, this instability is costly: no one can verify that two calibrations produced the same object. We ask two questions: when can independent calibrations yield the identical classifier, and what must that agreement cost? Perfect agreement is impossible, since a procedure that almost always returns one fixed answer cannot remain valid for every distribution, and exact agreement through shared randomness forces the procedure to ignore its data. Sharing a single random seed and rounding the calibrated threshold up to a coarse shared grid resolves the tension: the deployed classifier becomes identical across analysts with any desired probability, coverage guarantees survive, and the price is a quantified increase in set size and calibration data. Matching lower bounds show that no threshold method can pay less, and the method's one tuning constant vanishes asymptotically. Without any shared seed, a fixed grid still confines all analysts to two adjacent classifiers, and no method does better. Replicability also blocks gaming: selecting the most favorable of many recalibrations barely moves a replicable classifier, while the same selection silently undercovers standard conformal prediction. Experiments on real ImageNet outputs, a four-hospital site split, and four language-model families match the theory, including the measured sample-cost frontier.
Primary: Yale University
All Institutions: Yale University, The Wharton School, University Hospital Zurich, University of Pennsylvania, Hungarian Institute of Cardiology
[One sentence main contribution]. This paper establishes the theoretical and practical foundations for replicable conformal prediction, enabling identical deployment artifacts across independent calibration runs through shared randomness and grid rounding. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper makes a substantial contribution to the field of distribution-free uncertainty quantification by solving a critical deployment problem: the instability of conformal prediction sets across different calibration datasets. By introducing a shared-seed grid rounding mechanism, the authors provide a rigorous method to achieve replicability, backed by tight lower bounds and extensive empirical validation across diverse domains including computer vision and natural language processing. The work is theoretically deep, addressing impossibility results and sample complexity, and practically relevant, offering a solution to auditability and selection bias issues in conformal prediction.
The paper introduces "Replicable Conformal Prediction," a novel framework that addresses the instability of standard conformal prediction across independent calibration runs. The core methodological contribution is the use of a shared random seed to define a coarse grid for rounding the calibrated threshold. This ensures that two analysts deploying the model will produce identical prediction sets (or a certified small list of sets) with high probability. The authors provide rigorous theoretical analysis, including impossibility results for exact replication without shared randomness, lower bounds on sample complexity and set-size inflation, and explicit finite-sample coverage guarantees. The approach is mathematically sound, leveraging the Beta distribution of conformal scores to derive precise bounds.
The experimental evaluation is comprehensive and aligns well with the theoretical claims. The authors test the method on ImageNet classification scores, a multi-site clinical dataset (four-hospital split), and four large language model families (GPT-2, Llama, Qwen, etc.). The experiments demonstrate that standard conformal prediction fails to replicate (identity near zero), while the proposed method achieves high identity rates as predicted. The paper also quantifies the "selection attack" vulnerability of standard conformal prediction and shows how replicability mitigates it. The empirical results match the theoretical sample-complexity frontiers, providing strong evidence for the method's validity and cost.
The paper emphasizes reproducibility as a primary motivation and practice. It reports that all experiments were rerun end-to-end and that synthetic results regenerate deterministically. The use of fixed master seeds and detailed algorithmic descriptions enhances reproducibility. The inclusion of lower bounds and explicit constants allows other researchers to verify the theoretical limits.
The method requires a shared random seed, which may not be feasible in all decentralized or privacy-sensitive deployment scenarios (though the paper addresses the seedless case with a two-element list). The method assumes a local margin condition (bounded density) for optimal bounds, which may not hold for all score distributions. The asymptotic removal of the quadratic factor in sample complexity is noted as an open problem for finite samples. Additionally, the paper notes that real-image runs used cross-validated probabilities rather than frozen raw logits, which is a limitation for direct production deployment claims.
This work has significant implications for the deployment of uncertainty quantification in high-stakes domains like healthcare and finance, where auditability and consistency are critical. By making conformal prediction replicable, it reduces the risk of gaming through selective recalibration and facilitates regulatory approval and cross-site validation. It bridges the gap between statistical validity and practical deployment stability. [One sentence main contribution]. This paper establishes the theoretical and practical foundations for replicable conformal prediction, enabling identical deployment artifacts across independent calibration runs through shared randomness and grid rounding. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper makes a substantial contribution to the field of distribution-free uncertainty quantification by solving a critical deployment problem: the instability of conformal prediction sets across different calibration datasets. By introducing a shared-seed grid rounding mechanism, the authors provide a rigorous method to achieve replicability, backed by tight lower bounds and extensive empirical validation across diverse domains including computer vision and natural language processing. The work is theoretically deep, addressing impossibility results and sample complexity, and practically relevant, offering a solution to auditability and selection bias issues in conformal prediction.
Terminal agents are a compelling application of large language models (LLMs), with the potential to integrate deeply into users' daily workflows. Reinforcement learning (RL) is a key technique for improving their capabilities, making scalable training environments a central challenge. Since public real-user interaction data are scarce, synthetic environments provide a practical alternative, but often suffer from domain gaps and limited fidelity, leading to poor generalization. Existing work mainly scales the quantity and diversity of synthetic environments, while reward-signal quality and the mechanisms governing generalization remain under-explored. We study how RL improves terminal agents and propose the Agentic Compositional Generalization hypothesis: rather than teaching new domain-specific skills from scratch, RL primarily shapes high-level decision-making behaviors that compose and route low-level skills acquired during pre-training and supervised fine-tuning (SFT). This account is consistent with our empirical results and suggests that verifier quality, which determines which behaviors are reinforced, is more important than simply increasing environment quantity or diversity. Motivated by this insight, we propose River, a simple training recipe that improves reward quality by filtering low-quality environments and augmenting outcome rewards with process-level behavior regularization. Using this recipe, our RL-trained agent achieves the best performance among evaluated open-source RL-trained 8B models across four terminal-agent benchmarks. River also generalizes across model families, scales, agent harnesses, and RL objectives. Using fewer than 30% of the TMax training environments, River improves RL gains by 106% and 30% on average for models ranging from 2B to 27B on Terminal-Bench-Lite and Terminal-Bench-v2.1, respectively.
Primary: Carnegie Mellon University
All Institutions: Carnegie Mellon University, Salesforce AI Research
This paper presents a compelling argument and empirical evidence that reward signal quality, rather than environment quantity, is the primary driver of generalization in RL for terminal agents, proposing a simple yet effective training recipe that achieves state-of-the-art results with significantly reduced resource requirements.
The paper proposes a significant shift in perspective for training terminal agents, moving away from the prevailing trend of scaling synthetic environment quantity towards improving reward signal quality. The core theoretical contribution is the "Agentic Compositional Generalization" hypothesis, which posits that RL primarily refines high-level routing and decision-making rather than teaching low-level skills from scratch. This is supported by empirical evidence and leads to the proposed "River" training recipe. River combines low-quality environment filtering with process-level behavior regularization. While the individual components (filtering, process rewards) are not entirely new in isolation, their specific application and optimization for the terminal agent domain, coupled with the strong theoretical framing, constitute a novel and coherent methodological contribution. The approach is simple yet effective, challenging the "more data/environments is better" paradigm.
The experimental evaluation is robust and convincing. The authors demonstrate that their method outperforms existing open-source RL-trained 8B models across four terminal-agent benchmarks. Crucially, they show strong generalization across model families (2B to 27B), agent harnesses, and RL objectives. The efficiency gains are significant: using less than 30% of the training environments, they achieve substantial performance improvements (106% and 30% average gains on specific benchmarks). The results are consistent with their hypothesis and provide strong empirical support for the importance of reward quality over environment quantity/diversity. The inclusion of analysis on model scaling and cross-harness generalization adds depth to the evaluation.
The paper provides a clear description of the "River" recipe, including the filtering criteria and the nature of the process-level regularization. The mention of "TMax" as a resource suggests the use of existing, potentially reproducible, synthetic environments. However, the full text provided is a skeleton, and specific hyperparameters, exact filtering thresholds, and code implementations are likely in the appendix or repository (not fully visible here). Assuming standard practices for such venues, reproducibility is likely high given the simplicity of the proposed recipe, but full verification requires access to the codebase.
The paper focuses heavily on terminal agents, which, while important, is a specific sub-domain. The generalizability of the "compositional generalization" hypothesis to other agentic domains (e.g., web navigation, code generation in complex IDEs) is not fully explored. The reliance on synthetic environments means that domain gaps, while mitigated by quality filtering, may still persist compared to real-user data. The paper does not extensively discuss the computational cost of the process-level regularization compared to pure outcome rewards.
This work has significant implications for the development of autonomous AI agents. By demonstrating that reward quality is more critical than environment scale, it offers a more resource-efficient path to improving agent capabilities. This could lower the barrier to entry for training capable agents and reduce the environmental footprint of RL training. The insights into how RL shapes agent behavior contribute to the broader field of interpretability and mechanistic understanding of LLM-based agents. This paper presents a compelling argument and empirical evidence that reward signal quality, rather than environment quantity, is the primary driver of generalization in RL for terminal agents, proposing a simple yet effective training recipe that achieves state-of-the-art results with significantly reduced resource requirements.
Reinforcement learning with verifiable rewards (RLVR) is deployed to make models better at reasoning tasks, but its side effect on what models will divulge is under studied. Here we show that RLVR on facts increases extraction of personally identifiable information (PII) the instruct model had already memorized. We first confirm that instruct models have already memorized PII but leave them latent, rarely surfacing one when asked. We then apply RL on benign factual data that contains no PII of any kind, and re-probe: a targeted probe over name->email pairs, and an untargeted free-recall prompt that simply asks the model to list the addresses it knows. PII extraction rises sharply under both: on DeepSeek-V3.1, verbatim recall@k increases from 0.155 to 0.370, a 2.4x gain. The effect scales with model size: across three models spanning 8B to 671B parameters, absolute leakage is largest in the biggest model. Meanwhile model's reasoning abilities and refusal rates are retained, indicating that RL selectively changes which memorized information is accessible rather than broadly altering the model. In summary, memorized private data can be made markedly more extractable by training that never touches it. This gives an adversary a route to memorized data that requires no privacy-relevant training signal and no access to the data itself -- only the ability to fine-tune on something innocuous.
Primary: Carnegie Mellon University
All Institutions: Carnegie Mellon University
This paper demonstrates that Reinforcement Learning with Verifiable Rewards on benign data significantly amplifies the extractability of memorized personally identifiable information (PII) without degrading reasoning capabilities, revealing a critical, previously under-studied privacy risk in modern LLM post-training pipelines.
The paper employs a rigorous experimental design to isolate the effect of Reinforcement Learning with Verifiable Rewards (RLVR) on privacy leakage. By using a "benign" factual dataset (FACTS-Parametric) for training and probing on a completely separate, held-out PII dataset (Enron), the authors effectively rule out the possibility that the model is simply overfitting to private data during the RL phase. The use of both targeted probes (name->email) and untargeted free-recall prompts, along with synthetic decoy pools to control for hallucination/fabrication, constitutes a robust methodological framework for measuring memorization and access. The distinction between "access" and "capability" is theoretically sound and well-executed.
The experiments are comprehensive, spanning three model sizes (8B to 671B) to demonstrate scaling laws. The results are clear and significant: verbatim recall of PII increases by 2.4x on DeepSeek-V3.1 without any degradation in reasoning capabilities (MMLU) or significant changes in refusal rates. The inclusion of decoy controls proves that the increase in PII extraction is due to genuine retrieval of memorized data, not increased hallucination. The scaling analysis (larger models leaking more) adds critical weight to the findings, suggesting this is a fundamental property of scale and RL dynamics rather than a small-model artifact.
The paper provides detailed hyperparameters, dataset sources (Enron, FACTS-Parametric), and model names. The use of public datasets and standard RLVR implementations (GRPO) enhances reproducibility. The authors explicitly state they do not release the specific RL checkpoints or the target pool to prevent misuse, which is a responsible limitation for reproducibility but does not hinder the verification of the methodology.
The study is limited to a single type of PII (email addresses) from a single corpus (Enron). While the Enron corpus is a standard benchmark for this type of analysis, it represents a specific demographic (US corporate employees) and era. The authors acknowledge that generalization to other PII types (phone numbers, SSNs) or other languages is untested. Additionally, the study focuses on instruction-tuned models; the behavior of base models under RLVR is less relevant to current deployment paradigms but worth noting.
This paper has profound implications for the safety and deployment of LLMs. It challenges the assumption that safety alignment (refusal rates) is a reliable indicator of privacy protection. It reveals a "silent" risk vector where standard post-training techniques intended to improve reasoning inadvertently unlock private data. This necessitates new evaluation protocols for RLVR that include privacy leakage metrics, not just capability and safety benchmarks. It may also influence regulatory perspectives on model auditing and the liability of deploying models that have undergone RLVR. This paper demonstrates that Reinforcement Learning with Verifiable Rewards on benign data significantly amplifies the extractability of memorized personally identifiable information (PII) without degrading reasoning capabilities, revealing a critical, previously under-studied privacy risk in modern LLM post-training pipelines.
We propose Variance Driven Exploration (VarDE), a principled approach for pure exploration in highly stochastic environments, where the exploration process is dominated by stochastic variance. VarDE is built on a fundamental principle: sampling effort should be allocated to minimize the uncertainty of the final decision. We formalize the uncertainty of the final decision through a smooth decision function and derive allocation rules that explicitly capture how stochastic noise in individual components affects the reliability of the final output. We apply this methodology to three core problems of pure exploration -- Best Arm Identification (BAI), Monte Carlo Tree Search (MCTS), and Best-Policy Identification (BPI) -- with theoretical guarantees on variance decay and simple regret. Empirically, we demonstrate consistent and significant improvements of VarDE over existing methods, with especially strong gains in highly stochastic environments.
Primary: Hanoi University of Science and Technology
All Institutions: Hanoi University of Science and Technology, FPT Quantum AI & Cyber Security Institute
This paper presents a unified framework for pure exploration in reinforcement learning and bandits by minimizing decision-level variance through influence-weighted sampling. The theoretical derivation is sound, and the empirical results demonstrate consistent improvements over state-of-the-art methods in stochastic environments, offering a valuable perspective on allocation strategies that balances local noise with global decision impact.
The paper proposes Variance Driven Exploration (VarDE), a methodology that treats pure exploration as a decision-level uncertainty minimization problem. By approximating the non-differentiable decision function (e.g., max operation in BAI or MCTS) with a smooth surrogate (LogSumExp), the authors derive influence weights via first-order Taylor expansion. The core innovation is the allocation rule: sample the component (arm, edge, or state-action pair) that maximizes the expected reduction in the variance of the final decision, balancing influence weight and local empirical variance. This is applied to Best-Arm Identification (BAI), Monte Carlo Tree Search (MCTS), and Best-Policy Identification (BPI). The approach is theoretically grounded in variance decomposition and provides exponential concentration guarantees for BAI and MCTS, and asymptotic convergence for BPI. The methodology is elegant and provides a unified lens for disparate pure exploration problems.
The empirical evaluation covers standard benchmarks for BAI (synthetic arms), MCTS (grid worlds, synthetic trees), and BPI (RiverSwim, ForkedRiverSwim). VarDE consistently outperforms strong baselines (UCB-E, UGapE, UCT, MENTS, Q-UCB, etc.), particularly in highly stochastic environments where variance is high. The results support the claim that variance-aware allocation is superior to optimism-based or entropy-based methods in these specific regimes. However, the benchmarks are relatively standard and do not include large-scale or complex real-world domains (e.g., Atari, StarCraft, or large MDPs) which would better demonstrate scalability and robustness. The improvements are significant but not revolutionary in terms of magnitude on simple tasks.
The paper provides a clear algorithmic description and references an open-source code repository. The theoretical derivations are detailed, including lemmas and theorems. The experimental setup is described with hyperparameter tuning protocols. Reproducibility appears high.
The authors acknowledge several limitations: the reliance on first-order approximation which may be inaccurate early in learning; the bias-variance trade-off introduced by the temperature parameter in the LogSumExp surrogate; and the lack of non-asymptotic finite-sample bounds for the RL (BPI) component, which only offers asymptotic convergence. Additionally, the method assumes independence of components for the variance decomposition, which may not hold in complex planning or RL settings with correlated uncertainties.
The work contributes to the fundamental understanding of exploration in RL and bandits. By providing a principled way to handle high-variance environments, it can improve the efficiency of learning in safety-critical or resource-constrained applications where sample efficiency is paramount. It does not present immediate societal risks but advances the technical toolkit for AI researchers. This paper presents a unified framework for pure exploration in reinforcement learning and bandits by minimizing decision-level variance through influence-weighted sampling. The theoretical derivation is sound, and the empirical results demonstrate consistent improvements over state-of-the-art methods in stochastic environments, offering a valuable perspective on allocation strategies that balances local noise with global decision impact.
Discrete diffusion models have demonstrated strong performance across a range of datasets, including natural language data and graph-structured data. Among many variants, score-entropy discrete diffusion (SEDD) has achieved particularly strong empirical results. In SEDD, new samples are generated by iteratively evaluating a sequence of concrete score functions, which are learned by minimizing a score-entropy loss. While much of the prior theoretical literature on discrete diffusion has focused on the sampling efficiency of SEDD under the assumption of small score estimation error, recent work has begun to investigate the finite-sample properties of score estimation itself. In this work, we take a different route by investigating the fundamental statistical limits of concrete score estimation. We focus on uniform and masking discrete diffusions, two of the most widely adopted discrete diffusion models. We establish a minimax lower bound under the score-entropy loss, and propose an MLE-based thresholding estimator that matches this lower bound up to constant and polylogarithmic factors that depend on neighboring density ratios. We further show that, for any target distribution, this density ratio is naturally controlled under both uniform and masking discrete diffusion models, yielding nearly matching minimax lower and upper bounds for the aggregated score estimation error. Our results imply that, with appropriate initialization and discretization, SEDD can achieve nearly optimal minimax sample complexity, as measured by the KL divergence between the target and generated distributions.
Primary: The University of Texas at Austin
All Institutions: The University of Texas at Austin, Cornell University
This paper establishes the minimax optimality of score-entropy discrete diffusion by deriving tight lower bounds and proposing a matching MLE-based estimator, providing crucial theoretical grounding for the statistical efficiency of SEDD in discrete state spaces.
The paper presents a rigorous theoretical analysis of Score-Entropy Discrete Diffusion (SEDD). The core methodological contribution is the establishment of minimax lower bounds for concrete score estimation under the score-entropy loss, coupled with the proposal of an MLE-based thresholding estimator that nearly matches this bound. The authors analyze two specific transition kernels (uniform and masking) and derive bounds on the neighboring density ratios, which are critical for controlling the estimation error. The methodology is mathematically dense, relying on information-theoretic lower bounds (Fano’s method/Poissonization) and concentration inequalities. It correctly identifies that previous work focused on sampling efficiency assuming good scores, whereas this work addresses the fundamental statistical limits of score estimation itself. The approach is sound and fills a significant gap in the theoretical understanding of discrete diffusion models.
The experimental section is minimal and serves primarily to validate the theoretical scaling laws rather than to demonstrate state-of-the-art performance. The authors use a synthetic uniform distribution over binary strings of varying dimensions. They verify that the KL divergence scales linearly with the aggregated score error and that the score estimation error scales inversely with the sample size ($1/n$). While these results are consistent with the theory, the lack of evaluation on real-world discrete data (e.g., natural language or graphs) limits the empirical impact. The experiments are illustrative of the theory but do not provide strong evidence of practical superiority over existing methods in complex settings.
The paper provides detailed algorithms (Algorithm 1) and theoretical proofs. The MLE thresholding estimator is simple to implement. However, the theoretical results are asymptotic and depend on constants that are not explicitly calculated for finite samples. The synthetic experiments are straightforward to reproduce. The lack of code release (URL: none) is a minor drawback, but the simplicity of the proposed estimator suggests that reproduction would be feasible for researchers in the field.
The primary limitation is the reliance on the assumption of bounded neighboring density ratios. While the authors show this is naturally controlled for uniform and masking diffusions, real-world data distributions may not satisfy this condition tightly, potentially leading to larger constants in the bounds. Furthermore, the analysis is non-asymptotic but does not account for the approximation error introduced by neural network parameterizations of the score function; it assumes the MLE estimator is computed exactly. The experiments are limited to synthetic data, leaving the applicability to high-dimensional real-world discrete data (like text) theoretically motivated but empirically unverified in this work.
This paper provides foundational theoretical justification for the use of SEDD, establishing that it achieves near-optimal sample complexity for distribution estimation in discrete spaces. This helps demystify why SEDD performs well empirically and guides future theoretical work on discrete diffusion. It clarifies the trade-offs between different diffusion kernels (uniform vs. masking) regarding density ratio control. The work is significant for the theory of generative models but has limited immediate impact on practitioners who rely on empirical performance rather than minimax optimality. This paper establishes the minimax optimality of score-entropy discrete diffusion by deriving tight lower bounds and proposing a matching MLE-based estimator, providing crucial theoretical grounding for the statistical efficiency of SEDD in discrete state spaces.
The matrix-aware optimizer Muon improves large model training by balancing updates across singular directions, yet its scaling behavior and end-to-end efficiency on large Diffusion Transformers (DiTs) remain unclear. We first establish Muon's scaling behavior on DiTs from 1.3B to 15B parameters, showing that its optimization and generative quality advantages over AdamW persist across model scales. However, at scale, the 5-step Newton--Schulz iteration (NS5) performed at every optimization step, together with full-momentum materialization, introduces substantial computation and communication overhead that can offset Muon's step-efficiency advantage. We introduce \emph{Periodic Row-wise Muon}, which performs a full NS5 spectral update once every \(K\) steps and applies a low compute and communication cost row-wise constrained update based on the current momentum at the remaining steps. We further co-design a distributed implementation that operates directly on sharded momentum during non-refresh steps and accelerates spectral refreshes through bucketed all-gather and communication--computation overlap. Across all scales, Muon improves the best observed generative quality over AdamW by 12.9--19.1\%. Compared with vanilla Muon, Periodic Row-wise Muon remains within 0.5\% in best generative quality on the 1.3B--4B models and improves it by 4.5\% at 9B. It reduces optimizer time by 46.9--54.3\%, end-to-end step time by 15.7--24.3\%, and logical communication volume by 66.7\%, while reaching its respective best generative quality with 33.7--64.8\% less active training time. These results show that Periodic Row-wise Muon preserves Muon's generative quality advantage while translating it into end-to-end training efficiency for large DiTs.
Primary: Meta
All Institutions: Meta
[One sentence main contribution]. The paper introduces Periodic Row-wise Muon, a distributed optimizer that alternates between full spectral updates and low-cost row-wise normalization to preserve the generative quality advantages of Muon while significantly reducing the training overhead of large Diffusion Transformers. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. This paper represents a solid contribution to the field of large-scale model optimization. It correctly identifies a critical bottleneck in the application of advanced matrix-aware optimizers (Muon) to modern architectures (DiTs). The proposed solution is not merely an engineering hack but is supported by geometric intuition and theoretical analysis regarding the stability of the polar factor. The systematic evaluation across multiple model scales provides strong empirical evidence for the method's efficacy. The integration of algorithmic changes with distributed systems optimizations (sharded momentum, communication-computation overlap) demonstrates a high level of technical maturity. While the novelty is incremental (alternating updates are a known pattern), the specific application and co-design for this context are valuable. The paper is well-written, rigorous, and addresses a timely problem in the scaling of diffusion models.
The paper proposes "Periodic Row-wise Muon," a hybrid optimization strategy for Diffusion Transformers (DiTs). It addresses the high computational and communication overhead of the vanilla Muon optimizer, which relies on a 5-step Newton-Schulz (NS5) iteration at every step. The core innovation is alternating between full NS5 spectral updates (every $K$ steps) and low-cost RowNorm updates (every other step). The authors provide a theoretical justification for this alternation, arguing that the ideal polar direction (targeted by NS5) is stable under small momentum perturbations, allowing RowNorm to maintain direction while NS5 periodically corrects the global spectral geometry. They also co-design a distributed implementation that operates on sharded momentum during non-refresh steps, significantly reducing communication volume. The methodology is sound, well-motivated by the specific bottlenecks of large-scale distributed training, and mathematically grounded.
The experimental evaluation is comprehensive and rigorous. The authors train DiTs ranging from 1.3B to 15B parameters on the GPIC dataset. They demonstrate that vanilla Muon improves generative quality (FD-DINO) over AdamW by 12.9-19.1%, but suffers from higher wall-clock time due to optimizer overhead. Periodic Row-wise Muon recovers this quality advantage while reducing optimizer time by ~50% and end-to-end step time by ~20%. The results are consistent across model scales, with the largest efficiency gains at the 15B scale. The ablation studies effectively isolate the contributions of the periodic schedule, the RowNorm geometry, and the distributed system optimizations. The use of multiple generation metrics (FID, MMD, HPSv2, GenEval) provides a robust assessment of quality.
The paper provides detailed algorithmic pseudocode, complexity analysis, and distributed execution details. The experimental setup is clearly described, including dataset sources, model architectures, and hyperparameters. The inclusion of theoretical proofs in the appendix adds to the rigor. However, as is common with large-scale distributed systems research, full reproducibility might require access to the specific internal distributed training infrastructure (FSDP2) and hardware topology (32-node H100 cluster). The code is not explicitly linked, but the description is sufficient for a competent engineer to implement.
The authors acknowledge several limitations: the evaluation is limited to one DiT family, one dataset (GPIC), and one hardware configuration. They do not explore layer-wise or adaptive scheduling for the period $K$ or the RowNorm multiplier $\gamma$. The refresh steps still incur full momentum communication, which remains a bottleneck, albeit less frequent. The results may vary with different hardware topologies or parallelism strategies.
This work has significant implications for the efficiency of training large-scale generative models. By making matrix-aware optimizers like Muon practically viable for large DiTs, it enables researchers and practitioners to leverage the optimization benefits of global spectral structure without prohibitive computational costs. This contributes to the broader goal of making large-scale AI training more accessible and energy-efficient. [One sentence main contribution]. The paper introduces Periodic Row-wise Muon, a distributed optimizer that alternates between full spectral updates and low-cost row-wise normalization to preserve the generative quality advantages of Muon while significantly reducing the training overhead of large Diffusion Transformers. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. This paper represents a solid contribution to the field of large-scale model optimization. It correctly identifies a critical bottleneck in the application of advanced matrix-aware optimizers (Muon) to modern architectures (DiTs). The proposed solution is not merely an engineering hack but is supported by geometric intuition and theoretical analysis regarding the stability of the polar factor. The systematic evaluation across multiple model scales provides strong empirical evidence for the method's efficacy. The integration of algorithmic changes with distributed systems optimizations (sharded momentum, communication-computation overlap) demonstrates a high level of technical maturity. While the novelty is incremental (alternating updates are a known pattern), the specific application and co-design for this context are valuable. The paper is well-written, rigorous, and addresses a timely problem in the scaling of diffusion models.
Many machine-learning systems set a threshold at a quantile of a calibration set: conformal predictors that promise 90% coverage by drawing their cutoff at the calibration set's 90th percentile, abstention gates that decline to answer when a model's score falls below the calibration set's tenth percentile, safety filters that block any output scoring above the 99th percentile of a reference set. All of them promise that the threshold will hold at the stated rate on new data. The promise assumes the calibration examples are independent, and in modern pipelines they usually are not: they share a prompt, a document, a reasoning trace. Survey statistics has known how to discount correlated data since 1965, by counting how many independent observations a sample is worth, but only for averages. We show that a threshold needs a different count. The count depends on how often clustered scores land on the same side of the threshold, and that changes with where the threshold is set. How similar the scores are as numbers does not enter. We prove a closed-form law for the resulting effective sample size and for the spread of the coverage a deployed system actually sees. Three consequences follow. The correction now used in the conformal literature is the wrong quantity, and can miss in either direction. A dataset has no single effective sample size. It has one for each level the threshold is set at. And the damage is invisible in coverage averaged over many runs, and fully felt by whoever deploys once. On a released calibration set of 25,028 examples, we measure the reliability of about 1,300.
Primary: Unknown
All Institutions: Unknown
The paper provides a rigorous theoretical and empirical correction for the effective sample size in clustered conformal prediction, demonstrating that standard validity checks fail to detect coverage dispersion issues caused by data clustering, thereby enhancing the reliability of uncertainty quantification in ML systems.
The paper presents a rigorous theoretical derivation of the "exceedance design effect," correcting the application of Kish's design effect from survey statistics to the specific context of conformal prediction thresholds. The core methodological insight is that the effective sample size for a quantile-based threshold depends on the intra-class correlation of the *binary exceedance indicators* (whether scores fall above/below the threshold), not the correlation of the scores themselves. This distinction is critical because the binary nature of the threshold operation changes the variance structure, particularly in the tails. The authors derive a closed-form law for the coverage dispersion and extend it to ragged clusters (varying cluster sizes) using a size-biased mean substitution. The approach is mathematically sound, leveraging Bahadur representations and empirical process theory under clustering assumptions.
The experimental evaluation is strong and directly supports the theoretical claims. The authors utilize a released process-reward calibration set (25,028 examples) to demonstrate that clustered data can have an effective sample size of ~1,300, a significant reduction. They perform extensive simulations across various copula families (Gaussian, t, Clayton) to validate the theoretical predictions of coverage dispersion. Crucially, they analyze a real-world LLM calibration scenario (CoNLL-2003 sentences clustered by document) to show that standard permutation-based validity checks fail to detect the violation of exchangeability caused by clustering, leading to overconfident and miscalibrated systems. The empirical evidence effectively bridges the gap between abstract statistical theory and practical ML deployment issues.
The paper provides detailed mathematical derivations and references specific code paths (e.g., `marginal_guarantee_exact.py`) and datasets (released PRM calibration set, CoNLL-2003). The theoretical results are self-contained, and the simulation parameters are clearly defined. The use of a released dataset enhances reproducibility for the empirical portion. However, the "released artifact" mentioned in §6.1 might require specific access or processing steps not fully detailed in the text alone, though the general methodology is clear.
The theoretical results rely on asymptotic approximations (large number of clusters $b$) and specific assumptions about the dependence structure (exchangeability within clusters, continuous marginal distributions). The paper acknowledges that "informative cluster sizes" (where cluster size is correlated with the score) can introduce first-order bias that the design effect correction alone does not address, requiring additional weighting strategies. The correction is specific to the coverage dispersion and does not fully resolve bias in the mean coverage under complex sampling designs without further adjustments.
This work has significant implications for the reliability and safety of machine learning systems, particularly in conformal prediction, abstention gates, and safety filtering. By providing a correct method to estimate effective sample sizes in the presence of clustering, it helps practitioners avoid overconfident uncertainty estimates. This is crucial for high-stakes applications where miscalibration can lead to harmful decisions. The paper also highlights a systemic oversight in the ML literature regarding the distinction between shift and dependence in calibration data, encouraging more rigorous statistical validation in future work. The paper provides a rigorous theoretical and empirical correction for the effective sample size in clustered conformal prediction, demonstrating that standard validity checks fail to detect coverage dispersion issues caused by data clustering, thereby enhancing the reliability of uncertainty quantification in ML systems.
Federated learning enables privacy-preserving collaborative training, but highly heterogeneous client data remain challenging, especially in graph federated learning where clients possess structurally diverse graphs. Existing personalized federated learning (PFL) methods ignore the intrinsic geometric properties of diverse graph structures. We propose FlatLand, a novel personalized federated learning method that embeds different clients' data in tailored Lorentz space of hyperbolic geometry. Our key insight is that hyperbolic geometry naturally accommodates the intrinsic negative curvature prevalent in real-world graphs, while the time-like dimension in Lorentz space provides a principled way to encode client-specific heterogeneity. We develop a parameter decoupling strategy that separates heterogeneous information (captured in time-like parameters) from common knowledge (preserved in space-like parameters), enabling direct aggregation without requiring client similarity estimation and extra calculation modules. Empirical results on diverse federated graph learning tasks demonstrate that FlatLand achieves superior performance, particularly in low-dimensional settings.
Primary: The Chinese University of Hong Kong
All Institutions: The Chinese University of Hong Kong, Yale University, The Hong Kong University of Science and Technology (Guangzhou), Huawei Technologies Co., Ltd.
FlatLand introduces a novel geometric decoupling strategy for personalized federated graph learning, leveraging Lorentzian hyperbolic geometry to separate heterogeneous client information from common knowledge, thereby enhancing performance and privacy in structurally diverse federated settings.
The paper proposes FlatLand, a Personalized Federated Learning (PFL) framework for Graph Neural Networks (GNNs) that utilizes Lorentzian hyperbolic geometry. The core innovation lies in decoupling client-specific heterogeneity into a "time-like" dimension and common knowledge into "space-like" dimensions within the hyperbolic embedding space. This allows for aggregation of the space-like parameters while keeping time-like parameters local, theoretically addressing the challenge of structural heterogeneity in federated graph learning. The approach combines geometric insights with practical FL constraints. While the application of hyperbolic geometry to GNNs is not entirely new, its specific integration into a parameter-decoupling PFL framework for heterogeneous graphs is a distinct and well-motivated contribution. The mathematical formulation appears sound, leveraging the properties of the Lorentz model to handle negative curvature inherent in graph structures.
The authors evaluate FlatLand on diverse federated graph learning tasks. The abstract claims superior performance, particularly in low-dimensional settings. The paper includes 9 figures and 8 tables, suggesting a comprehensive empirical study. However, without access to the specific baselines and dataset details in the full text, the magnitude of improvement and statistical significance must be inferred from the abstract's claims and the venue's reputation. The focus on "low-dimensional settings" is a specific niche; performance in high-dimensional Euclidean spaces or compared to state-of-the-art Euclidean PFL methods is less clear from the summary. The inclusion of ablation studies on the decoupling strategy is implied but not explicitly detailed in the abstract.
The paper is 34 pages with extensive appendices, which typically indicates good documentation of hyperparameters and experimental setups. The methodology relies on standard hyperbolic operations and FL communication protocols, which are generally reproducible. The authors list affiliations with major tech companies and universities, suggesting access to computational resources. However, the specific implementation details of the "tailored Lorentz space" per client and the optimization dynamics are critical for reproduction and are likely contained in the full text.
A key limitation of hyperbolic embeddings is numerical instability and optimization difficulty, especially in high dimensions, although the paper claims success in low dimensions. The "tailored" aspect implies per-client geometry, which might increase memory overhead or complexity in communication if not handled carefully, although the decoupling strategy aims to mitigate this. The effectiveness of the method depends heavily on the assumption that graph heterogeneity can be effectively captured by the time-like dimension; if heterogeneity is more complex, the model might underfit. Additionally, the claim of "no client similarity estimation" is a strong advantage, but the quality of the decoupling relies on the learning dynamics, which might be sensitive to initialization or learning rates.
This work contributes to the growing field of geometric deep learning and privacy-preserving AI. By providing a principled way to handle heterogeneous data in federated settings without sharing sensitive client-specific parameters, it advances the practical applicability of FL in domains like social networks or biological graphs where data is inherently non-Euclidean and heterogeneous. The privacy-by-design aspect, avoiding the sharing of similarity matrices, is a significant benefit for sensitive applications. FlatLand introduces a novel geometric decoupling strategy for personalized federated graph learning, leveraging Lorentzian hyperbolic geometry to separate heterogeneous client information from common knowledge, thereby enhancing performance and privacy in structurally diverse federated settings.
For AI agents to be useful beyond simple chat, they must hold sensitive user context such as calendars, credentials, health records, and financial data. We study whether the mere presence of such secrets in a model's context window introduces hidden correlations into the model's benign outputs, allowing reconstruction even when the model correctly refuses direct extraction. We further study whether an adversary can actively engineer prompts that amplify this effect, using the model as a covert carrier to transmit secrets through seemingly innocuous text. In both cases, this limited leakage is exploited using a novel adaptive attack that assumes black-box access to the underlying model. In controlled experiments across eight proprietary models, we find that 2-digit in-context secrets are reconstructed with near-perfect accuracy and 4-digit secrets at 82\% exact match, all from outputs the model produces in response to ordinary, non-adversarial requests. We observe that more capable models leak more: stronger instruction-following amplifies sensitivity to in-context secrets, suggesting leakage is a byproduct of capability as opposed to a patchable bug. We show this leakage enables two practical attacks: (1) a trained classifier that infers semantic predicates about user memories (e.g., health conditions, financial events) from routine natural-language outputs, and (2) an RL-trained adversary that extracts full Social Security Numbers from a production-style agent.
Primary: University of California, San Diego
All Institutions: University of California, San Diego, University of Maryland
This paper presents a rigorous and impactful analysis of inadvertent context leakage in language models, demonstrating that sensitive information can be reconstructed from benign outputs through statistical side-channels, even when the model explicitly refuses to disclose it. The introduction of adaptive black-box decoders to exploit these channels provides a new and critical perspective on LLM privacy, with significant implications for the deployment of AI agents handling sensitive user data.
The paper introduces a novel threat model termed "inadvertent context leakage," moving beyond traditional direct extraction or prompt injection attacks. The core methodology involves training adaptive, black-box decoders (using LoRA-finetuned classifiers) to detect statistical correlations between in-context secrets and benign model outputs. The authors isolate specific leakage channels, including response length, stylistic features, and digit frequency profiles, demonstrating that these channels persist even when the model explicitly refuses to output the secret. The approach is rigorous, employing a controlled experimental setup across eight proprietary frontier models and utilizing both passive inference and active RL-optimized prompt injection to engineer covert channels.
The experimental evaluation is comprehensive and convincing. The authors test on eight leading proprietary models (Claude, Gemini, GPT, Grok) using a variety of secret lengths (1-8 digits) and semantic predicates. Key findings include near-perfect reconstruction of 2-digit secrets and 82% exact match for 4-digit secrets on Claude Opus 4.6. The paper provides strong ablation studies, including the "suppression" hypothesis (that stronger instructions to hide data paradoxically increase leakage by distorting the output distribution), cross-model transferability of decoders, and the effectiveness of the attack against production-style agents (SSN extraction). The results are statistically significant and robust across different prompt templates and context sizes.
The paper provides detailed descriptions of the attack pipelines, decoder architectures (Qwen-2.5-1.5B backbone with LoRA), and hyperparameters. The use of public APIs for proprietary models limits exact reproducibility of the victim models' internal states, but the attack methodology is clearly defined. The authors provide prompt templates and describe the GRPO optimization process for the active attack. The data collection process (1,000 secrets, 10 responses per pair) is well-documented, allowing other researchers to replicate the study on accessible models or verify the API interactions.
The study is limited to numeric secrets and binary semantic predicates; generalization to open-vocabulary, free-text secrets is not addressed due to the combinatorial explosion of the value space. The experiments are conducted in a black-box setting via APIs, which may not capture all side-channels available in local deployments. The "resistant" tier of models (Grok, GPT-nano) suggests that leakage is model-dependent, and the mechanisms for resistance are not fully understood or mitigated. The study assumes the adversary can query the model, which may not always be the case in all deployment scenarios.
This work has significant implications for the privacy of AI agents and the security of LLM deployments. It challenges the assumption that instruction-following and refusal mechanisms are sufficient for data protection. The findings suggest that privacy in LLMs requires fundamental architectural changes or information-theoretic guarantees rather than just prompt engineering or output filtering. This could drive new research into privacy-preserving model training, secure inference protocols, and standardized privacy benchmarks for AI agents. This paper presents a rigorous and impactful analysis of inadvertent context leakage in language models, demonstrating that sensitive information can be reconstructed from benign outputs through statistical side-channels, even when the model explicitly refuses to disclose it. The introduction of adaptive black-box decoders to exploit these channels provides a new and critical perspective on LLM privacy, with significant implications for the deployment of AI agents handling sensitive user data.
Hidden chain-of-thought (CoT) traces, especially those from frontier proprietary large reasoning models (LRMs), are valuable model assets. Yet whether these hidden CoTs can be directly extracted from black-box models remains largely unexplored. In this work, we systematically study whether hidden CoTs can be extracted near-verbatim from black-box LRMs through API interactions. We identify a previously overlooked reasoning replay surface between tool calls and develop EchoCoT, a multi-step attack that iteratively extracts hidden CoTs using API-returned fidelity signals. We further develop an LLM-based optimization framework that automatically searches for an effective universal injection trajectory across various datasets. We evaluate EchoCoT on three open-source and five frontier proprietary LRMs. On open-source LRMs, EchoCoT achieves up to 66.4\% near-verbatim extraction success, with the extracted trace length within 10\% of the target and at least 90\% of tokens exactly matching the target CoT. The same injection trajectory also generalizes to unseen datasets, achieving up to 80\% extraction success under the same criterion. For tested frontier proprietary LRMs, a substantial fraction of extracted CoTs closely align with provider-reported reasoning lengths and available CoT summaries. EchoCoT can also extract very long CoTs: on Gemini-2.5, it extracts 33,463 tokens from a 32,948-token target. These results establish hidden-CoT extraction as a practical security risk and highlight the need to better protect hidden CoT assets.
Primary: CISPA Helmholtz Center for Information Security
All Institutions: CISPA Helmholtz Center for Information Security
EchoCoT establishes a practical and effective method for extracting hidden Chain-of-Thought traces from black-box Large Reasoning Models, exposing a significant security vulnerability in current API designs and setting a new benchmark for CoT privacy research.
The paper proposes EchoCoT, a sophisticated black-box attack that exploits the "reasoning replay surface" inherent in tool-calling mechanisms of Large Reasoning Models (LRMs). The core insight is that while standard multi-turn conversations discard hidden CoTs, tool calls preserve them within the context window. By iteratively injecting instructions via a scratchpad tool, the authors force the model to reproduce its hidden reasoning. The novelty lies in the automated optimization framework (LLM-based Inject-Reflect-Distill) that searches for universal injection trajectories using only API-provided proxy signals (token counts and summaries) as fidelity feedback. This is a clever adaptation of prompt optimization techniques to the specific constraints of black-box CoT extraction. The methodology is technically sound and demonstrates a deep understanding of LRM internal states and API behaviors.
The evaluation is extensive and rigorous. The authors test EchoCoT on three open-source LRMs (DeepSeek, Qwen, GLM) with ground-truth CoTs available for precise measurement, and five frontier proprietary models (Gemini, Claude) where they use proxy metrics and qualitative analysis. The results are striking: on open-source models, they achieve near-verbatim extraction (Token-EM > 0.90) for a significant fraction of samples. The cross-dataset transferability is also strong, with optimized trajectories working well on unseen datasets. The ability to extract very long CoTs (33k+ tokens) is a significant empirical finding. The baselines (Direct Prompting, CoT Synthesis, REP) are appropriate, and EchoCoT vastly outperforms them.
The paper provides a detailed description of the attack pipeline, optimization framework, and evaluation metrics. The code is available on GitHub. The use of public APIs for proprietary models ensures that the attack can be reproduced by others, provided they have access to the same models. The open-source model evaluations are fully reproducible.
The attack relies on the availability of tool-calling interfaces and specific API signals (reasoning token counts, summaries). Models that disable tool calls or hide these signals would be resistant. The attack is also computationally expensive due to the iterative nature and the need for multiple LLM calls for optimization. Furthermore, the success rate, while high, is not 100%, and failures often stem from length mismatches or refusals. The paper notes that defenses are possible but currently insufficient.
This work has profound implications for the security and privacy of frontier LRM providers. It demonstrates that hidden CoTs, considered valuable intellectual property and safety assets, are not secure against determined black-box attackers. This could lead to industrial espionage, leakage of sensitive training data, or bypassing of safety filters embedded in the reasoning process. It highlights a critical gap in the security design of LRM APIs and necessitates new defense mechanisms, such as obfuscating reasoning steps, limiting tool-call context, or detecting adversarial tool usage. EchoCoT establishes a practical and effective method for extracting hidden Chain-of-Thought traces from black-box Large Reasoning Models, exposing a significant security vulnerability in current API designs and setting a new benchmark for CoT privacy research.
Multi-agent LLM systems split work across models, so answering often requires knowledge that sits in another agent's context: a Sharer has encoded information that a Receiver needs to complete its task. They usually communicate by exchanging text, which puts autoregressive decoding on the critical path and reduces the exchange to a discrete message written without sight of the receiver's state. Recent latent protocols instead translate the sharer's key-value (KV) cache into the receiver's: C2C supports heterogeneous models but requires both to read the same input, while LCF-X removes this shared-context requirement through position-free sharer-cache pooling. Three restrictions remain: LCF-X compresses the sharer alone, supplies the same layer-local summary to every receiver position with no joint cross-layer memory to retrieve from, and assumes matched layer count and KV geometry. We introduce XKV, which lifts all three: learned-query attention pools both caches; self-attention over receiver-aligned layer tokens, with a learned layer map reconciling different depths, mixes the pooled summaries into a compact joint memory; and a shared position decoder lets every raw receiver cache position retrieve its own per-head-gated residual in the receiver's native KV geometry. Both models stay frozen and may differ in family, depth, KV-head count, head dimension, and tokenizer; only the translator is trained. Across 45 dataset-model-pair settings (six heterogeneous and three same-model ordered pairings, five datasets), XKV attains the highest macro score and best average rank, improving on LCF-X on every dataset (by 4.6 exact-match and 4.2 F1 points on ROPES) and surpassing text communication on four of the five, while training 76% fewer parameters and translating a cache pair 10.3x faster (5.8 vs. 59.9 ms); end to end, XKV is 26% faster than LCF-X and 6.8x faster than text communication.
Primary: Amazon
All Institutions: Amazon
[One sentence main contribution]. XKV introduces a dual-cache latent communication protocol that pools both sharer and receiver KV caches into a joint memory, enabling efficient, receiver-aware, and geometry-agnostic knowledge transfer between heterogeneous frozen LLMs. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper makes a solid technical contribution to the emerging area of latent communication in multi-agent systems. By addressing the limitations of prior work (LCF-X) through dual-cache pooling and position-specific retrieval, it achieves state-of-the-art accuracy and efficiency on split-evidence reasoning tasks. The rigorous evaluation across heterogeneous model pairs and the clear demonstration of efficiency-accuracy trade-offs make this a valuable addition to the literature. The method is well-motivated, technically detailed, and empirically validated.
The paper proposes XKV, a "dual-cache" latent communication protocol for heterogeneous frozen LLMs. The core innovation lies in pooling both the sharer's and receiver's KV caches using learned-query attention to create a joint memory, rather than just compressing the sharer's cache as in prior work (LCF-X). This joint memory is then queried by each receiver position to retrieve position-specific, receiver-native KV residuals. The method includes a learned layer map to handle depth mismatches and a shared position decoder. The approach is technically sound, leveraging standard attention mechanisms in a novel configuration for cross-model state transfer. It addresses specific bottlenecks in previous latent communication methods: receiver-independence, layer-local single-summary bottlenecks, and matched-geometry assumptions.
The evaluation is comprehensive, covering 45 dataset-model-pair settings across five benchmarks (ROPES, MuSiQue, QASC, StrategyQA, HotpotQA-bridge) and nine ordered pairs of Qwen, Gemma, and Llama models. XKV consistently outperforms text-to-text (T2T) and the latent baseline LCF-X in accuracy/F1 scores. Crucially, it demonstrates significant efficiency gains: 10.3x faster communication latency than LCF-X and 6.8x faster end-to-end than T2T, with 76% fewer parameters. The ablation studies effectively isolate the contribution of the dual-cache pooling and position-specific retrieval. The results are robust and clearly presented.
The paper provides detailed mathematical formulations for the pooling, layer alignment, and decoding steps. It specifies the model families, dataset splits, and evaluation metrics. The claim that both base models are frozen and only the translator is trained simplifies the reproduction of the inference pipeline. The latency measurements are specific and comparable. While code is not explicitly linked in the text provided, the methodological description is sufficient for reproduction by a competent researcher.
The method assumes a "cross-context" setting where evidence is split between two models. It does not address multi-turn conversations or many-agent systems with complex topologies, though this is noted as future work. The performance gain over LCF-X, while consistent, is moderate in absolute terms (e.g., +4.6 EM on ROPES). The reliance on a learned translator introduces a small overhead, albeit minimal compared to the savings. The evaluation is limited to small-to-medium models (0.6B-3B); scaling to larger models (7B+) is not demonstrated.
This work advances the field of multi-agent LLM systems by enabling more efficient and accurate latent communication. By removing the need for text decoding/encoding cycles and supporting heterogeneous models, it lowers the barrier to building distributed LLM systems. The efficiency gains could lead to reduced latency and cost in deployed agent frameworks. The approach of joint state pooling is a generalizable technique that could inspire similar methods in other domains requiring cross-model information transfer. [One sentence main contribution]. XKV introduces a dual-cache latent communication protocol that pools both sharer and receiver KV caches into a joint memory, enabling efficient, receiver-aware, and geometry-agnostic knowledge transfer between heterogeneous frozen LLMs. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper makes a solid technical contribution to the emerging area of latent communication in multi-agent systems. By addressing the limitations of prior work (LCF-X) through dual-cache pooling and position-specific retrieval, it achieves state-of-the-art accuracy and efficiency on split-evidence reasoning tasks. The rigorous evaluation across heterogeneous model pairs and the clear demonstration of efficiency-accuracy trade-offs make this a valuable addition to the literature. The method is well-motivated, technically detailed, and empirically validated.
Self-normalized concentration inequalities are standard tools in bandit and reinforcement-learning analyses. A widely used weighted extension claims an analogous time-uniform guarantee for discounted least-squares estimators in non-stationary problems. A simple scalar Gaussian counterexample with a fixed parameter shows that the claimed bounded radius is crossed with probability one. For fixed discount and regularization parameters, we further show that, when $δ\leq1/2$ and $T/δ$ is sufficiently large, any deterministic anytime boundary valid uniformly over the stated conditionally sub-Gaussian model class must be at least of order $R\sqrt{\log(T/δ)}$ at some time by horizon $T$; for nondecreasing boundaries, this order is required at time $T$. We identify the proof error: different terminal times use different Gaussian mixing distributions, so the fixed-time mixtures do not form one supermartingale, and the stopping-time argument does not repair this failure. Finally, we show that the weighted inequality remains valid at each fixed deterministic time, give valid finite- and infinite-horizon corrections, and discuss consequences for downstream analyses.
Primary: Academia Sinica
All Institutions: Academia Sinica, National Tsing Hua University
This paper provides a crucial correction to the theoretical analysis of discounted least-squares in non-stationary bandits, identifying a fundamental flaw in the proof of a widely used time-uniform concentration inequality and establishing the necessity of logarithmic confidence costs for valid anytime bounds.
The paper employs a rigorous theoretical analysis to identify a fundamental flaw in the proof of a widely used self-normalized concentration inequality for discounted least-squares estimators. The methodology involves constructing a scalar Gaussian counterexample to disprove the time-uniform bound claimed in prior work (RVC19weightedLB) and deriving matching lower bounds for any valid anytime boundary. The authors correctly identify the logical error in the original proof: the use of different Gaussian mixing distributions at different times prevents the construction of a single supermartingale required for Ville's inequality, and the proposed "stopping-time repair" is invalid because it conditions on a sequence of directions rather than a fixed one. This is a classic example of "e-hacking" or pseudo-maximization in sequential analysis. The correction provided involves explicit confidence allocation (union bound) which restores validity but at the cost of a logarithmic factor in the horizon.
As a theoretical note, the paper does not present empirical experiments in the traditional sense (e.g., training models on datasets). Instead, the "experiments" are the mathematical proofs of the counterexample and the lower bounds. The scalar Gaussian example serves as a synthetic proof-of-concept that the theoretical bounds fail. The evaluation is purely analytical, relying on probability theory and concentration inequalities.
The mathematical derivations, proofs, and counterexamples are fully detailed in the text and appendices. The logic is self-contained and reproducible by any researcher with a background in stochastic processes and bandit theory. The identification of the error in prior work is precise and verifiable.
The primary limitation is that this is a "negative" result paper; it identifies an error and provides a correction, but does not propose a new, more efficient algorithm or a significantly tighter bound than the corrected union-bound approach. The correction (union bound) is standard but may be loose in practice compared to what a correct time-uniform bound might achieve if one existed (though the lower bound suggests the logarithmic factor is necessary for non-decreasing boundaries). The scope is limited to the specific discounted least-squares setting and does not address all possible variations of non-stationary bandit algorithms.
This paper has significant impact on the theoretical foundations of non-stationary bandit and reinforcement learning literature. By correcting a widely cited inequality, it invalidates the theoretical guarantees of several downstream papers (RCG20-NSGLB, KT20, TV20, etc.) that relied on the flawed anytime bound. It forces the community to re-evaluate the regret bounds of these algorithms, potentially requiring them to pay an additional logarithmic factor or rely on fixed-time analysis with union bounds. It serves as an important cautionary tale regarding the application of Ville's inequality and supermartingale arguments in time-varying settings. This paper provides a crucial correction to the theoretical analysis of discounted least-squares in non-stationary bandits, identifying a fundamental flaw in the proof of a widely used time-uniform concentration inequality and establishing the necessity of logarithmic confidence costs for valid anytime bounds.
Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal computation of large models make inference prohibitively expensive. We present TurboT2VA, a distillation and inference framework for accelerating a 19B-parameter joint video-audio model. Large-scale T2VA distillation is challenged by modality-imbalanced optimization, the difficulty of continuous-time consistency training at scale, and the quality--diversity trade-off. TurboT2VA addresses these issues with per-modality normalization and a progressive curriculum comprising discrete consistency warm-up, continuous consistency refinement, and joint consistency--distribution matching. The curriculum first establishes a stable, diverse generation trajectory and only then introduces distribution-level refinement. On LTX-2, four-step distillation reduces generator latency from 50.52s to 2.51s at the standard evaluation resolution of 512$\times$768, achieving a 20.1$\times$ speedup while maintaining strong visual quality, audio fidelity, diversity, and video-audio synchronization. We further develop an architecture-aware inference stack that combines guarded W8A8 and fused operators, padded-text compaction, and modality-aware sparse attention while preserving dense cross-modal and text-conditioning paths. Under the high-resolution deployment setting at 1024$\times$1792, the complete stack reduces generator latency from 318.74s to 5.83s on one NVIDIA H20, achieving a 54.67$\times$ generator-only speedup. Inference code and generation demos are available at https://github.com/thu-ml/TurboDiffusion/tree/main/turbot2va.
Primary: Tsinghua University
All Institutions: Tsinghua University, Zhejiang University, National University of Singapore, Qingdao University, Tianjin University, Lionrock AI
[One sentence main contribution]. TurboT2VA introduces a progressive curriculum for score-regularized consistency distillation and an architecture-aware inference stack to accelerate 19B-parameter joint text-to-video-audio generation, achieving up to 54x speedup with minimal quality loss. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. This paper makes a substantial contribution to the field of efficient multimodal generation. By successfully extending consistency distillation and distribution matching to a 19B-parameter joint video-audio model, it addresses critical challenges in modality imbalance and training stability. The proposed three-stage curriculum effectively balances the quality-diversity trade-off, a known issue in few-step generation. Furthermore, the integration of a specialized inference stack demonstrates a holistic approach to deployment efficiency, combining algorithmic improvements (distillation) with systems optimizations (quantization, sparse attention). The rigorous evaluation against state-of-the-art closed and open-source models validates the effectiveness of the approach. While the core distillation techniques are not entirely new, their application and adaptation to the complex, heterogeneous domain of joint video-audio generation, along with the detailed systems engineering, represent a significant step forward for practical deployment of large multimodal models. The work is highly relevant to researchers and practitioners working on efficient diffusion/flow models and multimodal synthesis.
The paper proposes TurboT2VA, a framework for accelerating 19B-parameter joint text-to-video-audio (T2VA) generation. The core methodological contribution is a progressive curriculum for score-regularized consistency distillation (sCM) combined with distribution matching distillation (DMD). The authors address three key challenges in joint modality distillation: modality imbalance (via per-modality normalization), numerical instability (via a three-stage curriculum: dCM warm-up, sCM refinement, sCM+DMD joint), and the quality-diversity trade-off. Additionally, they develop an architecture-aware inference stack involving modality-aware sparse attention (SageSLA), W8A8 quantization, and fused operators. The approach is technically sound, leveraging recent advances in consistency distillation and efficient inference. However, the novelty is somewhat incremental; it applies known distillation and acceleration techniques (sCM, DMD, W8A8, sparse attention) to a new, complex multimodal domain (T2VA) rather than introducing fundamentally new theoretical mechanisms. The "joint" aspect is significant but builds heavily on existing single-modality precedents.
The experimental evaluation is comprehensive and rigorous. The authors evaluate on LTX-2, comparing against closed-source (Sora 2, Kling v3, Veo 3) and open-source (JavisDiT, OVI, DaVinci-MagiHuman) baselines. They provide extensive metrics including JavisBench, VBench, TTA-Bench, and specific video-audio synchronization metrics (CAVP, AVH, Desync). The results demonstrate a significant speedup (20.1x at 512x768, 54.67x at 1024x1792) while maintaining competitive or superior quality and synchronization scores compared to the 40-step teacher and other baselines. The ablation studies effectively isolate the contributions of the curriculum stages and the inference components. The inclusion of high-resolution latency breakdowns adds practical value.
The paper provides detailed hyperparameters, training configurations (8 H20 GPUs, 7K steps), and links to code. The description of the inference stack is sufficiently detailed for implementation. The use of standard benchmarks (JavisBench, VBench) aids in reproducibility and comparison.
The primary limitation is the reliance on a specific teacher model (LTX-2) and the computational cost of the distillation process itself (though reduced by the curriculum). The inference stack, while effective, introduces complexity and potential compatibility issues (e.g., fallback to BF16 for unsupported shapes). The paper does not extensively discuss the failure modes of the sparse attention or the robustness of the audio synchronization under extreme compression. The "joint" nature means that if one modality fails, the other might be affected, though the paper claims robustness.
This work significantly lowers the barrier to deploying high-quality, synchronized video-audio generation models, which has implications for creative industries, accessibility, and immersive media. By enabling real-time or near-real-time generation on single GPUs, it democratizes access to powerful multimodal models. However, it also raises concerns about the potential misuse of deepfake technology, as the generation quality and synchronization are high. [One sentence main contribution]. TurboT2VA introduces a progressive curriculum for score-regularized consistency distillation and an architecture-aware inference stack to accelerate 19B-parameter joint text-to-video-audio generation, achieving up to 54x speedup with minimal quality loss. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. This paper makes a substantial contribution to the field of efficient multimodal generation. By successfully extending consistency distillation and distribution matching to a 19B-parameter joint video-audio model, it addresses critical challenges in modality imbalance and training stability. The proposed three-stage curriculum effectively balances the quality-diversity trade-off, a known issue in few-step generation. Furthermore, the integration of a specialized inference stack demonstrates a holistic approach to deployment efficiency, combining algorithmic improvements (distillation) with systems optimizations (quantization, sparse attention). The rigorous evaluation against state-of-the-art closed and open-source models validates the effectiveness of the approach. While the core distillation techniques are not entirely new, their application and adaptation to the complex, heterogeneous domain of joint video-audio generation, along with the detailed systems engineering, represent a significant step forward for practical deployment of large multimodal models. The work is highly relevant to researchers and practitioners working on efficient diffusion/flow models and multimodal synthesis.
Visual tokenizers increasingly inject semantic supervision into latent spaces to make downstream diffusion easier. Yet how these semantics should be organized to facilitate denoising remains underexplored. In this paper, we define the semantic recovery objective: the denoising process should recover the semantic content of the clean image from noisy latent, and a good tokenizer should make it easier. Existing approaches train a projector to predict the semantics directly from the noisy latent. We argue that this predicts the average of clean-image semantics, whereas what really needs to be aligned is the semantics of averaged clean latents. More importantly, we demonstrate that the semantic recovery error orthogonally decomposes into the error of the optimal semantic prediction directly from the noisy latent and the error between these two predictions. We therefore identify their consistency as the missing requirement and call it Semantic Affine Consistency (SAC). To examine whether this overlooked requirement is closely related to downstream generation, we introduce M_SAC, a tokenizer-side proxy for SAC. Across the evaluated tokenizers and diffusion model scales, M_SAC closely tracks generation quality, reaching a Pearson correlation of 0.960 with SiT-XL gFID, thereby motivating SAC-guided tokenizer training. We then introduce AffineTok, which promotes SAC through two complementary, training-only components. Global Semantic Coordination Token (GSCT) coordinates the semantic organization of clean latents, keeping semantic averaging meaningful, while Posterior-Mean Semantic Alignment (PMSA) predicts posterior-mean latents from noisy inputs and supervises their semantics. On ImageNet 256, compared with the baseline, AffineTok reduces gFID by 26% at 20 epochs and, with continued training, achieves a new state-of-the-art gFID of 1.21 without classifier-free guidance and 1.10 with guidance.
Primary: Fudan University
All Institutions: Fudan University, Alibaba Token Hub, Alibaba Group
The paper introduces AffineTok, a visual tokenizer that enforces semantic affine consistency to significantly improve diffusion model generation quality, achieving new state-of-the-art results on ImageNet. The theoretical decomposition of semantic recovery error provides a novel perspective on latent space organization, and the strong correlation between the proposed proxy metric and generation quality offers a valuable tool for tokenizer development. While the method is effective and well-evaluated on a standard benchmark, its scalability and computational efficiency remain areas for further investigation.
The paper proposes "AffineTok," a visual tokenizer designed to improve downstream diffusion performance by enforcing "Semantic Affine Consistency" (SAC). The core theoretical contribution is the decomposition of semantic recovery error into two orthogonal components: the error of predicting semantics from noisy latents and the error between this prediction and the semantics of averaged clean latents. The authors argue that existing methods optimize the former but neglect the latter, leading to suboptimal semantic organization. To address this, they introduce two components: Global Semantic Coordination Token (GSCT) to coordinate semantic averaging, and Posterior-Mean Semantic Alignment (PMSA) to supervise semantics using posterior-mean latents. The methodology is theoretically grounded in the properties of diffusion processes and latent space geometry. While the decomposition argument is insightful, the practical implementation of GSCT and PMSA appears to be a modification of existing projector training objectives rather than a radical architectural shift. The novelty lies in the specific formulation of the consistency constraint and the proxy metric $M_{SAC}$.
The experimental evaluation is strong, focusing on ImageNet 256x256 generation quality using SiT (Scaling Image Transformers) backbones. The key result is a new state-of-the-art gFID of 1.21 (without classifier-free guidance) and 1.10 (with guidance). The paper provides a compelling correlation analysis showing that their proposed proxy metric $M_{SAC}$ correlates with generation quality (Pearson 0.960 with SiT-XL gFID), which serves as strong evidence for the validity of their theoretical framework. The comparison against baselines shows significant improvements (26% reduction in gFID at 20 epochs). The results are robust and suggest that the proposed method effectively aligns the latent space with the denoising process requirements.
The paper provides a project page URL, which likely contains code or detailed implementation details. The methodology is described with sufficient detail regarding the loss functions and architectural modifications. However, as an arXiv preprint, full reproducibility depends on the availability of the code repository, which is not explicitly linked in the text provided (only the demo site is listed). The use of standard benchmarks (ImageNet) and standard metrics (gFID) aids in reproducibility and comparison.
The paper focuses primarily on ImageNet 256x256. It is unclear how the method scales to higher resolutions (e.g., 512x512 or 1024x1024) or more complex datasets like LAION-400M without further ablation. The computational overhead of the additional supervision steps (GSCT and PMSA) is not quantified in terms of training time or memory usage compared to baselines. Additionally, the claim of "state-of-the-art" is specific to the configuration (SiT-XL, ImageNet 256), and generalization to other architectures (e.g., Stable Diffusion variants) is not demonstrated.
This work contributes to the fundamental understanding of visual tokenizers for diffusion models. By providing a better-aligned latent space, it could lead to more efficient training and higher-quality image generation, benefiting applications in creative AI, simulation, and content creation. The improved efficiency (higher quality with potentially fewer steps or better guidance) could reduce the carbon footprint of training large generative models. The paper introduces AffineTok, a visual tokenizer that enforces semantic affine consistency to significantly improve diffusion model generation quality, achieving new state-of-the-art results on ImageNet. The theoretical decomposition of semantic recovery error provides a novel perspective on latent space organization, and the strong correlation between the proposed proxy metric and generation quality offers a valuable tool for tokenizer development. While the method is effective and well-evaluated on a standard benchmark, its scalability and computational efficiency remain areas for further investigation.
Recent Vision-Language Models encode high-resolution images into long visual token sequences, incurring prohibitive prefill costs. To compress them, existing methods score each visual token by averaging text-to-visual attention uniformly across all heads, which assumes every head matches the query. However, our empirical analysis shows that misaligned heads dominate the average, amplifying background tokens and drowning out fine-grained cues. To address this, we propose PAQ (Prompt-Grounded Attention Quality), a metric quantifying how well each head aligns the prompt with image regions. Built on PAQ, our pruning proceeds in three stages. Given a target FLOPs budget, we first partition the transformer layers into groups and allocate a visual token budget to each. Within each group, we then aggregate per-head attention maps via PAQ-weighted softmax into a group-level matrix. Finally, we score visual tokens by this matrix's magnitude and retain the allocated budget per group. By weighting heads with PAQ, our method scores tokens by attention signals that more faithfully reflect prompt relevance, rather than diluting them through uniform averaging. Across 18 benchmarks, our method delivers state-of-the-art trade-offs. Specifically, on LLaVA-1.5-7B (9 tasks), retaining only \textbf{5.6\%} tokens preserves \textbf{99.1\%} of the original performance, surpassing the strongest baseline AutoPrune by 4.2 points. Code is available in https://github.com/baokou-fw2/HAP.
Primary: Shanghai Jiao Tong University
All Institutions: Shanghai Jiao Tong University, University of Edinburgh
[One sentence main contribution]. [The paper introduces HAP, a training-free visual token pruning framework for VLMs that utilizes a novel information-theoretic metric (PAQ) to selectively aggregate attention from prompt-aligned heads, achieving state-of-the-art efficiency-accuracy trade-offs across multiple architectures and benchmarks without hyperparameter tuning.]
The paper proposes HAP, a training-free visual token pruning method for Vision-Language Models (VLMs). The core innovation is the Prompt-Grounded Attention Quality (PAQ) metric, which uses the uncertainty coefficient to quantify the mutual information between text tokens and visual tokens in each attention head. This allows the method to distinguish between "optimal" heads (which attend to prompt-relevant regions), "sink" heads (which attend to fixed background/edges), and "dispersed" heads. HAP then uses PAQ scores to weight the aggregation of attention maps across heads and layers, rather than averaging them uniformly as in prior work (e.g., PDrop, FastV). The method also employs a geometric pyramid schedule for token budget allocation across layers. The methodology is theoretically sound, leveraging information-theoretic principles to address a specific failure mode in existing pruning techniques. The integration with FlashAttention via a "dual-flash" strategy to avoid materializing full attention matrices is a practical and valuable engineering contribution.
The authors evaluate HAP on 18 benchmarks across 5 different VLM architectures (LLaVA-1.5-7B, Qwen2.5-VL-7B, Qwen3-VL-8B, InternVL3-8B, DeepSeek-VL2 Small-16B). The results demonstrate state-of-the-art performance-efficiency trade-offs. Notably, on LLaVA-1.5-7B, retaining only 5.6% of tokens preserves 99.1% of the original performance, outperforming the strongest baseline (AutoPrune) by 4.2 points in relative score. The paper includes extensive ablation studies, robustness checks (prompt variations, corruptions), and efficiency analyses (latency, KV cache). The evaluation is comprehensive and convincing. The use of multiple architectures strengthens the claim of generalizability.
The paper provides a GitHub link to the code. The methodology is described in detail, including the PAQ formula and the aggregation steps. The authors claim the method is training-free and requires no per-task hyperparameter tuning, which aids reproducibility. The closed-form FLOPs derivation and compatibility with FlashAttention are well-documented. The statistical robustness is verified over 10 runs.
The method operates only during the prefill stage, leaving the visual KV cache untouched during decoding. While the paper argues this still provides significant benefits, it does not address potential inefficiencies during long-generation tasks as thoroughly as methods that prune throughout decoding. Additionally, PAQ relies on cross-modal attention, which assumes paired text queries; the authors acknowledge that applying similar criteria to unimodal compression is future work. The method may be less effective on models with very different attention mechanisms or those that do not exhibit the sink/dispersed/optimal head patterns observed in the studied VLMs.
This work contributes to the efficient deployment of large multimodal models, making them more accessible for resource-constrained environments and real-time applications. By improving the efficiency of VLMs without retraining, it lowers the barrier to entry for using these powerful models. The insights into head heterogeneity in VLMs also provide valuable understanding for future model design and analysis. [One sentence main contribution]. [The paper introduces HAP, a training-free visual token pruning framework for VLMs that utilizes a novel information-theoretic metric (PAQ) to selectively aggregate attention from prompt-aligned heads, achieving state-of-the-art efficiency-accuracy trade-offs across multiple architectures and benchmarks without hyperparameter tuning.]
Reasoning-augmented text-to-image models such as GoT-R1 emit an explicit textual plan - object names, attributes, and bounding boxes - before generating image tokens. When such a model fails a compositional prompt, is the plan wrong, or is the plan right and the decoder unfaithful? Because the plan is machine-readable it can be edited before decoding, which makes the two separable. We first validate the ruler. Swapping the two bounding boxes inside the model's own chain demonstrably flips the generated layout: detector-based accuracy falls 0.75 -> 0.48 (p<1e-3), while a widely used VQA-based spatial metric rises. A five-rater human study agrees with the detector on 81% of items and with the VQA judge on 57%. All spatial results therefore use geometric scoring. Under sound measurement the decoder is a faithful executor: 94% of generated layouts realize the planned relation, and object-box binding survives reordering of the plan's object segments. The planner is the bottleneck. It writes wrong relations for phrasing-dependent reasons - 98% accuracy on "left" against 54% on "right" for semantically identical layouts, a raster-order bias we isolate with a mention-order control - and cluttered geometry that the decoder faithfully reproduces. Editing the plan therefore fixes the image without retraining: symbolic verification with resampling gives +5.0 points (p<1e-3), minimal in-place repair +6.0 (p=.02), rewriting only box geometry +10.7 (p<1e-4), and replacing the plan outright +13.3 (p=1e-4). Gains are indifferent to the plan's prose style and to its likelihood under the planner, but not to its geometry. Modular planner-decoder designs are therefore viable, provided the plan is internally consistent: box-text contradictions induce object duplication and identity fusion. We release the plan-fidelity evaluation protocol, all plans, and 12k generated images.
Primary: Carnegie Mellon University
All Institutions: Carnegie Mellon University
The paper makes a significant contribution by rigorously diagnosing the source of compositional failures in reasoning-augmented text-to-image models, demonstrating that the decoder is a faithful executor and the planner is the bottleneck, thereby validating inference-time plan repair as a highly effective, training-free intervention.
The paper employs a rigorous perturbation-based methodology to diagnose failure modes in reasoning-augmented text-to-image models (specifically GoT-R1). By treating the textual plan as an editable intermediate representation, the authors decouple planning errors from execution errors. The core methodological contribution is the "box swap" intervention and the "oracle dial" experiment, which systematically vary plan geometry and text style to determine what the decoder actually conditions on. This approach is methodologically sound, leveraging controlled interventions to isolate causal factors in complex generative pipelines. The use of paired permutation tests and hardware-matched controls adds significant statistical rigor compared to standard benchmark reporting.
The experimental evaluation is comprehensive and convincing. The authors validate their metric choice by showing that VQA-based spatial metrics are blind to layout inversions, while detector-based geometric scoring correctly identifies them. Their findings—that the decoder is a faithful executor (94% fidelity) and the planner is the bottleneck—are supported by extensive ablations, including seed robustness checks and scaling to a 7B model. The "oracle dial" results are particularly strong, demonstrating that clean geometric plans outperform the model's own plans regardless of text style or likelihood, directly challenging the co-adaptation hypothesis. The statistical significance of all reported gains is established.
The paper provides a high level of reproducibility. The authors explicitly state that they release the plan-fidelity evaluation protocol, all plans, 12k generated images, and analysis code. The experimental setup is well-documented, including specific model versions (GoT-R1-1B/7B), decoding parameters (CFG, temperature, seeds), and the exact nature of the interventions (verify, repair, replace). The inclusion of a reproducibility statement and the release of raw data and scripts significantly lowers the barrier for replication.
The primary limitation is the scope of the evaluation. The study focuses exclusively on the GoT-R1 model family (1B and 7B). While the authors acknowledge this and suggest generalizing to other architectures (T2I-R1, BAGEL), the findings regarding the decoder's faithfulness and the planner's specific biases (raster-order) may not transfer universally to all reasoning-augmented T2I models. Additionally, the evaluation is limited to validation subsets of T2I-CompBench++, and while the metrics are validated, the detector-based scoring inherits the limitations of the underlying open-vocabulary detector (OWLv2).
This work has significant implications for the design of multimodal generative systems. By proving that modular planner-decoder designs are viable and that the planner is the primary bottleneck, it shifts the research focus from end-to-end training to inference-time plan verification and repair. This could lead to more robust and controllable image generation systems. The ethical statement appropriately notes the dual-use nature of improved compositional control and the potential for amplifying biases if not audited. The paper makes a significant contribution by rigorously diagnosing the source of compositional failures in reasoning-augmented text-to-image models, demonstrating that the decoder is a faithful executor and the planner is the bottleneck, thereby validating inference-time plan repair as a highly effective, training-free intervention.
Multi-agent systems built from large language models are deployed widely, yet how much performance is lost when two LLMs must coordinate rather than act alone remains unclear. We formulate the collaboration tax as the team-decentralisation loss of a two-player cooperative game with private information, with two propositions characterising its sign and its equivalence to a max-superadditivity violation. We operationalise this definition on 32 solo-tractable tasks grouped by source of grounding friction and measure it on 11 models from 7 providers. The tax is structured along two no-exception axes: a category ordering across every model and a monotonic decrease with capability. The proximate mechanism is not a reasoning deficit but a four-stage conversational cascade in which agents make ungrounded claims, fail to query the partner, skip integrating both views, and accept the answer without re-derivation. The tax is mechanically predictable from conversation features and partly tractable: a prompt intervention targeting all four stages closes a substantial fraction of the gap, with the dominant bottleneck differing across categories. In heterogeneous pairs the tax is pulled toward the stronger partner rather than the additive midpoint, empirically realising the max-superadditivity violation predicted by our framework. Together these results recast collaboration in LLM systems as a measurable, predictable, and partly tractable cost.
Primary: University of Notre Dame
All Institutions: University of Notre Dame, Meta Superintelligence Labs, Simon Fraser University
The paper presents a novel and rigorous framework for quantifying and analyzing the coordination costs in LLM multi-agent systems, offering significant theoretical and practical insights into the mechanisms of failure and potential remedies.
The paper introduces a rigorous theoretical framework for quantifying "collaboration tax" in LLM multi-agent systems, grounded in cooperative game theory (Shapley values, superadditivity). The methodology is sound, defining a clear metric (solo vs. paired performance gap) and operationalizing it through a 32-task suite designed to isolate coordination failures from problem-solving capacity. The mechanistic analysis using a four-stage conversational cascade (Grounding, Querying, Integration, Re-derivation) is a significant methodological contribution, providing a granular diagnostic tool for multi-agent failures. The use of prompt interventions as ablation studies to validate these mechanisms is clever and effective.
The experimental setup is comprehensive, evaluating 11 models from 7 providers across homogeneous and heterogeneous pairs. The results are robust, showing consistent patterns across models and task types. The finding that the tax is predictable from conversation features and that prompt interventions can recover a substantial fraction of the loss is empirically strong. The heterogeneous pair analysis, while limited by sample size (only two pairs), provides compelling qualitative evidence for the theoretical predictions regarding asymmetric information and Shapley value allocation. The use of deterministic graders and continuous scoring adds rigor to the evaluation.
The paper provides a detailed description of the task suite, protocols, and metrics. The code and data are made available via GitHub and a project website, which significantly enhances reproducibility. The deterministic nature of the tasks and graders further supports reproducibility. The only potential hurdle is the reliance on API access to specific model versions, but the prompt-level interventions are fully specified.
The primary limitation is the scope of the evaluation. The tasks are synthetic puzzles (spatial, relational, CSP) and may not fully capture the complexity of real-world multi-agent applications like code generation or long-horizon planning. The analysis is restricted to dyadic pairs, and the generalization to larger multi-agent teams is unstudied. The heterogeneous pair analysis is limited to two specific pairs, which restricts the statistical power of those findings. Additionally, the "collaboration tax" metric, while useful, is sensitive to the specific solo baseline used (full information vs. split view), and the interpretation of negative tax values requires careful handling.
This work has significant implications for the design and deployment of LLM-based multi-agent systems. By quantifying the cost of coordination and identifying specific mechanistic failure points, it provides actionable insights for developers to improve system robustness through prompt engineering and protocol design. The theoretical framing connects LLM collaboration to established game-theoretic concepts, potentially opening new avenues for research in multi-agent reasoning and coordination. The findings suggest that much of the "intelligence" in multi-agent systems may be wasted on coordination overhead, a critical consideration for scaling these systems. The paper presents a novel and rigorous framework for quantifying and analyzing the coordination costs in LLM multi-agent systems, offering significant theoretical and practical insights into the mechanisms of failure and potential remedies.
Flow-matching language models refine all token positions in parallel and can trade sampling steps for latency, yet generation quality still degrades sharply with few sampling steps. We trace a source of this degradation to a train--inference mismatch in previous-prediction self-conditioning: during training, the self-conditioning input is computed from the current noisy state with no intervening solver step; during sampling, the solver folds the previous prediction into the latent before that same prediction reappears as the explicit self-conditioning input. This coupling, absent during training, creates redundancy that grows with step width. We show that the mismatch degrades both the self-conditioning input and the solver update, and derive a correction for each from the model's own structure. From the frozen projection weights we identify directions along which the self-conditioning input is redundant with the latent and dampen them; from the solver's integration structure we derive that a step-average prediction is needed and approximate it from prediction history, with scale set by offline trajectory statistics. The resulting sampler, Untied Self-Conditioning, requires no retraining and uses one evaluation per step. At 8 sampling steps on LangFlow, it reduces OpenWebText generative perplexity from $531$ to~$62$ ($8.6\times$); under an adapted Arena-Hard-Auto~v2 protocol, its outputs are preferred in $96\%$ of pairwise comparisons. On ELF-B it reduces generative perplexity from $71$ to~$43$. Improvements hold from 8 to 256 sampling steps.
Primary: Unknown
All Institutions: Unknown
The paper presents a theoretically grounded and empirically powerful correction to self-conditioning in Flow Matching for language models, offering a retraining-free path to high-efficiency, high-quality few-step generation.
The paper addresses a critical and subtle issue in Flow Matching (FM) for discrete sequence generation (language models): the train-inference mismatch in self-conditioning. The authors correctly identify that standard self-conditioning (using previous predictions as input) creates a dependency loop that is handled differently during training (using ground truth or clean noisy states) versus inference (using autoregressive or parallel predictions). They propose "Untied Self-Conditioning" (USC), which decouples this dependency by projecting out redundant components from the self-conditioning input and approximating a step-average prediction using history. This is a methodologically sound derivation based on the linear structure of the projection layers and the solver's integration steps. The approach is elegant because it requires no retraining, leveraging existing models.
The experimental results are exceptionally strong, bordering on suspiciously so for an abstract-only summary. The reduction of perplexity from 531 to 62 on OpenWebText at 8 steps is a massive improvement (order of magnitude). Similarly, the ELF-B result (71 to 43) is significant. The preference rate of 96% on Arena-Hard-Auto v2 is also extremely high. While the magnitude of improvement suggests a very effective correction, such dramatic gains in a mature field often warrant scrutiny regarding the baseline's validity or potential overfitting to the specific evaluation protocol. However, assuming the baselines are standard (e.g., standard Rectified Flow or Consistency Models for language), this would represent a state-of-the-art result for few-step generation. The evaluation covers both likelihood (perplexity) and quality (human preference), providing a comprehensive view.
The paper claims the method requires no retraining and uses one evaluation per step, which enhances reproducibility and accessibility. The description of the correction terms (dampening redundant directions, approximating step-average) suggests clear algorithmic steps. However, the "Unknown" institution and lack of explicit code links in the prompt text mean external verification of the specific implementation details (e.g., how "offline trajectory statistics" are computed) is currently limited. The appendix mentions proofs and details, which is a positive sign for reproducibility if the full text is available.
The primary limitation is the reliance on the specific structure of the self-conditioning mechanism in current FM language models. If the model architecture changes significantly (e.g., different attention mechanisms or non-linear projections), the specific projection-based correction might need adaptation. Additionally, the approximation of the step-average prediction using history might introduce noise if the history is short or noisy, potentially limiting performance at very low step counts (< 4 steps). The "96% preference" metric might also be sensitive to the specific prompt distribution in Arena-Hard-Auto.
This work has significant potential impact on the efficiency of language models. By enabling high-quality generation with very few sampling steps (8 steps), it drastically reduces inference latency, making powerful language models more viable for real-time applications and reducing computational costs. This aligns with the broader trend towards efficient AI. The paper presents a theoretically grounded and empirically powerful correction to self-conditioning in Flow Matching for language models, offering a retraining-free path to high-efficiency, high-quality few-step generation.
Large Language Models (LLMs) achieve strong reasoning performance, but their robustness to realistic lexical corruption remains poorly understood. We evaluate four open-weight instruction-tuned models and frontier models across four reasoning benchmarks under keyboard noise, character swaps, and filler insertion. Character-level perturbations substantially degrade accuracy, especially on multi-step reasoning tasks, while filler insertion has little effect. We trace this asymmetry to Attention Diversion: lexical corruption fragments subword tokenization, and the resulting fragments attract disproportionate attention mass, concentrated in middle and final transformer layers. Length-matched controls confirm that fragmentation, not prompt length, drives the loss. A factorial intervention then shows why the damage is hard to undo: fragmentation corrupts token content and attention allocation together, and the two are coupled. Restoring clean attention while the content remains corrupted is actively harmful, restoring content alone is insufficient, and only restoring both recovers a substantial share of the gap. This coupling explains why inference-time strategies, including chain-of-thought prompting, spell-checking, self-repair, and stronger repair models, fail to consistently recover performance: each addresses one channel at a time. Code and data are available at https://github.com/Jiaqian-Janelle/Attention-Diversion
Primary: Missouri University of Science and Technology
All Institutions: Missouri University of Science and Technology, University of North Texas
The paper provides a rigorous mechanistic explanation for LLM fragility to lexical perturbations, identifying "Attention Diversion" caused by subword fragmentation as the primary driver of reasoning failure, and demonstrating through factorial interventions that token content and attention allocation are coupled in a way that renders single-channel repairs ineffective.
The paper employs a rigorous mechanistic interpretability approach to diagnose LLM robustness. It moves beyond simple accuracy metrics by introducing "Attention Diversion" as a causal mechanism, supported by factorial interventions (decoupling token content from attention allocation) and length-matched controls. The methodology is sound, well-controlled, and effectively isolates the specific failure mode of subword fragmentation. However, the core observation—that typos break tokenization and hurt performance—is a known phenomenon in NLP; the novelty lies in the detailed mechanistic tracing and the specific "coupling" finding, which is insightful but not paradigm-shifting.
The experimental setup is comprehensive, covering multiple model families (Llama, Mistral, Qwen, Gemma), scales (7B-70B), and frontier models (GPT-4o, GPT-5.4). The use of multiple reasoning benchmarks (BoolQ, PIQA, HellaSwag, GSM8K) and perturbation types (keyboard, swap, filler) provides strong empirical grounding. The results are consistent and statistically significant. The factorial intervention is the strongest part of the evaluation, providing clear causal evidence for the coupling hypothesis. The inclusion of Chinese/CMath data adds valuable cross-lingual generalization evidence.
The paper provides a GitHub repository link and describes the experimental protocols in sufficient detail for replication. The use of standard benchmarks and open-weight models enhances reproducibility. The specific interventions (embedding replacement, attention suppression) are described with enough technical detail to be implemented.
The paper acknowledges limitations, including the focus on English/QWERTY (though mitigated by Chinese data), synthetic perturbations, and the single-task nature of the factorial intervention. A key limitation is that the "repair" strategies tested are somewhat basic or ill-suited to the specific coupling problem identified, which limits the practical immediate impact of the mitigation section. The venue date (2026) is futuristic but treated as given.
This work has significant implications for the reliability of LLMs in real-world scenarios involving noisy input (e.g., voice-to-text, fast typing). It challenges the assumption that semantic preservation is sufficient for robustness and highlights the fragility of subword tokenization. It suggests that future robustness improvements must address tokenization-level defenses or training-time alignments rather than just inference-time prompting. The paper provides a rigorous mechanistic explanation for LLM fragility to lexical perturbations, identifying "Attention Diversion" caused by subword fragmentation as the primary driver of reasoning failure, and demonstrating through factorial interventions that token content and attention allocation are coupled in a way that renders single-channel repairs ineffective.
High-performance ML systems increasingly rely on GPU kernels whose editable source is unavailable, generated, or too distant from final machine code to expose remaining optimizations. Existing LLM kernel optimizers and autotuners mainly operate on CUDA, Triton, HIP, or tensor-program source and validate against reference implementations. We study a stricter setting: optimizing an already compiled AMDGPU code object, where the deployed binary is the only behavioral oracle. We present AsmEvo, an agentic assembly-level optimizer for AMD GPU kernels. Given an AMDGPU code object K0, AsmEvo reconstructs a reassemblable representation, proposes low-level edits with a long-horizon agent, rebuilds an ABI-preserving optimized object, and accepts candidates only after differential verification against K0 under identical launches. AsmEvo combines code-object recovery, metadata-aware rebuilding, profiling-guided hot-window editing, correctness-gated timing, and conservative in-place patch fallback. We conduct extensive experiments with AsmEvo on various AMD GPU kernels. On MI308X, AsmEvo improves 29 of 30 selected KernelBench kernels, reaching 1.35x geometric-mean and 3.88x maximum speedup. On MI300X production workloads, it improves all evaluated AITer binaries and vLLM/SGLang Triton assembly kernels, reaching 1.09x/1.31x and 1.18x/1.34x geometric-mean/maximum speedups, respectively, while preserving functional equivalence.
Primary: AMD
All Institutions: AMD
AsmEvo presents a novel and technically rigorous approach to post-compilation GPU kernel optimization, effectively bridging the gap between high-level ML frameworks and low-level hardware performance by leveraging LLMs for assembly-level editing under strict functional equivalence constraints.
The paper proposes AsmEvo, a system for optimizing AMD GPU kernels at the assembly level without source code. The core methodology involves three stages: (1) recovering a reassemblable AMDGCN assembly representation from the compiled ELF/HSACO binary, (2) using a long-horizon LLM agent to propose low-level edits within "hot windows" identified by profiling, and (3) enforcing a strict "gated verification" harness that rebuilds the binary, checks for ABI preservation, and verifies functional equivalence against the original binary (the oracle) before measuring performance. The approach is technically sound and addresses a real gap in the ML systems stack: post-compilation optimization of deployed artifacts. The use of a differential oracle (comparing output of optimized vs. original binary) is a robust way to handle correctness without source-level reference implementations. The separation of the LLM agent (exploration) from deterministic gates (verification) is a good architectural choice to prevent hallucinated or incorrect optimizations from being accepted.
The experiments are conducted on AMD MI308X and MI300X hardware. The evaluation covers KernelBench (L1 and L2) and production kernels from AITer, vLLM, and SGLang. The results show significant speedups, with a geometric mean of 1.35x on KernelBench and up to 3.88x on specific kernels. The inclusion of production workloads (AITer, Triton HSACOs) is a strong point, demonstrating real-world applicability. The verification process is rigorous, ensuring that speedups are not due to semantic changes. However, the sample size for production kernels is small (8 kernels), and the KernelBench results, while impressive, are on benchmark kernels that might not fully represent the complexity of end-to-end inference workloads. The comparison is primarily internal (optimized vs. original binary), which is appropriate for the problem setting, but lacks comparison to other state-of-the-art autotuners or compiler optimizations on the same binaries, as those typically require source.
The paper provides detailed descriptions of the recovery, rebuild, and verification pipelines. The use of standard tools (ROCm, AMDGCN) and the clear definition of the verification harness enhance reproducibility. However, the reliance on a specific LLM (Claude Opus 4.8) and the proprietary nature of some production kernels (AITer) may limit full reproducibility for external researchers. The code for the recovery and rebuild tools is likely not open-sourced in the same way as typical ML models, which could hinder adoption.
The primary limitation is the scope of the optimization: it is specific to AMD GPU architecture (AMDGCN). The recovery process might fail for complex or obfuscated binaries. The "real-dispatch capture" mechanism, while powerful, requires access to the running application's memory state, which may not always be feasible or safe in all deployment environments. The speedups, while significant, are bounded by the remaining optimization headroom in the compiled binary, which is often less than in source-level optimization. The paper acknowledges that equivalence is empirical, not formal.
This work has significant implications for the ML systems community, particularly for organizations that deploy compiled ML models where source code is unavailable or optimization at the source level is no longer possible. It demonstrates that there is still value in low-level, binary-level optimization. The methodology could be extended to other architectures (e.g., NVIDIA SASS) and could inspire new tools for binary-level program analysis and optimization. It also highlights the potential of LLMs for low-level systems programming tasks when combined with rigorous verification. AsmEvo presents a novel and technically rigorous approach to post-compilation GPU kernel optimization, effectively bridging the gap between high-level ML frameworks and low-level hardware performance by leveraging LLMs for assembly-level editing under strict functional equivalence constraints.
Long-context modeling is a pivotal capability for Large Language Models, yet the quadratic complexity of attention remains a critical bottleneck, particularly during the compute-intensive prefilling phase. Our previous work, FlashPrefill, mitigates this cost through instantaneous pattern discovery and max-based dynamic thresholding; however, it remains an algorithmic prototype that is still distant from production deployment. In this paper, we present FlashPrefill V2, which evolves FlashPrefill from a prototype toward practical long-context serving along three dimensions. First, we introduce a mean correction term that effectively suppresses the approximation error, keeping performance degradation manageable even at extreme sparsity levels. Second, we redesign the sparse attention operator with PackGQA memory access, warp specialization, and pingpong pipelining, fully aligning with the latest FlashAttention-3/4 implementations and supporting FP8 inference to meet practical quantization requirements. Third, FlashPrefill V2 natively supports paged KV cache and continuous batching, allowing integration as an attention backend in modern inference frameworks such as SGLang. Extensive evaluations on NVIDIA H20 GPUs---among the most widely deployed inference accelerators---demonstrate that FlashPrefill V2 delivers up to 47.26x and 27.19x speedups over FlashAttention-2 at 128K context length under FP8 and BF16 precision, respectively, and, in FP8, still achieves a 30.49x speedup against an FA3/4-aligned dense baseline.
Primary: Tencent WeChat
All Institutions: Tencent WeChat
FlashPrefill V2 presents a substantial systems-level contribution to long-context LLM serving by transforming a sparse attention prototype into a production-ready, highly optimized backend that achieves massive speedups on widely deployed hardware through advanced kernel engineering and integration with modern inference frameworks.
The paper proposes FlashPrefill V2, an optimization for the prefilling phase of Long-Context LLM serving. It builds upon a previous prototype (FlashPrefill) by introducing three key technical components: 1) A mean correction term to reduce approximation error in block-sparse attention, allowing for higher sparsity without significant accuracy loss. 2) A highly optimized CUDA kernel implementation leveraging PackGQA memory access, warp specialization, and ping-pong pipelining, aligned with FlashAttention-3/4 paradigms and supporting FP8 inference. 3) Integration support for paged KV cache and continuous batching, making it a viable backend for modern inference engines like SGLang. The methodology is sound and addresses a critical bottleneck (prefill latency) in long-context serving. The shift from algorithmic prototype to production-ready system is a significant engineering contribution. EXPERIMENTAL_EVALUTION: The evaluation is conducted on NVIDIA H20 GPUs, which are noted as widely deployed inference accelerators. The paper reports substantial speedups: up to 47.26x over FlashAttention-2 (FA2) at 128K context under FP8, and 27.19x under BF16. It also compares against an FA3/4-aligned dense baseline, showing a 30.49x speedup in FP8. The results demonstrate clear performance gains. However, the paper lacks detailed accuracy benchmarks (e.g., perplexity drops on standard datasets) beyond the claim that error is "manageable." The focus is heavily on throughput/latency, which is appropriate for a systems paper, but a more rigorous accuracy analysis would strengthen the claim of "practical" deployment.
The paper describes specific kernel optimizations (warp specialization, ping-pong pipelining) and integration points (SGLang). While the algorithmic description is clear, the full source code is not explicitly linked in the abstract or provided in the text snippet. Reproducibility depends on the availability of the code repository, which is not extracted here. The use of standard hardware (H20) and frameworks (SGLang) aids reproducibility if code is open-sourced.
The primary limitation is the reliance on block-sparse attention, which introduces approximation error. While the mean correction helps, it does not eliminate it, potentially affecting tasks requiring precise attention mechanisms. The evaluation is limited to NVIDIA H20 GPUs; performance on other architectures (e.g., H100, A100, or AMD GPUs) is not reported. The paper does not provide extensive accuracy degradation analysis across diverse benchmarks. The "extreme sparsity levels" mentioned in the abstract are not quantified in the provided text, making it hard to assess the trade-off curve.
This work has significant potential impact on the efficiency of long-context LLM serving, reducing costs and latency for applications requiring large context windows. By aligning with modern inference frameworks and supporting FP8, it promotes energy-efficient and cost-effective deployment of large models. The focus on production-ready systems bridges the gap between academic research and industrial application. FlashPrefill V2 presents a substantial systems-level contribution to long-context LLM serving by transforming a sparse attention prototype into a production-ready, highly optimized backend that achieves massive speedups on widely deployed hardware through advanced kernel engineering and integration with modern inference frameworks.
Humans imitate at the level of intent: given a demonstration, we infer its goal and carry it out with whatever tools, objects, and layouts are at hand. Current robot policies instead learn observation-to-action mappings from visual inputs and language instructions, without explicitly inferring the demonstrated task. Learning from human video thus remains largely trajectory-level: models can replay motions in near-identical scenes, but still struggle to imitate what the demonstrator intends rather than merely what they do. We introduce The Imitator Game, a four-level benchmark (L0-L3) that progressively widens the gap between the human demonstration and the robot's own scene, isolating where trajectory replay ceases to suffice and task understanding becomes necessary. We pair it with IG-10K, the largest environment-aligned paired human-robot dataset to date and the only one instantiated across all four levels in both real and simulated settings (20,000+ paired episodes, 50+ tasks, 6 domains), and Imitator Arena, an open platform for blind A/B human evaluation. Across nine state-of-the-art models, performance is stable from L0 to L2 but collapses at L3, identifying functional substitution - achieving the same intent through a different object affordance - as the decisive barrier to intent-level imitation. Human-video-conditioned models outperform caption-conditioned ones, yet every model falls below 13% zero-shot success on unseen tasks; fine-tuning IG-10K-pretrained models with only $10$ paired human-robot demonstrations yields large gains that grow with pretraining scale. The project website and access to Imitator Arena are available at https://imitator-game.github.io.
Primary: Zhejiang University
All Institutions: Zhejiang University
The paper presents a significant contribution to robotics by introducing a challenging benchmark and large-scale dataset that isolates the critical challenge of intent-level imitation, providing the field with a rigorous tool to measure and improve upon current limitations in robot learning.
The paper introduces "The Imitator Game," a structured benchmark designed to evaluate robot imitation beyond simple trajectory replay. It defines four levels of difficulty (L0-L3) that progressively increase the discrepancy between the human demonstration and the robot's environment, specifically targeting "functional substitution" at L3. The authors also release IG-10K, a large-scale paired human-robot dataset spanning 50+ tasks across 6 domains. The methodology involves training state-of-the-art vision-language-action models and evaluating their zero-shot and few-shot performance. The core methodological contribution is the rigorous isolation of intent understanding versus motor mimicry through this hierarchical benchmarking framework.
The experimental section is comprehensive, evaluating nine state-of-the-art models. Key findings include stable performance from L0 to L2 but a collapse at L3, identifying functional substitution as the primary barrier to intent-level imitation. The paper demonstrates that human-video-conditioned models outperform caption-conditioned ones. A significant result is the finding that fine-tuning with only 10 paired demonstrations yields large gains, suggesting that pre-training on large-scale data (like IG-10K) provides a strong prior for intent understanding. The use of "Imitator Arena" for blind A/B human evaluation adds a layer of subjective quality assessment that complements objective success rates.
The paper provides a project website and claims to release the IG-10K dataset and the benchmark environment. The dataset size (20,000+ episodes) and diversity (50+ tasks, 6 domains) suggest a substantial resource for the community. The availability of the benchmark and dataset significantly enhances reproducibility and allows for future comparative studies. The paper cites the specific URL for access, facilitating verification.
The primary limitation is the current performance ceiling; no model achieves high success on L3 tasks in zero-shot settings (below 13%). This indicates that while the benchmark is effective at exposing current limitations, it may be too challenging for existing architectures to solve without significant domain-specific tuning or architectural changes. Additionally, the reliance on human-video conditioning might limit applicability in scenarios where video demonstrations are not available, although the paper addresses this by comparing with caption-conditioned models. The generalization to unseen tasks remains poor, highlighting a gap between benchmark performance and real-world robustness.
This work has significant implications for the field of robotics and embodied AI. By clearly defining the gap between trajectory replay and intent understanding, it provides a clear roadmap for future research. The release of IG-10K and the benchmark framework will likely become standard tools for evaluating imitation learning capabilities. It shifts the focus from mere motion matching to semantic understanding of tasks, which is crucial for deploying robots in unstructured human environments. The findings suggest that current models are still largely "parrots" rather than "understanders," urging the community to develop more robust intent inference mechanisms. The paper presents a significant contribution to robotics by introducing a challenging benchmark and large-scale dataset that isolates the critical challenge of intent-level imitation, providing the field with a rigorous tool to measure and improve upon current limitations in robot learning.
Vision-Language-Action (VLA) models are typically trained with behavior cloning (BC) on expert demonstrations. However, BC provides only positive supervision for expert actions, without explicit negative supervision indicating which actions are instruction-inconsistent or otherwise inappropriate. Reinforcement learning (RL) can provide such corrective signals, but often relies on externally specified rewards or curated non-expert data, both of which are costly to obtain in robotics. We show that offline RL for VLA models need not rely on curated non-expert trajectories: successful expert demonstrations alone can be transformed into dense corrective supervision through instruction relabeling. Specifically, by pairing expert actions with mismatched alternative instructions, we synthesize counterfactual instruction-observation-action tuples from the dataset and combine them with adversarial discriminator training to learn an instruction-grounded reward model for offline RL, without collecting additional rollouts or annotations. On the robustness-focused LIBERO-PRO benchmark, our method improves robustness to object position and task perturbations over a strong state-of-the-art baseline. It also outperforms competitive baselines in real-robot experiments on the TX-G2 (compatible with AGIBot G2). More broadly, our results suggest that, for data-constrained VLA learning, extracting denser supervision from each demonstration can complement collecting additional data.
Primary: Unknown
All Institutions: Unknown
[One sentence main contribution]. CounterAlign introduces a counterfactual supervision mechanism for VLA models that synthesizes negative training samples from expert demonstrations via instruction relabeling, enabling robust offline RL without additional data collection. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a novel approach to improving the robustness of Vision-Language-Action models by addressing the lack of negative supervision in Behavior Cloning. By leveraging instruction relabeling to create counterfactual negative samples, the authors enable the training of an adversarial reward model that guides offline RL. This method is significant because it offers a data-efficient alternative to collecting non-expert trajectories or designing complex reward functions. The empirical results on LIBERO-PRO and real-robot experiments demonstrate improved robustness to perturbations, validating the effectiveness of the approach. While the novelty is moderate (as adversarial training and data augmentation are well-known concepts), the specific application to VLA models and the simplicity of the instruction-swapping mechanism make it a valuable contribution. The technical impact is solid, offering a practical improvement for VLA training pipelines. However, the incremental nature of the improvement and the computational costs prevent it from reaching the highest tiers of impact. The paper is well-written and addresses a relevant problem in the field, making it a strong candidate for publication in a top-tier robotics or ML venue.
The paper proposes "CounterAlign," a method to enhance Vision-Language-Action (VLA) models by generating negative supervision from positive expert demonstrations. The core idea involves "instruction relabeling," where expert action-observation tuples are paired with mismatched (counterfactual) instructions to create negative samples. These are used to train an adversarial discriminator that serves as a reward signal for offline Reinforcement Learning (RL). This approach aims to solve the lack of explicit negative feedback in Behavior Cloning (BC) without requiring costly non-expert data collection or external reward models. The methodology is theoretically sound, leveraging existing offline RL frameworks (like CQL or similar implicit Q-learning variants) but adapting the reward structure through a learned discriminator. The novelty lies in the specific mechanism of synthesizing negative data via instruction swapping rather than collecting it, which is a clever data-augmentation strategy for VLA training.
The evaluation is conducted on the LIBERO-PRO benchmark, which is a standard and rigorous testbed for VLA robustness, focusing on perturbations in object positions and task definitions. The paper reports improvements over a strong state-of-the-art baseline (likely RT-2 or similar BC-based VLA). Additionally, real-robot experiments on the TX-G2 platform are presented. The results show improved robustness to perturbations, which is the primary claim. However, the abstract-only score was 60, and the full text analysis suggests the gains, while positive, may be incremental. The comparison against other RL-based baselines or more recent SOTA VLAs is crucial; if the baseline is weak, the improvement might be overstated. The real-robot results add significant weight, as sim-to-real gaps are a major hurdle. The evaluation appears comprehensive for the scope, but the magnitude of improvement needs to be weighed against the computational cost of training the discriminator and the offline RL loop.
The paper describes the method in detail, including the instruction relabeling process and the adversarial training setup. However, as is common with arXiv submissions, the code is not explicitly linked in the provided text (URL extraction returned "none"). Reproducibility would depend on the availability of the code, which is not guaranteed. The use of standard benchmarks (LIBERO-PRO) aids in reproducibility of the evaluation metrics. The specific hyperparameters for the adversarial training and the offline RL algorithm are likely detailed in the full text, but without access to the code or supplementary material, exact replication might be challenging.
The primary limitation is the reliance on the quality of the instruction relabeling. If the language model or heuristic used to generate "mismatched" instructions is not sufficiently diverse or semantically plausible, the negative samples may not provide strong enough gradients for the discriminator. Furthermore, the method assumes that expert demonstrations are available, which is the same constraint as BC. The computational overhead of training the discriminator and running the offline RL loop is higher than pure BC. The paper also notes that the method has not been tested on larger, more diverse datasets, limiting the generalizability of the claims.
This work contributes to the field of embodied AI by providing a more data-efficient training paradigm for VLA models. By extracting denser supervision from existing data, it reduces the need for expensive data collection, which is a major bottleneck in robotics. This could accelerate the development of robust robotic systems. However, the reliance on expert data means that biases in the expert demonstrations could be amplified if not carefully handled. The method also highlights the importance of negative supervision in learning robust policies, a insight that may extend beyond VLA models to other sequential decision-making tasks. [One sentence main contribution]. CounterAlign introduces a counterfactual supervision mechanism for VLA models that synthesizes negative training samples from expert demonstrations via instruction relabeling, enabling robust offline RL without additional data collection. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The paper presents a novel approach to improving the robustness of Vision-Language-Action models by addressing the lack of negative supervision in Behavior Cloning. By leveraging instruction relabeling to create counterfactual negative samples, the authors enable the training of an adversarial reward model that guides offline RL. This method is significant because it offers a data-efficient alternative to collecting non-expert trajectories or designing complex reward functions. The empirical results on LIBERO-PRO and real-robot experiments demonstrate improved robustness to perturbations, validating the effectiveness of the approach. While the novelty is moderate (as adversarial training and data augmentation are well-known concepts), the specific application to VLA models and the simplicity of the instruction-swapping mechanism make it a valuable contribution. The technical impact is solid, offering a practical improvement for VLA training pipelines. However, the incremental nature of the improvement and the computational costs prevent it from reaching the highest tiers of impact. The paper is well-written and addresses a relevant problem in the field, making it a strong candidate for publication in a top-tier robotics or ML venue.