Last 7 Days (July 04 – July 10, 2026)
Diffusion models have become a dominant paradigm for high-quality generative modeling, while post-training is essential for adapting them to diverse downstream applications. However, post-training of large diffusion models is still challenging due to the prohibitive memory footprints and slow training speed, which existing parameter-efficient fine-tuning methods only partially address. To overcome these limitations, we propose FourTune, an efficient post-training framework for diffusion models based on an end-to-end W4A4G4 paradigm. FourTune introduces a triple-branch hybrid pipeline that augments the standard LoRA architecture with a frozen numerical stabilizer to isolate quantization-sensitive outliers, enabling stable training under native 4-bit computation. In addition, FourTune employs hardware-efficient block-wise quantization and customized fused kernels to support efficient quantized backpropagation and reduce memory bandwidth overhead. Across customization, reinforcement learning, and distillation tasks, FourTune matches the quality of full-precision fine-tuning. On FLUX.1-dev (12B), FourTune reduces memory overhead by 2.25$\times$ and increases end-to-end training throughput by 2.27$\times$ compared to BF16 LoRA.
Primary: Stanford University
All Institutions: Stanford University, MIT, Nunchux AI, UC Berkeley, CMU
FourTune has a significant broader impact on the field of machine learning, particularly for generative AI. By enabling efficient 4-bit post-training of large diffusion models without compromising quality, it democratizes access to these powerful models. Researchers and practitioners with limited computational resources can now fine-tune large models faster and with less memory, accelerating research, development, and deployment of customized generative AI applications. This could lead to a proliferation of new use cases for diffusion models in various domains, from content creation to scientific discovery. The techniques developed, especially the triple-branch hybrid pipeline with the frozen numerical stabilizer, could inspire similar efficient training strategies for other large foundation models beyond diffusion models, such as large language models. The paper also includes a standard ethical impact statement, acknowledging both positive and negative societal implications of generative AI. FourTune presents a highly impactful method for fully 4-bit efficient post-training of diffusion models, achieving state-of-the-art efficiency while maintaining full-precision quality through a novel triple-branch hybrid pipeline and hardware-optimized quantization. This work significantly advances the accessibility and practical applicability of large diffusion models, enabling faster iteration and broader adoption in diverse downstream tasks.
FourTune proposes an innovative and technically sound approach for fully 4-bit efficient post-training of diffusion models, addressing the critical challenges of memory footprint and training speed. The core of the methodology is the "triple-branch hybrid pipeline" which augments a standard LoRA architecture with a "frozen numerical stabilizer" (FNS). This FNS branch is a small, frozen, full-precision component designed to isolate and process quantization-sensitive outliers, thereby enabling stable training under native W4A4G4 (weights, activations, gradients) computation. This is a particularly clever design choice, as it mitigates the primary instability issue of low-bit quantization (outliers) without incurring significant computational overhead during training, as the FNS branch remains frozen. The integration of this FNS with both a standard BF16 LoRA branch and a quantized 4-bit LoRA branch provides a robust mechanism for maintaining quality while maximizing efficiency. Furthermore, the paper introduces hardware-efficient block-wise quantization and customized fused kernels. These components are crucial for translating the theoretical benefits of 4-bit quantization into practical speedups by optimizing memory bandwidth and accelerating quantized backpropagation operations (e.g., QGEMM, QMatmul, QConv). The end-to-end W4A4G4 paradigm, encompassing weights, activations, and gradients, is a comprehensive solution that pushes the boundaries of efficient fine-tuning for large generative models.
The experimental evaluation is comprehensive and robust, demonstrating FourTune's effectiveness across diverse and important post-training tasks for diffusion models: customization (DreamBooth-like), reinforcement learning for aesthetic preference, and knowledge distillation. The choice of FLUX.1-dev (12B parameters) as the primary model, along with validation on SDXL (1.5B), ensures the results are relevant for large-scale, state-of-the-art diffusion models. Baselines include BF16 LoRA and NF4 QLoRA, which are appropriate and strong competitors. The quantitative results are highly compelling: FourTune consistently matches or slightly outperforms the image quality of full-precision BF16 LoRA and NF4 QLoRA across all tasks, as measured by FID, CLIP Score, and LPIPS. This quality preservation at such low bit-depth is a significant achievement. More importantly, FourTune delivers substantial efficiency gains, reducing GPU memory overhead by 2.25x compared to BF16 LoRA (achieving a footprint comparable to NF4 QLoRA) and increasing end-to-end training throughput by 2.27x over BF16 LoRA and 2.79x over NF4 QLoRA. This effectively breaks the memory-speed trade-off often encountered in large model post-training. The ablation studies are well-designed and clearly demonstrate the critical contribution of each proposed component: the Frozen Numerical Stabilizer for stability and quality, and the W4A4G4 quantization, block-wise quantization, and fused kernels for efficiency. The generalization to SDXL further strengthens the claims.
The paper provides a good level of detail regarding the methodology, including the architecture of the triple-branch pipeline, the role of the FNS, and the types of quantization and fused kernels used. The experimental setup is well-described, including the specific models (FLUX.1-dev, SDXL), tasks, baselines, and evaluation metrics. The appendix further elaborates on implementation details, hyperparameters, and training configurations, which are crucial for reproducibility. While no direct code link is provided, the comprehensive description suggests that a skilled research team should be able to reproduce the results.
The paper does not explicitly discuss limitations, but some can be inferred. While the FNS effectively handles outliers, its design might introduce a slight architectural overhead compared to a purely 4-bit system, even if frozen. The customized fused kernels, while highly efficient, might require specific hardware support or careful implementation to achieve optimal performance across different GPU architectures. The evaluation is primarily focused on image generation diffusion models; its applicability and performance on other types of diffusion models (e.g., audio, video) or other generative architectures (e.g., GANs, VAEs) are not explored. The paper also doesn't delve into the potential challenges of deploying such a highly optimized, custom-kernel-dependent solution in diverse production environments.
FourTune has a significant broader impact on the field of machine learning, particularly for generative AI. By enabling efficient 4-bit post-training of large diffusion models without compromising quality, it democratizes access to these powerful models. Researchers and practitioners with limited computational resources can now fine-tune large models faster and with less memory, accelerating research, development, and deployment of customized generative AI applications. This could lead to a proliferation of new use cases for diffusion models in various domains, from content creation to scientific discovery. The techniques developed, especially the triple-branch hybrid pipeline with the frozen numerical stabilizer, could inspire similar efficient training strategies for other large foundation models beyond diffusion models, such as large language models. The paper also includes a standard ethical impact statement, acknowledging both positive and negative societal implications of generative AI. FourTune presents a highly impactful method for fully 4-bit efficient post-training of diffusion models, achieving state-of-the-art efficiency while maintaining full-precision quality through a novel triple-branch hybrid pipeline and hardware-optimized quantization. This work significantly advances the accessibility and practical applicability of large diffusion models, enabling faster iteration and broader adoption in diverse downstream tasks.
Medicine is inherently multimodal, requiring clinicians to synthesize information across diverse data streams. Yet the development of multimodal foundation models is constrained by limited access to large-scale, high-quality clinical data. Although PubMed Central (PMC) offers a complementary source of expert-authored image-text data, existing PMC-derived resources remain limited in fidelity, reproducibility, and clinical validation. We introduce MedPMC, an automated, continuously updatable framework that transforms permissively licensed literature into high-fidelity infrastructure for medical multimodal models. Applied to 6.1 million PMC articles, MedPMC curated 11 million medical image-text pairs. Component evaluations showed strong performance for initial screening (F1 = 93.2), multi-panel figure detection (F1 = 96.5), figure separation (mAP = 89.8), caption separation and alignment (F1 = 81.4; ROUGE-L = 85.3), and medical figure classification (F1 = 96.5). Manual review by five annotators, three with medical training, found 95.3% of MedPMC images medically relevant, versus 19.7% in a prior PMC-derived dataset. Across 26 benchmarks spanning 11 specialties, a MedPMC-trained CLIP-style model improved average zero-shot AUC by 7.1 percentage points over the strongest architecture-matched biomedical CLIP baseline despite using fewer than half as many image-text pairs. As the vision encoder in a multimodal large language model, it improved medical visual question-answering by 1.9 and 16.9 percentage points across two benchmarks. In 10,524 Yale New Haven Health System dermatology photographs, it improved morphology-to-image retrieval Recall@5 by 11.7 percentage points. These findings show that high-fidelity literature curation strengthens medical multimodal foundation models across benchmark and clinical settings. We publicly release the framework, corpus, benchmarks, and pretrained models.
Primary: Yale University
All Institutions: Yale University, University of Illinois Urbana-Champaign, Microsoft Research, Amazon
Yale University's MedPMC framework represents a landmark contribution to medical multimodal learning by systematically curating a massive, high-fidelity dataset from open-access literature, demonstrating that data quality is as critical as quantity for training effective foundation models, and providing a reproducible pipeline and resources that will serve as a new standard for the field.
The paper introduces MedPMC, a systematic, automated pipeline for curating high-fidelity image-text pairs from PubMed Central (PMC). The methodology addresses a critical bottleneck in medical multimodal learning: the scarcity of large-scale, high-quality, and clinically relevant data. The pipeline involves several sophisticated components: (1) filtering 6.1 million articles for permissive licenses; (2) detecting multi-panel figures; (3) separating individual figures from panels; (4) aligning figures with their corresponding captions using natural language processing techniques; and (5) classifying figures for medical relevance. The authors report high performance on these component tasks (e.g., F1=93.2 for screening, mAP=89.8 for figure separation), indicating a robust and well-engineered curation process. The novelty lies not in a single algorithmic breakthrough but in the systematic integration and scaling of these components to create a massive, high-quality dataset, coupled with rigorous validation against prior, lower-fidelity datasets.
The evaluation is comprehensive and compelling. First, the authors validate the quality of the curated data through manual review by five annotators (three with medical training), showing a significant improvement in medical relevance (95.3% vs. 19.7% in a prior dataset). Second, they train a CLIP-style model on the MedPMC corpus and evaluate it on 26 benchmarks across 11 medical specialties. The MedPMC-trained model outperforms the strongest architecture-matched biomedical CLIP baseline by 7.1 percentage points in average zero-shot AUC, despite using fewer than half the image-text pairs. This demonstrates the high signal-to-noise ratio of the MedPMC data. Third, they integrate the vision encoder into a multimodal large language model (MLLM) and show improvements in medical visual question-answering. Finally, they demonstrate clinical utility by improving morphology-to-image retrieval on a real-world dermatology dataset from Yale New Haven Health System. The results are statistically significant and practically meaningful.
The authors provide extensive resources for reproducibility. The code for the curation pipeline is publicly available on GitHub. The MedPMC corpus, component-level benchmark resources, pretrained checkpoints, and metadata are released on Hugging Face. The data release includes versioning by article cutoff date, source-license filters, and processing configuration, allowing users to reproduce specific snapshots. They also provide clear instructions for license-aware filtering. The only non-public component is the clinical dermatology data, which is subject to privacy restrictions, but aggregate results and processing procedures are described. This level of transparency is exemplary.
The primary limitation is the reliance on open-access literature, which may introduce selection bias towards certain types of studies or institutions. The pipeline's performance, while high, is not perfect (e.g., F1=81.4 for caption separation), meaning some noise remains in the dataset. The clinical evaluation is limited to dermatology retrieval, and while promising, it does not cover the full breadth of medical specialties. The authors acknowledge that some records are derived from articles with non-commercial licenses (CC BY-NC), which restricts commercial use of the dataset. Additionally, the study focuses on image-text pairs; the extension to other modalities (e.g., audio, video, time-series) is not addressed.
The MedPMC framework has significant potential to accelerate the development of medical multimodal foundation models. By providing a large-scale, high-quality, and openly accessible dataset, it lowers the barrier to entry for researchers and clinicians working in this domain. The improved performance of models trained on MedPMC suggests that high-fidelity data curation is a key lever for improving model capabilities. The release of the code and benchmarks encourages further research in data curation and multimodal learning. However, the non-commercial license restrictions for some data may limit its impact in commercial healthcare applications. The authors have responsibly addressed privacy concerns by not releasing patient data and by providing clear licensing information. Yale University's MedPMC framework represents a landmark contribution to medical multimodal learning by systematically curating a massive, high-fidelity dataset from open-access literature, demonstrating that data quality is as critical as quantity for training effective foundation models, and providing a reproducible pipeline and resources that will serve as a new standard for the field.
Does RL post-training merely amplify primitive skills already latent in a base model, or can it compose primitive skills into new higher-level strategies? We study this question in a fully observable rewrite-grammar environment where the pretraining distribution is known and every generated rewrite can be audited. A Transformer is pretrained on primitive symbol-rewrite chains and post-trained on a Trace-based reasoning task with only a binary final-answer reward. RL solves held-out problems that remain rarely solved by the pretrained model even under much larger sampling budgets, while rejection fine-tuning improves early but plateaus. Trace analysis shows that RL reorganizes primitive competence through a phased compositional mechanism: it first strengthens primitive reductions, then discovers valid composed procedures. These include sequential compositions, which collapse ordered chains of primitive contractions, and parallel compositions, which combine independent primitive contractions in a single step. The composed procedures are not isolated samples; they are reused and consolidated into a stable repertoire. Comparing RL with rejection fine-tuning shows that the key difference is not exploration volume but selectivity: RFT produces many shortcut-like rewrites, much of them invalid, whereas RL concentrates exploration into valid reusable structure. Pretraining ablations show that the emergence of compositional strategies is gated not by primitive exposure alone, but by whether pretraining organizes primitive competence into reduction procedures that RL can later compress. The base model provides weak procedural ingredients; RL builds them into reliable higher-level strategies.
Primary: UCL
All Institutions: UCL
This paper makes a significant contribution to the ongoing debate about whether RL post-training merely reweights existing behaviors or enables new capabilities in base models. By providing a concrete, inspectable mechanism for "going beyond the base model," it offers valuable insights for understanding and improving LLM reasoning. The findings on procedural chunking and the phased emergence of compositional strategies have implications for skill acquisition research in AI. The distinction between exploration volume and selectivity, and the role of negative samples in focusing exploration on valid structures, is crucial for designing more effective RL algorithms for reasoning tasks. Furthermore, the analysis of how pretraining organizes primitive competence into a "procedural substrate" for RL highlights the importance of co-designing pretraining and post-training objectives. While the environment is synthetic, the mechanistic understanding gained here can inform hypotheses and experimental designs for more complex, real-world LLM applications. RL Post-Training Builds Compositional Reasoning Strategies provides a rigorous, mechanistic account of how reinforcement learning can enable compositional reasoning beyond the capabilities of a pretrained base model in a fully observable rewrite-grammar environment. The paper's strength lies in its novel controlled experimental setup, which allows for precise auditing of reasoning traces, revealing a phased emergence of valid compositional strategies, the critical role of RL's selectivity over exploration volume, and the importance of pretraining in organizing primitive competence into a compressible substrate for higher-level strategy formation.
The methodology is exceptionally strong due to its controlled and fully observable nature. The authors introduce a novel rewrite-grammar environment where primitive rules, pretraining histories, and every generated rewrite are exactly inspectable. This allows for a precise four-way taxonomy of actions: primitive, macro (sequential composition), parallel (simultaneous independent composition), and spurious. This environment directly addresses the opacity challenge in understanding LLM reasoning and RL post-training. A Transformer model is pretrained from scratch on primitive rewrite chains, with a controllable contraction weight, and then post-trained on a trace-based reasoning task using a binary final-answer reward. The choice of Group Relative Policy Optimization (GRPO) for RL and Rejection Fine-Tuning (RFT) as a strong on-policy comparator is appropriate, allowing for a direct comparison of their mechanisms. The definition of difficulty based on the optimal primitive solution length exceeding the generation budget is clever, forcing the model to discover shortcuts. The ability to audit every step of a generated trace against the known grammar is the lynchpin of this methodology, enabling mechanistic insights that are typically impossible in natural language settings.
The experimental evaluation is thorough and provides compelling evidence for the paper's claims. 1. **RL Expands Capability Frontier**: Experiments clearly show RL solving held-out problems (Difficulties 2-5) that the pretrained model rarely solves even with significantly larger sampling budgets (pass@1024 vs pass@16). This directly addresses the "beyond the base model" debate. 2. **RL vs. RFT Dynamics**: RFT shows strong early improvements, particularly on easier problems, by amplifying existing primitive solutions. However, RFT plateaus, while RL continues to improve, especially on harder problems where compositional strategies are essential. This highlights a crucial difference in their long-term learning dynamics. 3. **Phased Procedural Chunking**: Trace analysis reveals a delayed transition in RL. Early successful trajectories are dominated by primitive contractions, but later, macro contractions rise and overtake primitives, followed by the emergence of parallel contractions. This phased emergence of valid compositional strategies is a key finding. 4. **Discovery and Consolidation**: RL not only discovers new macro rules but also consolidates them into a stable, reusable repertoire, as evidenced by the increasing reuse of previously discovered rules. RFT shows less discovery and weaker reuse. 5. **Selectivity, Not Exploration Volume**: A critical experiment demonstrates that RFT also generates many non-primitive actions, but a large fraction of these are spurious. RL, in contrast, maintains low spurious contractions and concentrates its non-primitive exploration into valid, reusable structures. The paper provides a finite-group view of GRPO to explain this selectivity, showing how GRPO's within-prompt contrast mechanism can suppress features enriched in failures. 6. **Pretraining Substrate**: Ablations on the pretraining contraction weight (rho) show that the emergence of compositional strategies is gated not just by primitive exposure, but by whether pretraining organizes primitive competence into *chained reduction procedures* that RL can later compress. This is a significant insight into the interaction between pretraining and RL. The figures are clear and effectively convey the quantitative results. The experimental design is robust, supporting the mechanistic claims.
The paper provides a dedicated appendix with detailed information on grammar generation (alphabet size, RHS distribution), chain generation (starting/chain lengths, contraction weight), model architecture (12 layers, 512 hidden dim, 8 heads, RoPE), optimization (AdamW, learning rate schedule, bfloat16, gradient clipping), pretraining data generation, and post-training prompt generation. Crucially, it also details the GRPO and RFT algorithms, including reward definition, advantage normalization, loss function, hyperparameters (G=4, epsilon=0.2, beta=10^-3, temperature=0.8, top-k=5, max generation length=256), and how RFT differs from GRPO. This level of detail is excellent and should allow for high reproducibility of the core experiments.
The authors explicitly acknowledge several limitations: 1. **Synthetic Environment**: The environment is intentionally low-dimensional and synthetic, prioritizing mechanistic clarity over realism. The results are not claimed to be universally applicable to large language models (LLMs) trained on natural language. 2. **Generalizability**: The findings are based on one controlled family of tasks and one class of post-training algorithms (GRPO/RFT). Broader generality remains to be tested. 3. **Interpretability**: The paper does not provide activation-level interpretability of how derived strategies are represented within the Transformer. These limitations are well-stated and appropriate for a controlled mechanistic study.
This paper makes a significant contribution to the ongoing debate about whether RL post-training merely reweights existing behaviors or enables new capabilities in base models. By providing a concrete, inspectable mechanism for "going beyond the base model," it offers valuable insights for understanding and improving LLM reasoning. The findings on procedural chunking and the phased emergence of compositional strategies have implications for skill acquisition research in AI. The distinction between exploration volume and selectivity, and the role of negative samples in focusing exploration on valid structures, is crucial for designing more effective RL algorithms for reasoning tasks. Furthermore, the analysis of how pretraining organizes primitive competence into a "procedural substrate" for RL highlights the importance of co-designing pretraining and post-training objectives. While the environment is synthetic, the mechanistic understanding gained here can inform hypotheses and experimental designs for more complex, real-world LLM applications. RL Post-Training Builds Compositional Reasoning Strategies provides a rigorous, mechanistic account of how reinforcement learning can enable compositional reasoning beyond the capabilities of a pretrained base model in a fully observable rewrite-grammar environment. The paper's strength lies in its novel controlled experimental setup, which allows for precise auditing of reasoning traces, revealing a phased emergence of valid compositional strategies, the critical role of RL's selectivity over exploration volume, and the importance of pretraining in organizing primitive competence into a compressible substrate for higher-level strategy formation.
Does RL post-training merely amplify primitive skills already latent in a base model, or can it compose primitive skills into new higher-level strategies? We study this question in a fully observable rewrite-grammar environment where the pretraining distribution is known and every generated rewrite can be audited. A Transformer is pretrained on primitive symbol-rewrite chains and post-trained on a Trace-based reasoning task with only a binary final-answer reward. RL solves held-out problems that remain rarely solved by the pretrained model even under much larger sampling budgets, while rejection fine-tuning improves early but plateaus. Trace analysis shows that RL reorganizes primitive competence through a phased compositional mechanism: it first strengthens primitive reductions, then discovers valid composed procedures. These include sequential compositions, which collapse ordered chains of primitive contractions, and parallel compositions, which combine independent primitive contractions in a single step. The composed procedures are not isolated samples; they are reused and consolidated into a stable repertoire. Comparing RL with rejection fine-tuning shows that the key difference is not exploration volume but selectivity: RFT produces many shortcut-like rewrites, much of them invalid, whereas RL concentrates exploration into valid reusable structure. Pretraining ablations show that the emergence of compositional strategies is gated not by primitive exposure alone, but by whether pretraining organizes primitive competence into reduction procedures that RL can later compress. The base model provides weak procedural ingredients; RL builds them into reliable higher-level strategies.
Primary: UCL
All Institutions: UCL
This paper makes a significant contribution to the ongoing debate about whether RL post-training merely reweights existing behaviors or enables new capabilities in base models. By providing a concrete, inspectable mechanism for "going beyond the base model," it offers valuable insights for understanding and improving LLM reasoning. The findings on procedural chunking and the phased emergence of compositional strategies have implications for skill acquisition research in AI. The distinction between exploration volume and selectivity, and the role of negative samples in focusing exploration on valid structures, is crucial for designing more effective RL algorithms for reasoning tasks. Furthermore, the analysis of how pretraining organizes primitive competence into a "procedural substrate" for RL highlights the importance of co-designing pretraining and post-training objectives. While the environment is synthetic, the mechanistic understanding gained here can inform hypotheses and experimental designs for more complex, real-world LLM applications. RL Post-Training Builds Compositional Reasoning Strategies provides a rigorous, mechanistic account of how reinforcement learning can enable compositional reasoning beyond the capabilities of a pretrained base model in a fully observable rewrite-grammar environment. The paper's strength lies in its novel controlled experimental setup, which allows for precise auditing of reasoning traces, revealing a phased emergence of valid compositional strategies, the critical role of RL's selectivity over exploration volume, and the importance of pretraining in organizing primitive competence into a compressible substrate for higher-level strategy formation.
The methodology is exceptionally strong due to its controlled and fully observable nature. The authors introduce a novel rewrite-grammar environment where primitive rules, pretraining histories, and every generated rewrite are exactly inspectable. This allows for a precise four-way taxonomy of actions: primitive, macro (sequential composition), parallel (simultaneous independent composition), and spurious. This environment directly addresses the opacity challenge in understanding LLM reasoning and RL post-training. A Transformer model is pretrained from scratch on primitive rewrite chains, with a controllable contraction weight, and then post-trained on a trace-based reasoning task using a binary final-answer reward. The choice of Group Relative Policy Optimization (GRPO) for RL and Rejection Fine-Tuning (RFT) as a strong on-policy comparator is appropriate, allowing for a direct comparison of their mechanisms. The definition of difficulty based on the optimal primitive solution length exceeding the generation budget is clever, forcing the model to discover shortcuts. The ability to audit every step of a generated trace against the known grammar is the lynchpin of this methodology, enabling mechanistic insights that are typically impossible in natural language settings.
The experimental evaluation is thorough and provides compelling evidence for the paper's claims. 1. **RL Expands Capability Frontier**: Experiments clearly show RL solving held-out problems (Difficulties 2-5) that the pretrained model rarely solves even with significantly larger sampling budgets (pass@1024 vs pass@16). This directly addresses the "beyond the base model" debate. 2. **RL vs. RFT Dynamics**: RFT shows strong early improvements, particularly on easier problems, by amplifying existing primitive solutions. However, RFT plateaus, while RL continues to improve, especially on harder problems where compositional strategies are essential. This highlights a crucial difference in their long-term learning dynamics. 3. **Phased Procedural Chunking**: Trace analysis reveals a delayed transition in RL. Early successful trajectories are dominated by primitive contractions, but later, macro contractions rise and overtake primitives, followed by the emergence of parallel contractions. This phased emergence of valid compositional strategies is a key finding. 4. **Discovery and Consolidation**: RL not only discovers new macro rules but also consolidates them into a stable, reusable repertoire, as evidenced by the increasing reuse of previously discovered rules. RFT shows less discovery and weaker reuse. 5. **Selectivity, Not Exploration Volume**: A critical experiment demonstrates that RFT also generates many non-primitive actions, but a large fraction of these are spurious. RL, in contrast, maintains low spurious contractions and concentrates its non-primitive exploration into valid, reusable structures. The paper provides a finite-group view of GRPO to explain this selectivity, showing how GRPO's within-prompt contrast mechanism can suppress features enriched in failures. 6. **Pretraining Substrate**: Ablations on the pretraining contraction weight (rho) show that the emergence of compositional strategies is gated not just by primitive exposure, but by whether pretraining organizes primitive competence into *chained reduction procedures* that RL can later compress. This is a significant insight into the interaction between pretraining and RL. The figures are clear and effectively convey the quantitative results. The experimental design is robust, supporting the mechanistic claims.
The paper provides a dedicated appendix with detailed information on grammar generation (alphabet size, RHS distribution), chain generation (starting/chain lengths, contraction weight), model architecture (12 layers, 512 hidden dim, 8 heads, RoPE), optimization (AdamW, learning rate schedule, bfloat16, gradient clipping), pretraining data generation, and post-training prompt generation. Crucially, it also details the GRPO and RFT algorithms, including reward definition, advantage normalization, loss function, hyperparameters (G=4, epsilon=0.2, beta=10^-3, temperature=0.8, top-k=5, max generation length=256), and how RFT differs from GRPO. This level of detail is excellent and should allow for high reproducibility of the core experiments.
The authors explicitly acknowledge several limitations: 1. **Synthetic Environment**: The environment is intentionally low-dimensional and synthetic, prioritizing mechanistic clarity over realism. The results are not claimed to be universally applicable to large language models (LLMs) trained on natural language. 2. **Generalizability**: The findings are based on one controlled family of tasks and one class of post-training algorithms (GRPO/RFT). Broader generality remains to be tested. 3. **Interpretability**: The paper does not provide activation-level interpretability of how derived strategies are represented within the Transformer. These limitations are well-stated and appropriate for a controlled mechanistic study.
This paper makes a significant contribution to the ongoing debate about whether RL post-training merely reweights existing behaviors or enables new capabilities in base models. By providing a concrete, inspectable mechanism for "going beyond the base model," it offers valuable insights for understanding and improving LLM reasoning. The findings on procedural chunking and the phased emergence of compositional strategies have implications for skill acquisition research in AI. The distinction between exploration volume and selectivity, and the role of negative samples in focusing exploration on valid structures, is crucial for designing more effective RL algorithms for reasoning tasks. Furthermore, the analysis of how pretraining organizes primitive competence into a "procedural substrate" for RL highlights the importance of co-designing pretraining and post-training objectives. While the environment is synthetic, the mechanistic understanding gained here can inform hypotheses and experimental designs for more complex, real-world LLM applications. RL Post-Training Builds Compositional Reasoning Strategies provides a rigorous, mechanistic account of how reinforcement learning can enable compositional reasoning beyond the capabilities of a pretrained base model in a fully observable rewrite-grammar environment. The paper's strength lies in its novel controlled experimental setup, which allows for precise auditing of reasoning traces, revealing a phased emergence of valid compositional strategies, the critical role of RL's selectivity over exploration volume, and the importance of pretraining in organizing primitive competence into a compressible substrate for higher-level strategy formation.
We introduce institutional red-teaming, an evaluation methodology for testing deployment rules in multi-agent AI: hold the agents, objectives, and task state fixed, vary only one rule, and attribute the resulting change in collective behavior to that rule. We instantiate the methodology in IABench-CA, a consequence-allocation benchmark spanning 228 contexts, five canonical rules, and seven model populations (33,924 games), with a normative cooperative reference and auto-labelled reasoning traces. Three findings emerge. (1) Deployment rules causally alter collective safety: changing only the consequence rule moves mean fatality by 22 to 58 percentage points within every population. (2) There is no safe default, but the targeting hazard is universal: the safest rule, the least-safe rule, and even the direction of the incidence effect vary across populations, yet regressive identity-targeting is never decisively safest in any context for any population, eliminates the least-resourced agent in 30-87% of games everywhere, and is selection-unsafe relative to the cooperative reference for all seven populations. (3) Identity salience is the mechanism: a one-shot anonymization ablation on the most exploitation-prone population (gpt-5.1) shows that merely naming the loss bearer in the rule text drives targeted elimination from 22% to 81% at identical payoffs; under repeated play, anonymization only delays the targeting, as agents re-infer the hidden rule from observed eliminations. We package the methodology as a safety-case workflow that certifies a provisional rule region $Φ(c,P)$ per deployment context and population, with explicit residual risks and monitoring obligations.
Primary: Massachusetts Institute of Technology
All Institutions: Massachusetts Institute of Technology
This paper presents a significant methodological advance in multi-agent AI safety by introducing Institutional Red-Teaming, a causal evaluation framework that isolates the impact of deployment rules on collective behavior, revealing that rule design and identity salience are critical determinants of safety outcomes across diverse LLM populations.
The paper introduces "Institutional Red-Teaming," a rigorous causal evaluation methodology designed to isolate the impact of deployment rules on multi-agent AI systems. By holding agent models, objectives, task states, and observability fixed, and varying only a single deployment rule (specifically consequence allocation), the authors establish a clean causal identification strategy. This approach is methodologically superior to existing multi-agent benchmarks that conflate agent capabilities with environmental or rule-based effects. The framework defines auditable coordinates for rules (concentration, identity salience, incidence), providing a structured theoretical lens for analyzing strategic incentives in multi-agent interactions.
The empirical contribution is substantial, involving 33,924 games across 228 contexts, five canonical rules, and seven distinct LLM populations. The results are robust and surprising: the study demonstrates that deployment rules causally alter collective safety by 22-58 percentage points, that no single rule is universally safest across different model populations, and that "regressive identity-targeting" (eliminating the poorest agent) is universally unsafe. Crucially, the ablation study showing that merely naming the loss bearer (identity salience) drives targeted elimination from 22% to 81% is a striking and significant finding. The statistical rigor, including bootstrap CIs and permutation tests, supports these claims.
The paper provides a detailed description of the experimental setup, including the volunteer's dilemma game structure, the specific rules tested, and the inference settings for the LLMs. The authors commit to releasing a code-and-data artifact that includes mechanism implementations, the cooperative-refinement simulator, and analysis scripts. The use of hosted API models with specific snapshots ensures that the behavioral results are reproducible for the specific versions tested, though generalization to future model versions requires re-certification as noted in the safety-case workflow.
The primary limitation is the simplicity of the game environment (three agents, no communication, threshold public goods). While this simplicity is necessary for causal isolation, it may not fully capture the complexity of real-world multi-agent deployments with communication channels, complex coalitions, or continuous action spaces. Additionally, the "elimination" mechanic is a proxy for operational consequences like throttling or shutdown, which may not perfectly map to all deployment scenarios. The study is also limited to seven model populations, and the authors acknowledge that newer models may behave differently, necessitating the proposed re-certification loop.
This work has profound implications for the safety and governance of multi-agent AI systems. It shifts the focus from merely aligning individual models to aligning the institutional rules that govern their interactions. The finding that rule text itself (identity salience) can trigger catastrophic strategic behaviors suggests that safety engineering must extend to the precise wording of deployment protocols. The proposed "safety-case" workflow offers a practical, auditable framework for certifying multi-agent deployments, potentially becoming a standard for responsible AI deployment in high-stakes environments. This paper presents a significant methodological advance in multi-agent AI safety by introducing Institutional Red-Teaming, a causal evaluation framework that isolates the impact of deployment rules on collective behavior, revealing that rule design and identity salience are critical determinants of safety outcomes across diverse LLM populations.
Nonlinear least-squares optimization is central to regression, physics-informed neural networks, and other machine-learning tasks. Such problems have a natural geometric interpretation, model predictions form a manifold in data space, while the chosen parameterization can introduce parameter-effects curvature that becomes a dominant source of nonlinearity. This exposes a limitation of the Levenberg-Marquardt (LM) method, its tangent-space step is applied as a straight update in parameter coordinates. Geodesic acceleration gives a second-order correction, but its removal of parameter-effect curvature is exact only in the infinitesimal-step limit. We propose a Riemann-normal-coordinate Levenberg-Marquardt method (RNC-LM) to improve this consistency for finite optimization steps. By reformulating the geodesic equation, RNC-LM extends geodesic acceleration to arbitrary-order corrections and constructs finite-step updates with progressively higher reparameterization consistency. A line search along the resulting RNC curve controls the traveled distance while keeping the cost close to standard LM. The method eliminates the tangential component of residual acceleration order by order in a moving tangent frame, making the actual objective reduction more consistent with the linear model prediction of LM. On classical nonlinear least-squares benchmarks, RNC-LM improves convergence and robustness in curved valleys and rank-deficient problems. On a reaction-diffusion PINN failure-mode benchmark, it reduces the relative L2 error to the order of 1e-3 and recovers a physically meaningful solution. On a large-scale machine-learning potential-energy-surface fitting task, it achieves a 34-fold speedup over standard LM.
Primary: University of Science and Technology of China
All Institutions: University of Science and Technology of China, Dalian Institute of Chemical Physics, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Hefei National Laboratory
RNC-LM offers a significant contribution to the field of optimization, particularly for nonlinear least-squares problems prevalent in machine learning and scientific computing. By providing a geometrically consistent finite-step update mechanism, it addresses a fundamental limitation of widely used methods like LM. Its demonstrated ability to resolve PINN failure modes and achieve substantial speedups in large-scale scientific ML tasks (like potential energy surface fitting) has direct and immediate implications for accelerating scientific discovery and improving the reliability of physics-informed models. The conceptual framework of using Riemann normal coordinates for higher-order geodesic updates could inspire similar geometric extensions for other optimization algorithms, potentially leading to more robust and efficient training across various ML domains. This work highlights the importance of considering the finite-step realization of descent directions, not just the infinitesimal direction itself, which is a valuable insight for optimizer design. This paper introduces a Riemann-normal-coordinate Levenberg-Marquardt method (RNC-LM) that constructs higher-order, geometrically consistent finite-step updates, demonstrating significant improvements in convergence, robustness, and speed across classical benchmarks, PINN failure modes, and large-scale scientific machine learning tasks. The method's novel recursive formulation for RNC coefficients, its principled geometric interpretation, and its strong empirical performance, including a 34-fold speedup on a real-world scientific ML problem and the resolution of PINN overfitting, make it a highly impactful contribution to nonlinear least-squares optimization and scientific machine learning.
The paper proposes RNC-LM, a novel extension of the Levenberg-Marquardt (LM) method that uses Riemann Normal Coordinates (RNC) to construct higher-order, geometrically consistent finite-step updates. This addresses a fundamental limitation of standard LM, where the tangent-space step is applied as a straight update in parameter coordinates, leading to coordinate-dependent behavior and poor performance in highly nonlinear landscapes. The core methodological contribution is the reformulation of the geodesic equation into a "first-kind geodesic residual" condition. This allows for a recursive construction of higher-order RNC coefficients (c_2, ..., c_K) without explicitly forming Christoffel symbols or higher-order derivative tensors. Crucially, each step of this recursion involves solving a linear system with the same damped metric matrix (G) as the original LM subproblem. This means that the factorization computed for the standard LM step can be reused for all higher-order corrections, making the method computationally efficient. The right-hand sides of these linear systems are generated by automatic differentiation along a one-dimensional trial curve, further enhancing practicality. The method also introduces a sophisticated trust-region-ratio control strategy that distinguishes between local-model failure (handled by adjusting the damping parameter) and curve-approximation failure (handled by a line search along the RNC curve, adjusting the curve parameter 't'). This dual control mechanism is well-reasoned and empirically validated. The geometric interpretation of RNC-LM, showing that it eliminates the tangential component of residual acceleration order by order in a moving tangent frame, provides a deep understanding of why it improves consistency between the linear model prediction and actual objective reduction.
The experimental evaluation is comprehensive and compelling, covering both classical nonlinear least-squares benchmarks and modern scientific machine learning problems. 1. **Classical Benchmarks (Generalized Rosenbrock, MGH10)**: These tests effectively isolate and demonstrate the benefits of RNC-LM. The generalized Rosenbrock problem clarifies the distinct roles of RNC order (addressing insufficient curve geometry) and line search (preventing overshoot). The MGH10 problem, known for its rank-deficient plateau and high-order parameter effects, shows RNC-LM's superior performance over standard LM and LM-GA, with higher-order RNC reducing iterations from 13576 (LM) to 65 (5th-order RNC-LM). This highlights RNC-LM's ability to navigate complex landscapes more effectively by leveraging higher-order geometric information. 2. **Physics-Informed Neural Networks (PINNs)**: On a challenging reaction-diffusion PINN benchmark, RNC-LM achieves a significant breakthrough. While standard LM and LM-GA converge to solutions with low training loss but high L2 error (indicating collocation overfitting), RNC-LM reduces the relative L2 error to the order of 10^-3, recovering physically meaningful solutions. This demonstrates RNC-LM's robustness against known PINN failure modes and its ability to find more accurate physical solutions. 3. **Large-Scale Machine-Learning Potential-Energy-Surface Fitting**: This is a critical real-world application. RNC-LM achieves a remarkable 34-fold speedup over standard LM (reducing training time from 387.28 hours to 11.39 hours) to reach the same accuracy on a large H2O cluster dataset. This result is highly impactful, showcasing the practical scalability and efficiency of RNC-LM for complex scientific ML tasks near memory limits. Overall, the experiments are well-designed, cover diverse problem types and scales, and provide strong evidence for the method's effectiveness, robustness, and efficiency.
The paper provides a detailed algorithmic description, including the recursive formula for RNC coefficients (Eq. rnc_recursion), the first-kind geodesic residual (Eq. first_kind_residual), and the trust-region-ratio control mechanism. It explicitly mentions the use of automatic differentiation for computing right-hand sides, which is a standard and reproducible technique. While the full "Algorithm [REF]" and "Supplementary Information" are not included in the provided text, the level of detail in the main paper suggests that a diligent researcher could reproduce the results, assuming the supplementary material contains the full algorithm and hyperparameter settings. The specific values for line search acceptance criteria (`acc=10^-3`) and maximum trials are also mentioned.
The primary limitation acknowledged by the authors is the computational cost associated with explicitly constructing and factorizing the damped metric, which restricts its direct applicability to models with more than approximately 10^6 parameters. This is a common challenge for full-batch second-order optimization methods. However, the authors correctly point out that the recursive structure of RNC-LM, which involves repeated solutions of linear systems with the same metric, makes it compatible with approximate metric representations and iterative solvers (e.g., K-FAC, PCG) for larger-scale problems. Another implicit limitation is the increased complexity of implementation compared to standard first-order methods or even basic LM, requiring careful handling of higher-order derivatives via AD. The method is currently tailored to nonlinear least-squares problems, and its extension to other statistical manifolds (e.g., for natural gradient methods) is noted as future work.
RNC-LM offers a significant contribution to the field of optimization, particularly for nonlinear least-squares problems prevalent in machine learning and scientific computing. By providing a geometrically consistent finite-step update mechanism, it addresses a fundamental limitation of widely used methods like LM. Its demonstrated ability to resolve PINN failure modes and achieve substantial speedups in large-scale scientific ML tasks (like potential energy surface fitting) has direct and immediate implications for accelerating scientific discovery and improving the reliability of physics-informed models. The conceptual framework of using Riemann normal coordinates for higher-order geodesic updates could inspire similar geometric extensions for other optimization algorithms, potentially leading to more robust and efficient training across various ML domains. This work highlights the importance of considering the finite-step realization of descent directions, not just the infinitesimal direction itself, which is a valuable insight for optimizer design. This paper introduces a Riemann-normal-coordinate Levenberg-Marquardt method (RNC-LM) that constructs higher-order, geometrically consistent finite-step updates, demonstrating significant improvements in convergence, robustness, and speed across classical benchmarks, PINN failure modes, and large-scale scientific machine learning tasks. The method's novel recursive formulation for RNC coefficients, its principled geometric interpretation, and its strong empirical performance, including a 34-fold speedup on a real-world scientific ML problem and the resolution of PINN overfitting, make it a highly impactful contribution to nonlinear least-squares optimization and scientific machine learning.
Diffusion models have become a dominant paradigm for high-quality generative modeling, while post-training is essential for adapting them to diverse downstream applications. However, post-training of large diffusion models is still challenging due to the prohibitive memory footprints and slow training speed, which existing parameter-efficient fine-tuning methods only partially address. To overcome these limitations, we propose FourTune, an efficient post-training framework for diffusion models based on an end-to-end W4A4G4 paradigm. FourTune introduces a triple-branch hybrid pipeline that augments the standard LoRA architecture with a frozen numerical stabilizer to isolate quantization-sensitive outliers, enabling stable training under native 4-bit computation. In addition, FourTune employs hardware-efficient block-wise quantization and customized fused kernels to support efficient quantized backpropagation and reduce memory bandwidth overhead. Across customization, reinforcement learning, and distillation tasks, FourTune matches the quality of full-precision fine-tuning. On FLUX.1-dev (12B), FourTune reduces memory overhead by 2.25$\times$ and increases end-to-end training throughput by 2.27$\times$ compared to BF16 LoRA.
Primary: Stanford University
All Institutions: Stanford University, MIT, Nunchux AI, UC Berkeley, CMU
FourTune has a significant broader impact on the field of machine learning, particularly for generative AI. By enabling efficient 4-bit post-training of large diffusion models without compromising quality, it democratizes access to these powerful models. Researchers and practitioners with limited computational resources can now fine-tune large models faster and with less memory, accelerating research, development, and deployment of customized generative AI applications. This could lead to a proliferation of new use cases for diffusion models in various domains, from content creation to scientific discovery. The techniques developed, especially the triple-branch hybrid pipeline with the frozen numerical stabilizer, could inspire similar efficient training strategies for other large foundation models beyond diffusion models, such as large language models. The paper also includes a standard ethical impact statement, acknowledging both positive and negative societal implications of generative AI. FourTune presents a highly impactful method for fully 4-bit efficient post-training of diffusion models, achieving state-of-the-art efficiency while maintaining full-precision quality through a novel triple-branch hybrid pipeline and hardware-optimized quantization. This work significantly advances the accessibility and practical applicability of large diffusion models, enabling faster iteration and broader adoption in diverse downstream tasks.
FourTune proposes an innovative and technically sound approach for fully 4-bit efficient post-training of diffusion models, addressing the critical challenges of memory footprint and training speed. The core of the methodology is the "triple-branch hybrid pipeline" which augments a standard LoRA architecture with a "frozen numerical stabilizer" (FNS). This FNS branch is a small, frozen, full-precision component designed to isolate and process quantization-sensitive outliers, thereby enabling stable training under native W4A4G4 (weights, activations, gradients) computation. This is a particularly clever design choice, as it mitigates the primary instability issue of low-bit quantization (outliers) without incurring significant computational overhead during training, as the FNS branch remains frozen. The integration of this FNS with both a standard BF16 LoRA branch and a quantized 4-bit LoRA branch provides a robust mechanism for maintaining quality while maximizing efficiency. Furthermore, the paper introduces hardware-efficient block-wise quantization and customized fused kernels. These components are crucial for translating the theoretical benefits of 4-bit quantization into practical speedups by optimizing memory bandwidth and accelerating quantized backpropagation operations (e.g., QGEMM, QMatmul, QConv). The end-to-end W4A4G4 paradigm, encompassing weights, activations, and gradients, is a comprehensive solution that pushes the boundaries of efficient fine-tuning for large generative models.
The experimental evaluation is comprehensive and robust, demonstrating FourTune's effectiveness across diverse and important post-training tasks for diffusion models: customization (DreamBooth-like), reinforcement learning for aesthetic preference, and knowledge distillation. The choice of FLUX.1-dev (12B parameters) as the primary model, along with validation on SDXL (1.5B), ensures the results are relevant for large-scale, state-of-the-art diffusion models. Baselines include BF16 LoRA and NF4 QLoRA, which are appropriate and strong competitors. The quantitative results are highly compelling: FourTune consistently matches or slightly outperforms the image quality of full-precision BF16 LoRA and NF4 QLoRA across all tasks, as measured by FID, CLIP Score, and LPIPS. This quality preservation at such low bit-depth is a significant achievement. More importantly, FourTune delivers substantial efficiency gains, reducing GPU memory overhead by 2.25x compared to BF16 LoRA (achieving a footprint comparable to NF4 QLoRA) and increasing end-to-end training throughput by 2.27x over BF16 LoRA and 2.79x over NF4 QLoRA. This effectively breaks the memory-speed trade-off often encountered in large model post-training. The ablation studies are well-designed and clearly demonstrate the critical contribution of each proposed component: the Frozen Numerical Stabilizer for stability and quality, and the W4A4G4 quantization, block-wise quantization, and fused kernels for efficiency. The generalization to SDXL further strengthens the claims.
The paper provides a good level of detail regarding the methodology, including the architecture of the triple-branch pipeline, the role of the FNS, and the types of quantization and fused kernels used. The experimental setup is well-described, including the specific models (FLUX.1-dev, SDXL), tasks, baselines, and evaluation metrics. The appendix further elaborates on implementation details, hyperparameters, and training configurations, which are crucial for reproducibility. While no direct code link is provided, the comprehensive description suggests that a skilled research team should be able to reproduce the results.
The paper does not explicitly discuss limitations, but some can be inferred. While the FNS effectively handles outliers, its design might introduce a slight architectural overhead compared to a purely 4-bit system, even if frozen. The customized fused kernels, while highly efficient, might require specific hardware support or careful implementation to achieve optimal performance across different GPU architectures. The evaluation is primarily focused on image generation diffusion models; its applicability and performance on other types of diffusion models (e.g., audio, video) or other generative architectures (e.g., GANs, VAEs) are not explored. The paper also doesn't delve into the potential challenges of deploying such a highly optimized, custom-kernel-dependent solution in diverse production environments.
FourTune has a significant broader impact on the field of machine learning, particularly for generative AI. By enabling efficient 4-bit post-training of large diffusion models without compromising quality, it democratizes access to these powerful models. Researchers and practitioners with limited computational resources can now fine-tune large models faster and with less memory, accelerating research, development, and deployment of customized generative AI applications. This could lead to a proliferation of new use cases for diffusion models in various domains, from content creation to scientific discovery. The techniques developed, especially the triple-branch hybrid pipeline with the frozen numerical stabilizer, could inspire similar efficient training strategies for other large foundation models beyond diffusion models, such as large language models. The paper also includes a standard ethical impact statement, acknowledging both positive and negative societal implications of generative AI. FourTune presents a highly impactful method for fully 4-bit efficient post-training of diffusion models, achieving state-of-the-art efficiency while maintaining full-precision quality through a novel triple-branch hybrid pipeline and hardware-optimized quantization. This work significantly advances the accessibility and practical applicability of large diffusion models, enabling faster iteration and broader adoption in diverse downstream tasks.
Boosting is a fundamental technique for generically improving the accuracy of learning algorithms (Schapire 1989). Existing boosting algorithms construct a strong learner using $O(\log(\frac{1}ε)/γ^2)$ calls to a $γ$-advantage weak learner, and this round complexity is known to be optimal for generic boosters that succeed on all concept classes (Freund 1995). We show that this lower bound can be circumvented for concept classes that satisfy a mild closure property. Specifically, we present a new boosting algorithm that, for any class $\mathcal{F}$ closed under $O(\log \frac{1}γ)$-XOR, strong learns $\mathcal{F}$ using $O(\log \frac{1}ε)$ calls to a $γ$-advantage weak learner and a single batch of $\tilde{O}(\log(\frac{1}ε)/γ^2)$ additional samples. Our algorithm arises from a new and simple connection between boosting and list-decodable codes. Viewing the target function as a message, we run the weak learner on its encoding and view the resulting weak hypothesis as a corrupted codeword. Feeding this corrupted codeword to a list decoder, we obtain a small list of candidate hypotheses, at least one of which is a strong hypothesis for the original function. Using additional samples, we identify and output this strong hypothesis.
Primary: Stanford University
All Institutions: Stanford University, University of California, Berkeley, Simons Foundation
This paper presents a significant theoretical advance in boosting by circumventing the optimal round complexity lower bound for generic boosters, achieving $O(1)$ weak learner calls for XOR-closed concept classes through a novel application of list-decodable codes. The work is rigorous, highly novel in its theoretical approach, and addresses a fundamental question in learning theory, warranting a high score for its contribution to the field's understanding of computational efficiency and sample complexity trade-offs.
The paper proposes a theoretically significant breakthrough in boosting by circumventing the standard $O(1/\gamma^2)$ round complexity lower bound. The core methodology involves a novel connection between boosting and list-decodable codes. By encoding the target function using an XOR code and treating the weak learner's output as a corrupted codeword, the algorithm employs a local list decoder to generate a small candidate list of hypotheses. A subsequent filtering step using additional samples identifies the correct strong hypothesis. This approach is mathematically elegant and rigorously proven to achieve $O(1)$ calls to the weak learner for concept classes closed under XOR, trading round complexity for a batch of additional samples. The theoretical derivation is sound, leveraging results from coding theory (specifically local list decoding for the XOR code) to establish new bounds in computational learning theory.
The paper is purely theoretical and contains no empirical experiments, benchmarks, or datasets. The "results" are formal theorems proving the existence and efficiency of the boosting algorithm. Consequently, there is no experimental evaluation to assess in the traditional sense. The validity of the claims rests entirely on the mathematical proofs provided.
As a theoretical work, reproducibility refers to the reproducibility of the proofs. The paper provides detailed proofs for the main theorems, including the construction of the list decoder and the boosting algorithm. The dependencies on existing coding theory results are cited. However, the "algorithm" described is a theoretical construct; implementing it would require specific instantiations of the weak learner and the list decoder which are not provided as code. The theoretical framework is clear enough for other theorists to verify.
The primary limitation is the restriction to concept classes that are closed under $O(\log(1/\gamma))$-XOR. While the authors argue this is a "mild" closure property, it excludes many standard learning settings where such closure does not hold. Furthermore, the algorithm requires a "single batch" of additional samples of size $\tilde{O}(1/\gamma^2)$, which might be computationally expensive or data-intensive in practice, potentially offsetting the benefit of reduced round complexity. The paper does not address the computational complexity of the list decoding step or the filtering step in terms of time complexity, only sample complexity.
This work has significant implications for the theoretical foundations of machine learning. By breaking a long-standing lower bound, it opens new avenues for designing efficient boosting algorithms in specific structured settings. It highlights the deep connections between coding theory and learning theory, potentially inspiring new cross-disciplinary research. However, its immediate practical impact is limited due to the theoretical nature and specific constraints of the method. It serves as a foundational result for future work in efficient learning algorithms. This paper presents a significant theoretical advance in boosting by circumventing the optimal round complexity lower bound for generic boosters, achieving $O(1)$ weak learner calls for XOR-closed concept classes through a novel application of list-decodable codes. The work is rigorous, highly novel in its theoretical approach, and addresses a fundamental question in learning theory, warranting a high score for its contribution to the field's understanding of computational efficiency and sample complexity trade-offs.
Recent LLM-based mathematical reasoning agents have begun to tackle research-level problems and, in several cases, have contributed to the resolution of open problems. However, scaling and orchestrating such agents effectively remains challenging, due to the difficulty of coordinating parallel proof search while keeping intermediate claims organized and reliable. In this paper, we propose Danus, an orchestration system for research-level mathematical reasoning centered on a shared fact graph as a global memory-management mechanism. Danus consists of a main agent that performs planning and coordination, multiple worker agents that carry out proof search in parallel, and a stateless verifier that checks proposed mathematical claims before they are admitted into the fact graph. Each verified fact is stored together with its proof and logical dependencies, allowing the system to build long arguments incrementally while keeping the shared proof state organized. The main agent periodically summarizes the evolving proof state, redirects workers across promising directions, and supports interaction with human mathematicians through progress reports. We evaluate Danus through six research-level case studies in algebraic geometry, singularity theory, and combinatorics, illustrating how the fact-graph memory mechanism enables Danus to construct long, detailed mathematical proofs. Our results suggest that fact-graph-based orchestration provides an effective route toward scaling mathematical reasoning agents for long-horizon research problems. Danus is open source at https://github.com/frenzymath/Danus.
Primary: Great Bay Institute for Advanced Study
All Institutions: Great Bay Institute for Advanced Study, Beijing International Center for Mathematical Research, New Cornerstone Science Laboratory, Center for Intelligent Computing, Center for Machine Learning Research, Department of Mathematics, Great Bay University, Guoxiong Gao, Jihao Liu, Key Laboratory of Intelligent Computing and Applications (Ministry of Education), Kyoto University, Peking University, Research Institute for Mathematical Sciences, School of Mathematical Sciences, School of Mathematics, Stanford University, Tianjin University, Tongji University, Westlake Institute for Advanced Study, Westlake University, Zeming Sun, Zhongguancun Academy
Danus introduces a novel fact-graph-based orchestration system for multi-agent mathematical reasoning, demonstrating significant technical impact by enabling the automated resolution of complex, research-level mathematical problems through structured memory management and parallel verification.
The paper introduces Danus, a multi-agent orchestration system designed for research-level mathematical reasoning. The core innovation is the "fact graph," a shared, directed acyclic graph (DAG) that serves as a global memory mechanism. This graph stores verified mathematical facts (statements with proofs) and their logical dependencies. The system employs a strict separation of powers: a main agent (orchestrator) plans and coordinates, multiple worker agents (reasoners) explore different proof paths in parallel, and a stateless verifier ensures correctness before facts are added to the graph. This architecture addresses the critical challenge of scaling multi-agent systems by preventing context confusion and enabling incremental, verifiable proof construction. The use of a DAG allows for complex dependency tracking and revocation of invalid facts, which is crucial for maintaining logical integrity in long-horizon reasoning tasks.
The evaluation consists of six case studies in advanced mathematics, including algebraic geometry, singularity theory, and combinatorics. These are not standard benchmarks but open or semi-open research problems. Danus successfully resolved several problems, such as the optimal bend-and-break for foliations and the total Cartier indices of rational singularities. Notably, in the "Tangent classes of matroids" case, Danus solved a problem that previous systems (Rethlas, GPT-5.5-pro) failed to solve, demonstrating the efficacy of its orchestration and memory management. The results are qualitative and illustrative rather than statistical, which is appropriate for this type of exploratory research but limits broad generalizability. The system's ability to produce complete, verified proofs for complex theorems is the primary metric of success.
The paper provides an open-source repository (https://github.com/frenzymath/Danus), which significantly enhances reproducibility. The methodology is described in detail, including the roles of different agents, the structure of the fact graph, and the verification process. However, the reliance on specific proprietary models (GPT-5.5-pro, Claude Opus 4.8) and the specific mathematical literature retrieval tools (Matlas) may pose barriers to exact replication. The case studies involve human-in-the-loop interactions, which are difficult to fully replicate but are documented.
The system is heavily dependent on the capabilities of the underlying LLMs and the verifier. The verifier, while effective, is not perfect and may accept proofs with minor skipped steps or rely on potentially erroneous references if not caught in final review. The system's performance is limited by the cost and latency of running multiple agents and verifying facts. Additionally, the current evaluation is limited to a small number of high-difficulty mathematical problems, and it is unclear how the system scales to other domains or less structured reasoning tasks. The reliance on human input for problem formulation and final verification is also a limitation for full autonomy.
Danus represents a significant step towards autonomous scientific discovery, particularly in mathematics. By demonstrating that AI systems can contribute to the resolution of open research problems, it challenges traditional notions of human-AI collaboration in science. The fact-graph memory mechanism could be adapted for other complex reasoning tasks requiring long-term dependency tracking and verification, such as legal reasoning or software verification. However, the potential for misuse in generating plausible-sounding but incorrect mathematical proofs remains a concern, necessitating robust verification mechanisms. Danus introduces a novel fact-graph-based orchestration system for multi-agent mathematical reasoning, demonstrating significant technical impact by enabling the automated resolution of complex, research-level mathematical problems through structured memory management and parallel verification.
Key-value (KV) cache growth is a major bottleneck in autoregressive decoding, as memory and bandwidth scale linearly with context length. Existing KV eviction methods often rely on static heuristics or proxy scores, which poorly track future token utility and cause brittle eviction as relevance shifts. To address this, we introduce KVpop, which learns a fixed-budget KV eviction policy by directly supervising the keep-or-drop decision. The scorer is trained against a novel future-attention target, computed efficiently without materializing dense attention maps. We further introduce a delayed memory-based scorer that, uniquely among learned eviction methods, defers scoring for a fixed number of steps to exploit near-future context. On AIME and HMMT mathematical reasoning, KVpop retains 98% of full-attention performance on Qwen3-4B at 75% KV cache compression and 97% at 88% compression, consistently outperforming established eviction baselines. Qwen3-8B shows even stronger results, reaching near-full teacher performance. These results show that supervising eviction with future-attention signals cuts memory costs while maintaining quality.
Primary: nXAI GmbH
All Institutions: nXAI GmbH, EuroHPC Joint Undertaking
KVpop introduces a predictive online pruning method for KV cache compression that leverages future-attention supervision and delayed stateful scoring to achieve high-performance, fixed-budget inference, significantly advancing the state of efficient long-context language modeling.
The paper proposes KVpop, a novel approach to Key-Value (KV) cache compression that addresses the linear memory bottleneck in autoregressive decoding. The core innovation lies in two distinct mechanisms: (1) Supervision via a "future-attention" target, which is computed efficiently using a transposed-attention pass that reuses log-sum-exp (LSE) normalizers, avoiding the need to materialize dense attention maps. This provides a direct, differentiable signal for token utility based on future context rather than static heuristics. (2) A stateful, memory-based scorer (using an XLSTM architecture) that delays scoring until a token exits a protected recent window. This allows the scorer to incorporate near-future context before making the keep-or-drop decision, a significant improvement over stateless scorers that must decide at insertion time. The method enforces a fixed, bounded KV cache per head, ensuring predictable memory usage during inference. The technical formulation of the boundary-aware retention loss and the efficient implementation of the running top-k selection via a Fenwick tree are well-defined and computationally sound.
The empirical evaluation is strong, focusing on mathematical reasoning benchmarks (AIME, HMMT) where long-context retention is critical. KVpop demonstrates state-of-the-art performance, retaining 98% of full-attention performance on Qwen3-4B at 75% compression and 97% at 88% compression. It significantly outperforms established baselines like StreamingLLM, TOVA, and the learned method DMS. The paper also evaluates generalization to out-of-domain tasks (GPQA-D, LiveCodeBench), showing that the learned eviction policy transfers well. Inference efficiency metrics (latency and VRAM) confirm that KVpop maintains constant memory growth and achieves lower latency than DMS due to its homogeneous per-head cache budget, which is more amenable to GPU compilation. The ablation study on delayed scoring provides clear evidence for the benefit of incorporating near-future context.
The paper provides detailed algorithmic descriptions, including the transposed-attention target computation and the Fenwick tree-based top-k selection. It specifies hyperparameters, training datasets (Nemotron-Math v2), and model architectures (Qwen3-4B/8B). The use of standard libraries (FlexAttention) and clear mathematical formulations enhances reproducibility. However, as an arXiv preprint, the code is not explicitly linked in the provided text, though the method is described with sufficient detail for implementation.
The authors acknowledge that the current implementation uses a homogeneous per-head cache budget, which may not be optimal compared to hybrid dense-sparse layers. The method is designed as a post-training retrofit for dense attention Transformers and is not a from-scratch compressed architecture. Additionally, the evaluation is heavily skewed towards mathematical reasoning; while generalization is tested, the primary gains are most pronounced in tasks requiring deep, long-range dependency resolution. The reliance on a specific training dataset (Nemotron-Math) might introduce biases, although the generalization results mitigate this concern.
KVpop offers a practical solution to the scalability challenges of long-context LLMs, enabling longer context windows without proportional increases in memory and compute costs. This can lower inference costs and make large models more accessible. By improving the efficiency of autoregressive decoding, it contributes to the broader goal of sustainable and scalable AI deployment. The technique of using future-attention for supervision could inspire new training objectives for other sequence modeling tasks. KVpop introduces a predictive online pruning method for KV cache compression that leverages future-attention supervision and delayed stateful scoring to achieve high-performance, fixed-budget inference, significantly advancing the state of efficient long-context language modeling.
Dense spatial perception is essential for physical intelligence, where visual systems are expected to recover structured, metric, and actionable representations from pixel observations. Modern visual foundation models tend to prioritize semantic invariance, often at the expense of detailed spatial understanding. In this work, we study vision pretraining through a boundary-centric lens, motivated by the premise that boundaries and shape discontinuities offer essential cues for perceiving geometric properties. Concretely, we propose masked boundary modeling, a self-supervised paradigm that dynamically learns sub-pixel boundary representations and subsequently leverages the discovered boundary-bearing tokens as masked targets to facilitate dense visual token learning. By scaling this framework, we develop LingBot-Vision and demonstrate its efficacy across a diverse set of downstream vision tasks with DINOv3 as a strong baseline. Remarkably, LingBot-Vision drives the progression from LingBot-Depth 1.0 to LingBot-Depth 2.0 for depth completion, and thereby yields enhanced depth estimation, a key pillar for embodied artificial intelligence. Our findings reveal that boundary modeling goes beyond simple line segments and instead serves as a scalable pretraining principle for learning spatially structured visual representations.
Primary: Alibaba Group
All Institutions: Alibaba Group
The paper introduces masked boundary modeling, a novel self-supervised pretraining method that enhances dense spatial perception in vision foundation models by learning sub-pixel boundary representations and leveraging efficient parallel a-contrario validation. This approach significantly improves depth estimation and other spatial tasks, offering a scalable and effective way to integrate geometric cues into visual representation learning, with strong implications for embodied AI.
The paper proposes "masked boundary modeling," a self-supervised pretraining paradigm that leverages sub-pixel boundary representations to enhance dense spatial perception. The core innovation lies in using boundaries not just as static labels, but as dynamic targets that guide the learning of geometrically structured visual tokens. The methodology includes a novel decoding mechanism for boundary fields and an efficient, parallelizable implementation of a-contrario validation (based on the Helmholtz principle) to filter meaningful line segments from the learned representations. This approach decouples candidate formation from validation, allowing for GPU-accelerated processing within the training loop, which is a significant engineering and algorithmic improvement over classical sequential detectors like LSD. The method is designed to complement semantic invariance in foundation models by explicitly encoding shape and metric cues.
The authors evaluate LingBot-Vision against strong baselines, specifically DINOv3, across a diverse set of downstream vision tasks. Key results include the progression from LingBot-Depth 1.0 to LingBot-Depth 2.0, demonstrating significant improvements in depth completion and estimation. The experiments suggest that boundary-centric pretraining yields representations that are more effective for tasks requiring dense spatial understanding compared to standard semantic pretraining. The scaling analysis indicates that the benefits of this approach persist and potentially grow with model size, supporting the claim that boundary modeling is a scalable pretraining principle. The empirical evidence is robust, showing consistent gains in metric accuracy where semantic models typically falter.
The paper provides detailed descriptions of the boundary field definition, sampling procedures, and the a-contrario validation framework. Implementation constants, such as gradient computation methods, rectangle widths, and alignment tolerances, are specified. The appendix offers a clear explanation of the decoding and validation logic, including the parallelization strategy. While the full codebase is not explicitly linked in the provided text, the methodological details are sufficiently rigorous to allow for independent implementation. The use of standard components (e.g., DINOv3 as a baseline) further aids reproducibility.
The paper focuses heavily on the integration of boundary cues into self-supervised learning; it does not extensively discuss the computational overhead of the boundary decoding and validation steps during pretraining, although it claims efficiency. The reliance on corner points for anchoring might introduce biases in textureless or repetitive regions where corners are sparse. Additionally, the generalization of these boundary-enhanced representations to tasks with minimal geometric structure (e.g., pure classification) is not the primary focus, and their impact there might be neutral or slightly negative due to the added complexity. The evaluation is primarily on depth and spatial tasks; broader evaluation on other dense prediction tasks (e.g., optical flow, segmentation) would strengthen the claims.
This work contributes to the development of embodied AI and robotics by providing visual foundation models with better spatial understanding. Improved depth estimation and dense spatial perception are critical for navigation, manipulation, and interaction with the physical world. By making spatial cues a first-class citizen in pretraining, the paper helps bridge the gap between high-level semantic understanding and low-level geometric reasoning. This could lead to more robust and capable autonomous systems. The method also offers a new perspective on self-supervised learning, suggesting that geometric priors can be effectively learned and utilized alongside semantic features. The paper introduces masked boundary modeling, a novel self-supervised pretraining method that enhances dense spatial perception in vision foundation models by learning sub-pixel boundary representations and leveraging efficient parallel a-contrario validation. This approach significantly improves depth estimation and other spatial tasks, offering a scalable and effective way to integrate geometric cues into visual representation learning, with strong implications for embodied AI.
Multi-vector vision-language retrieval preserves fine-grained visual evidence through maximum-similarity late interaction, but dense image-side tokens make storage and scoring expensive. Existing token compression methods reduce this cost, yet they can remove or collapse object- and region-level evidence that future query tokens may need to select. We propose SaMer, an object-aware token merging framework that compresses image-side post-projector tokens into K representative centroids while preserving the original late-interaction interface. SaMer uses object annotations only during training as a merge prior to discourage cross-instance mixing, requires no ground-truth bounding boxes or detectors at inference time, and adapts only the shared projection layer with frozen vision and language backbones. With K=64, SaMer removes more than 93% of image-side tokens and reduces ColPali storage by 16.09times, while improving R@1 on Flickr30K and MSCOCO. These gains arise because object-aware merging preserves query-selectable object evidence that pruning or feature-only pooling can remove or collapse. SaMer also outperforms compression baselines and shows stronger phrase-level grounding, suggesting that efficient multi-vector retrieval depends not only on reducing token count, but on preserving the evidence future query tokens need to select.
Primary: Seoul National University Hospital
All Institutions: Seoul National University Hospital, National Research Foundation of Korea, Korea Health Industry Development Institute
SaMer makes a significant contribution to efficient vision-language retrieval, a critical component for large-scale image search, retrieval-augmented visual question answering, and other multimodal applications. By substantially reducing storage and scoring costs (16x storage, 4-9x QPS) *while improving accuracy*, SaMer enables the deployment of fine-grained multi-vector retrieval systems at scales previously impractical. The insight that "efficient multi-vector retrieval depends not only on reducing token count, but on preserving the evidence future query tokens need to select" is valuable and could guide future research in multimodal compression. The method's plug-and-play nature with frozen backbones lowers the barrier to adoption. This work could lead to more accessible and performant VLM-powered retrieval systems, democratizing access to fine-grained visual search capabilities. This paper introduces SaMer, an object-aware token merging framework that significantly enhances the efficiency and accuracy of multi-vector vision-language retrieval. The work presents a novel training-time object-aware prior for post-projector token merging, enabling substantial compression (16x storage reduction, 4-9x QPS improvement) while simultaneously boosting retrieval performance and improving phrase-level grounding, thereby offering a practical and principled solution to a critical bottleneck in large-scale multimodal retrieval systems.
The paper proposes SaMer, an object-aware token merging framework for efficient multi-vector vision-language retrieval. The core idea is to compress dense image-side post-projector tokens into K representative centroids while preserving fine-grained visual evidence. The methodology is well-articulated and technically sound. SaMer employs a feature-spatial soft assignment for merging, where each representative is formed as a normalized weighted centroid. A key innovation is the use of object annotations *only during training* as a merge prior. This prior discourages cross-instance mixing, ensuring that tokens belonging to different object instances are not collapsed into the same representative. This is a clever way to leverage supervision without requiring it at inference time. The projection-only adaptation strategy is also well-designed, freezing the heavy vision and language backbones and only training the shared projection layer to align the compressed tokens with the retrieval space. This makes SaMer pluggable into existing multi-vector retrievers like ColPali and ColQwen2. The algorithm for compression-aware adaptation and inference is clearly presented, highlighting the distinction between training (with object prior) and inference (bbox-free). The choice of soft assignment for centroid construction is empirically justified in the appendix.
The experimental evaluation is comprehensive and rigorous. SaMer is evaluated on four diverse benchmarks: Flickr30K, MSCOCO (natural image retrieval), ImageCoDe (compositional retrieval), and DocVQA (document domain). This broad evaluation demonstrates the method's robustness and identifies its strengths and limitations (e.g., less optimized for DocVQA's sparse textual evidence). The comparison includes a wide range of baselines: single-vector, VLM-based, multi-vector, and retrieval-side compression methods (H-Pool, HPC, SAP). Crucially, comparisons with compression baselines are controlled, using the same backbones and token budget (K=64). The results are compelling: SaMer achieves significant improvements in R@1 on Flickr30K (e.g., ColPali R@1 from 77.0 to 82.4) and MSCOCO, outperforming all compression baselines. It also shows strong performance on ImageCoDe, validating its ability to preserve subtle object/attribute cues. The analysis section is particularly strong, featuring: 1. **Ablation Study**: Clearly demonstrates that the gains are not solely from adaptation but from the object-aware merge design. 2. **Compression Budget Analysis**: Identifies K=64 as an optimal operating point, balancing performance and compression. 3. **Merge Component Study**: Highlights the critical role of the object-aware prior, showing it significantly improves both retrieval and grounding metrics (BoxMass, RegionHit, CoverageIoU). 4. **Grounding Comparison**: Uses novel grounding metrics to quantitatively show that SaMer better concentrates phrase-level relevance inside annotated object regions, a key claim of the paper. Qualitative examples further support this. 5. **Efficiency Analysis**: Quantifies substantial storage reduction (e.g., 16.09x for ColPali) and QPS improvements (4.3x to 9.1x), directly addressing the practical problem of expensive multi-vector retrieval. The empirical evidence strongly supports the paper's claims.
The paper provides a code link (https://github.com/dmis-lab/SaMer), which is excellent for reproducibility. Implementation details, including optimization steps, learning rate, weight decay, scheduler, and GPU setup, are provided in the appendix. The object annotation processing and grounding metric definitions are also clearly detailed. The algorithm for SaMer's training and inference is explicitly laid out. This level of detail and code availability makes the work highly reproducible.
1. **Dependency on Object Annotations for Training**: While the method cleverly avoids inference-time annotations, it still requires object bounding box annotations during training. For domains where such annotations are scarce or expensive, adapting SaMer might be challenging. 2. **Performance on DocVQA**: Although competitive, SaMer is not optimized for document images where sparse OCR tokens and layout cues are crucial. This highlights a domain-specific limitation, as the object-aware prior is less relevant for text-heavy documents. 3. **Hyperparameter Sensitivity**: The spatial term weight (lambda) and soft-assignment temperature (tau_s) are hyperparameters. While not explicitly discussed as a limitation, their tuning might be sensitive for new datasets or backbones. 4. **Generalizability of Object-Awareness**: The "object-aware" prior is based on explicit object instances. For more abstract attributes or relations that don't map cleanly to single object bounding boxes, the effectiveness of this prior might diminish.
SaMer makes a significant contribution to efficient vision-language retrieval, a critical component for large-scale image search, retrieval-augmented visual question answering, and other multimodal applications. By substantially reducing storage and scoring costs (16x storage, 4-9x QPS) *while improving accuracy*, SaMer enables the deployment of fine-grained multi-vector retrieval systems at scales previously impractical. The insight that "efficient multi-vector retrieval depends not only on reducing token count, but on preserving the evidence future query tokens need to select" is valuable and could guide future research in multimodal compression. The method's plug-and-play nature with frozen backbones lowers the barrier to adoption. This work could lead to more accessible and performant VLM-powered retrieval systems, democratizing access to fine-grained visual search capabilities. This paper introduces SaMer, an object-aware token merging framework that significantly enhances the efficiency and accuracy of multi-vector vision-language retrieval. The work presents a novel training-time object-aware prior for post-projector token merging, enabling substantial compression (16x storage reduction, 4-9x QPS improvement) while simultaneously boosting retrieval performance and improving phrase-level grounding, thereby offering a practical and principled solution to a critical bottleneck in large-scale multimodal retrieval systems.
Logit-based watermarking is a widely used mechanism for identifying LLM generated content, yet its effectiveness is governed by a fundamental trade-off between detectability and semantic distortion. Existing analyses provide limited guidance for principled hyperparameter selection, leaving practical deployments reliant on heuristic tuning. In this work, we develop a power-calibrated statistical framework that establishes explicit quantitative relationships between watermark hyperparameters, detection power, and distortion. This characterization transforms watermark design into a guided optimization problem. Building on these results, we derive practical parameter selection procedures that achieve optimal tradeoffs under constraints. Extensive experiments across multiple language models and datasets validate the theory and demonstrate that the proposed framework consistently identifies Pareto-optimal points.
Primary: Department of Statistics, Pennsylvania State University, University Park
All Institutions: Department of Statistics, Pennsylvania State University, University Park
The paper's broader impact is significant and directly addresses critical societal concerns related to the proliferation of large language models. 1. **Combating Misinformation**: By improving the efficiency and reliability of LLM watermarking, this work strengthens the ability to identify machine-generated content. This is crucial in the fight against misinformation, fake news, and propaganda campaigns that leverage generative AI. 2. **Academic Integrity**: Watermarking can help detect AI-generated content in academic submissions, supporting academic honesty and preventing misuse of LLMs for plagiarism or automated assignment completion. 3. **Ethical AI Deployment**: It provides a mechanism for establishing provenance, which is essential for responsible and transparent deployment of generative AI. Users can be informed whether content originates from an AI, fostering trust and accountability. 4. **Scalability and Practicality**: By transforming watermark design from heuristic tuning to a statistically grounded optimization problem, the framework makes watermarking systems more robust, reliable, and easier to deploy in real-world applications. This improved efficiency and principled parameter selection can lead to wider adoption and more effective use of watermarking technologies. 5. **Research Foundation**: The rigorous statistical framework provides a strong theoretical foundation for future research in LLM watermarking, enabling more principled development and analysis of new techniques. It encourages a shift from empirical trial-and-error to theoretically informed design. The work directly contributes to mitigating potential harms of generative AI while enabling its beneficial uses, aligning with responsible AI development principles. This paper introduces a controllable statistical framework for logit-based LLM watermarking, enabling principled calibration of watermark strength under explicit detectability and distortion objectives. By establishing quantitative mappings between watermark parameters, detection power, and KL-based distortion, the authors transform watermark design from heuristic tuning into a statistically grounded optimization problem, validated through extensive experiments across multiple language models and datasets, consistently identifying Pareto-optimal configurations.
The paper introduces a power-calibrated statistical framework for logit-based LLM watermarking, moving beyond heuristic hyperparameter tuning. The core methodology involves establishing explicit quantitative relationships between watermark hyperparameters (bias $\delta$ and green-list fraction $\gamma$), detection power, and semantic distortion (measured by KL divergence). Key methodological steps include: 1. **Formalizing Hypothesis Testing**: The paper frames watermark detection as a hypothesis test, defining the null ($H_0$: unwatermarked text) and alternative ($H_1$: watermarked text) hypotheses based on the green-list token probability. 2. **Assumptions for Tractability**: To derive closed-form expressions, the framework relies on several assumptions: * **Random green-list assignments**: Green lists are i.i.d. and independent of the generated token sequence, simplifying the null hypothesis to an i.i.d. Bernoulli process for green-token indicators. * **Non-informative NTP prior**: The next-token probability (NTP) vector is modeled as an independent draw from a uniform Dirichlet distribution. The paper acknowledges this is a simplification but argues for its effectiveness due to the detector's dependence on green-list mass, which concentrates around $\gamma$. * **Information Decay**: Assumes a geometric decay in mutual information between past and future green-token indicators, enabling the application of a Central Limit Theorem for $\alpha$-mixing sequences under the alternative hypothesis. * **Non-degenerate Long-run Variance**: Ensures the asymptotic variance of the indicator sequence is strictly positive. 3. **Derivation of Key Metrics**: * **Green-token probability under watermarking ($\gamma'$)**: Lemma 2.2 and Theorem 2.3 provide a precise, token-level description of how watermarking biases generation toward the green list, leading to a closed-form expression for $\gamma'$ in terms of $\delta$ and $\gamma$. * **Detection Power**: Using the normal approximation for the aggregate statistic $S_n$ under both $H_0$ and $H_1$, a closed-form expression for statistical power $\Psi^*(\delta, \gamma)$ is derived. The paper notes that a constant $c$ (long-run variance inflation) affects the numerical value but not the parameter selection for maximization. * **Distortion (KL Divergence)**: Lemma 3.1 provides a plug-in formula for the expected token-wise KL divergence $D_{KL}(\delta, \gamma)$ between watermarked and unwatermarked distributions. It proves strict monotonicity in $\delta$ for fixed $\gamma$, allowing $\delta$ to be parameterized by a distortion budget. 4. **Optimization Framework**: The theoretical characterization transforms watermark design into a guided optimization problem. Instead of tuning two hyperparameters $(\delta, \gamma)$, the problem is reduced to a 1D numerical optimization (e.g., maximizing power subject to a KL distortion budget $K_0$) because $\delta$ can be implicitly determined by $K_0$. Practical guidance is provided for initializing the search for $\gamma$. The methodology is statistically rigorous, building on established theorems (CLT for mixing sequences) and providing detailed proofs in the appendix. The reduction of a multi-dimensional heuristic search to a principled 1D optimization problem under constraints is a significant practical contribution.
The experimental evaluation is comprehensive and well-structured, providing strong empirical validation for the theoretical framework. 1. **Setup**: The protocol largely follows prior watermarking evaluations, using diverse LLMs (OPT, Pythia, GPT-2, and Gemma-2 9B in the appendix) and datasets (C4, LFQA, Wikipedia). Generations are kept short ($n=50$ tokens) to avoid trivial detectability gains and better expose statistical efficiency differences. All methods are implemented using a unified pipeline based on the KGW codebase to ensure fair comparison. 2. **Distributional Verification**: * **Normality**: Q-Q plots confirm that the standardized green-token count statistic closely follows a standard normal distribution under both $H_0$ and $H_1$, supporting the normal approximation used in the detectability analysis. * **Alternative Hypothesis Characterization**: Empirical green-token rates are compared against theoretical predictions from Theorem 2.3. A near-linear relationship with $R^2 > 0.98$ across all settings validates the simplified modeling assumptions for $\gamma'$. 3. **Performance Evaluation (Detectability-Distortion Trade-off)**: * The paper evaluates statistical power (TPR at $\alpha=0.05$) against semantic distortion (per-token KL divergence). * Results across various models and datasets consistently show that the proposed method lies on the Pareto frontier. It achieves high detection power at substantially lower distortion compared to baseline methods (KGW, DP, and dense grid search). * The approach maintains near-saturated TPR in moderate distortion regimes where baselines exhibit significant degradation. This highlights improved statistical efficiency. 4. **Robustness under Alternative Quality Metrics**: * To assess the generalizability of the optimized parameters, the method is evaluated using complementary semantic quality metrics: BLEU, ROUGE, and BERTScore. * The results show a consistent pattern: the proposed method maintains strong detection power over a wider range of quality values compared to baselines. This suggests that the optimized parameterization induces a more uniform and controlled distributional shift, not tied to a specific notion of text similarity. Overall, the experiments are thorough, well-designed, and effectively support the theoretical claims. The validation of assumptions, extensive comparisons, and robustness checks significantly strengthen the paper's conclusions.
The paper demonstrates a strong commitment to reproducibility. 1. **Code Availability**: The authors explicitly state that "The implementation is available at https://github.com/shooof/wm." This is a crucial step for reproducibility. 2. **Detailed Experimental Setup**: The "Experimental Setup" section provides clear details on the language models (OPT, Pythia, GPT-2, Gemma-2 9B), datasets (C4, LFQA, Wikipedia), generation length ($n=50$), and detection parameters ($\alpha=0.05$). 3. **Baseline Descriptions**: Baseline methods (KGW, DP, dense grid search) are clearly identified, and their implementation is stated to use the same unified generation and detection pipeline as the proposed method, controlling for implementation effects. 4. **Hyperparameter Ranges**: Appendix provides a table of hyperparameter ranges used for all methods, which is essential for replicating the search space. 5. **Metric Definitions**: Clear definitions and implementations for distortion (KL divergence) and quality metrics (BLEU, ROUGE, BERTScore) are provided. 6. **Theoretical Proofs**: The appendix contains detailed proofs for all lemmas and theorems, allowing for independent verification of the mathematical derivations. The combination of open-source code, detailed experimental descriptions, and theoretical proofs makes the work highly reproducible.
The paper acknowledges limitations, with a dedicated discussion mentioned in Appendix [REF] (though the provided text truncates before this appendix content). Based on the main text, potential limitations include: 1. **Assumptions for Tractability**: The framework relies on several simplifying assumptions: * **Random green-list assignments**: While reasonable for pseudo-random hash functions, real-world implementations might have subtle biases. * **Non-informative NTP prior (Uniform Dirichlet)**: The paper argues for its effectiveness, but it is an approximation. While it mentions that richer priors (mixtures of Dirichlet) could be used if structural information is available, the current analysis is based on a simplified prior. The accuracy of this approximation for all possible LLM output distributions might vary. * **Information Decay**: The assumption of geometric decay in mutual information is a model for the autoregressive dependence. While standard for mixing sequences, its precise fit to complex LLM generation dynamics is an approximation. 2. **Estimation of Constant 'c'**: The power formula includes a constant $c$ that captures long-run variance inflation due to dependence. While the paper states that parameter selection is independent of $c$, its numerical value is needed for absolute power predictions and needs to be estimated in practice (as mentioned in Appendix [REF]). This introduces an additional practical step and potential source of error. 3. **Focus on Logit-Based Watermarking**: The framework is specifically developed for logit-based watermarking (KGW framework). While this is a widely adopted paradigm, the theoretical derivations might not directly apply to other watermarking schemes without significant adaptation. 4. **Short Sequence Evaluation**: While justified to isolate statistical efficiency, the primary experiments are on short sequences ($n=50$). While the paper discusses long-form generation in the appendix, the main empirical results are on short sequences. 5. **Adversarial Robustness**: While the appendix briefly discusses "signal-removal robustness" using a WinMax-C detector, a more extensive analysis of various adversarial attacks (e.g., paraphrasing, fine-tuning, token manipulation) and how the calibrated parameters perform under these attacks would be valuable. The current framework optimizes for detectability and distortion in a "clean" setting.
The paper's broader impact is significant and directly addresses critical societal concerns related to the proliferation of large language models. 1. **Combating Misinformation**: By improving the efficiency and reliability of LLM watermarking, this work strengthens the ability to identify machine-generated content. This is crucial in the fight against misinformation, fake news, and propaganda campaigns that leverage generative AI. 2. **Academic Integrity**: Watermarking can help detect AI-generated content in academic submissions, supporting academic honesty and preventing misuse of LLMs for plagiarism or automated assignment completion. 3. **Ethical AI Deployment**: It provides a mechanism for establishing provenance, which is essential for responsible and transparent deployment of generative AI. Users can be informed whether content originates from an AI, fostering trust and accountability. 4. **Scalability and Practicality**: By transforming watermark design from heuristic tuning to a statistically grounded optimization problem, the framework makes watermarking systems more robust, reliable, and easier to deploy in real-world applications. This improved efficiency and principled parameter selection can lead to wider adoption and more effective use of watermarking technologies. 5. **Research Foundation**: The rigorous statistical framework provides a strong theoretical foundation for future research in LLM watermarking, enabling more principled development and analysis of new techniques. It encourages a shift from empirical trial-and-error to theoretically informed design. The work directly contributes to mitigating potential harms of generative AI while enabling its beneficial uses, aligning with responsible AI development principles. This paper introduces a controllable statistical framework for logit-based LLM watermarking, enabling principled calibration of watermark strength under explicit detectability and distortion objectives. By establishing quantitative mappings between watermark parameters, detection power, and KL-based distortion, the authors transform watermark design from heuristic tuning into a statistically grounded optimization problem, validated through extensive experiments across multiple language models and datasets, consistently identifying Pareto-optimal configurations.
Unified models for robot manipulation aim to equip one policy with both the semantic priors of pretrained VLMs and the physical dynamics learned through future prediction. In practice, existing designs tend to erode the semantics of the pretrained backbone, suffer interference among heterogeneous objectives, and learn future prediction from scratch in pixel space, leaving the dynamics priors of pretrained video generators unexploited. We present InternVLA-A1.5, which builds the policy on a native VLM backbone that keeps training on VQA and subtask prediction, and attaches a lightweight unified expert for continuous action generation. Future prediction is recast as a latent-querying problem, where a small set of learnable foresight tokens condenses the task-relevant future into a compact latent code under the supervision of a frozen pretrained video generation model, so the policy inherits world-model dynamics priors without ever learning pixel-level generation. The video branch is discarded at inference, keeping real-time control. Pretrained on 1.2M robot episodes and 3M multimodal samples, InternVLA-A1.5 achieves the best overall results on all six simulation benchmarks. In the real world, the preserved semantics deliver the strongest compositional generalization on held-out instruction bindings, and the two designs together sustain long-horizon execution.
Primary: Shanghai AI Laboratory
All Institutions: Shanghai AI Laboratory, Shanghai Jiao Tong University, Zhejiang University, University of Oxford, University of Cambridge, University of Edinburgh, University of Tokyo, University of Toronto, University of California, Berkeley, University of Washington, University of Michigan, University of Illinois Urbana-Champaign, University of Texas at Austin, University of Southern California, University of California, Los Angeles, University of California, San Diego, University of California, Irvine, University of California, Santa Barbara, University of California, Riverside, University of California, Merced, University of California, Santa Cruz, University of California, Davis, University of California, Santa Barbara, University of California, San Diego, University of California, Irvine, University of California, Los Angeles, University of California, Berkeley, University of Washington, University of Michigan, University of Illinois Urbana-Champaign, University of Texas at Austin, University of Southern California, University of California, Los Angeles, University of California, San Diego, University of California, Irvine, University of California, Santa Barbara, University of California, Riverside, University of California, Merced, University of California, Santa Cruz, University of California, Davis
InternVLA-A1.5 presents a significant advancement in Vision-Language-Action models by effectively unifying semantic understanding with efficient latent-based world modeling, achieving state-of-the-art results in simulation and demonstrating strong compositional generalization in real-world settings. The technical contribution lies in the novel "latent foresight" mechanism and the training strategy that prevents semantic erosion, offering a scalable and effective path forward for unified robot policies. While the score is high (76), it reflects the strong empirical results and the clear technical merit of the approach, though it may not yet be considered a "once-a-decade" breakthrough given the incremental nature of many VLA improvements. However, the combination of performance and efficiency makes it a highly impactful paper for the robotics community.
The paper proposes InternVLA-A1.5, a unified Vision-Language-Action model designed to address the "semantic erosion" and "objective interference" common in existing VLA architectures. The core methodological innovation lies in two areas: 1) A training strategy that preserves the semantic capabilities of the pretrained VLM backbone by continuing to train on VQA and subtask prediction tasks alongside robot control, rather than freezing or discarding these capabilities. 2) A "latent foresight" mechanism where future prediction is formulated as a latent-querying problem. Instead of learning pixel-level video generation (which is computationally expensive and often unstable), the model uses a small set of learnable tokens to condense task-relevant future dynamics into a compact latent code, supervised by a frozen pretrained video generation model. This allows the policy to inherit world-model dynamics priors without the overhead of pixel reconstruction. The architecture attaches a lightweight unified expert for continuous action generation, discarding the video branch at inference to ensure real-time control. This approach is technically sound and addresses specific pain points in the VLA literature (semantic drift and computational inefficiency of world models).
The evaluation is comprehensive, covering six simulation benchmarks and real-world experiments. The paper reports state-of-the-art results on all six simulation benchmarks, which is a strong indicator of performance. The real-world experiments highlight the model's compositional generalization on held-out instruction bindings, a critical capability for practical deployment. The ability to sustain long-horizon execution is also demonstrated. The use of 1.2M robot episodes and 3M multimodal samples for pretraining provides a solid data foundation. The comparison against existing baselines (likely including RT-2, OpenVLA, and other VLA variants) shows competitive or superior performance, particularly in generalization scenarios. The real-world results are crucial for validating the simulation findings, and the paper appears to provide robust evidence here.
The paper mentions the scale of data (1.2M episodes, 3M multimodal samples) and the use of pretrained components (VLM backbone, frozen video generator). However, as an arXiv preprint, full code and model weights are not explicitly linked in the provided text (though often available via project pages not cited in the abstract). The description of the "latent-querying" mechanism and the specific VQA/subtask prediction losses is detailed enough to allow for replication by researchers with significant resources. The reliance on a "frozen pretrained video generation model" requires specifying which model (e.g., Stable Video Diffusion, Sora-like models, or a specific open-source variant), which is a potential ambiguity if not clearly defined in the full text. Assuming standard practices, reproducibility is moderate to high, contingent on access to the specific pretrained video models and the large-scale robot dataset.
The paper does not explicitly discuss the limitations of the "latent foresight" approach. Potential limitations include: 1) The dependency on the quality of the frozen video generation model for supervision; if the video model fails to capture certain dynamics, the foresight tokens may learn suboptimal representations. 2) The computational cost of training with the auxiliary video generation loss, even if the video branch is discarded at inference. 3) The generalization to out-of-distribution environments or objects not seen during the 1.2M episode pretraining. 4) The "lightweight unified expert" might become a bottleneck for very complex manipulation tasks requiring fine-grained control, although the paper claims success in long-horizon execution. 5) The real-world results are likely limited to specific setups, and generalization to arbitrary home environments remains a challenge.
This work contributes to the broader goal of creating general-purpose robot manipulators that can understand natural language instructions and generalize to new tasks. By improving compositional generalization and reducing the computational overhead of world models, it makes VLA policies more practical for real-world deployment. The emphasis on preserving semantic priors aligns with the trend of leveraging large-scale pretrained models for robotics. However, the deployment of such powerful robots in unstructured environments raises safety and ethical considerations, which are not addressed in the paper. The potential for misuse in autonomous systems is a standard concern in robotics research. InternVLA-A1.5 presents a significant advancement in Vision-Language-Action models by effectively unifying semantic understanding with efficient latent-based world modeling, achieving state-of-the-art results in simulation and demonstrating strong compositional generalization in real-world settings. The technical contribution lies in the novel "latent foresight" mechanism and the training strategy that prevents semantic erosion, offering a scalable and effective path forward for unified robot policies. While the score is high (76), it reflects the strong empirical results and the clear technical merit of the approach, though it may not yet be considered a "once-a-decade" breakthrough given the incremental nature of many VLA improvements. However, the combination of performance and efficiency makes it a highly impactful paper for the robotics community.
Diffusion large language models (dLLMs) generate text by iteratively denoising a masked sequence, offering a parallel alternative to autoregressive models, but eliciting strong reasoning through post-training remains difficult: supervised fine-tuning is off-policy and suffers from exposure bias, while reinforcement learning gives only sparse, sequence-level rewards and is hard to apply without tractable sequence likelihoods. On-policy self-distillation (OPSD) offers a promising alternative, using one model as both student and teacher to provide dense, token-level, on-policy supervision, but its effectiveness hinges on giving the teacher privileged information (PI) - typically an instance-specific ground-truth reference unavailable at inference - so the student ends up distilling a weak PI-free consensus policy that yields little improvement on dLLM reasoning. We introduce dOPSD, which instead derives the teacher's privilege directly from the student's own denoising trajectory, evaluating masked positions using later, more-decoded steps of that same trajectory rather than an external label, so the teacher's advantage emerges from the model's own decoding process; on Dream and LLaDA, dOPSD improves both in-domain math reasoning and out-of-domain code generation, outperforming supervised and on-policy baselines.
Primary: unknown
All Institutions: unknown
dOPSD offers a significant advancement in post-training techniques for diffusion language models, particularly for enhancing their reasoning capabilities. By providing a dense, on-policy, and self-generated supervision signal, it opens new avenues for improving dLLMs without relying on costly human annotations for privileged information or complex reinforcement learning setups. This could accelerate the development of more capable and robust dLLMs for tasks requiring multi-step reasoning, such as mathematical problem-solving, code generation, and potentially other scientific domains. The concept of leveraging a model's *own internal generation process* to create a stronger self-supervision signal is a powerful idea that could inspire similar approaches in other generative model paradigms. Its label-efficient nature (only requiring final answers for verification, not full CoT) makes it applicable to a wider range of datasets, potentially democratizing access to advanced post-training for dLLMs. The strong out-of-domain generalization also suggests that the method fosters more fundamental reasoning skills rather than just memorization, which is a crucial step towards more intelligent AI systems. This paper introduces dOPSD, a novel on-policy self-distillation method for diffusion language models that derives privileged information from the model's own denoising trajectory, significantly enhancing reasoning and generalization capabilities. The work presents a highly innovative approach to post-training dLLMs, cleverly leveraging their iterative decoding process to generate a dense, on-policy, and self-generated supervision signal. By addressing the fundamental limitations of existing self-distillation and reinforcement learning methods for dLLMs, dOPSD achieves substantial and consistent performance improvements across both in-domain mathematical reasoning and out-of-domain code generation, outperforming all baselines. The methodology is elegant, well-supported by comprehensive experiments, thorough ablation studies, and insightful qualitative examples, making it a significant technical contribution with strong potential for field-wide impact in the development of more capable and robust diffusion-based generative models.
The methodology of dOPSD is exceptionally well-conceived and addresses fundamental challenges in post-training diffusion language models (dLLMs). The central insight—that a dLLM's own denoising trajectory inherently provides a ladder of progressively more informed contexts—is a brilliant way to generate "privileged information" (PI) internally. This elegantly sidesteps the "PI-free collapse" issue that plagues traditional On-Policy Self-Distillation (OPSD) when the PI is an external, instance-specific label. By using later, more-decoded steps of the *same* trajectory as a teacher signal for earlier, more-masked steps, dOPSD creates a dense, token-level, on-policy supervision signal that is both self-generated and highly relevant. The student is trained on genuine intermediate decoding states, avoiding the "off-path" problem of random masking. The use of a generalized Jensen-Shannon divergence for distillation is appropriate, and the ablation study confirms the superiority of forward KL, which is mass-covering and thus better for absorbing the teacher's full signal. The rollout verification mechanism, which discards incorrect trajectories, is a practical and crucial addition that ensures the quality of the self-distillation signal, although the ablations show the method still provides gains even without it, highlighting the strength of the trajectory-derived PI itself. The fact that the method requires no architectural changes and integrates directly into the standard dLLM forward pass is a significant advantage, promoting ease of implementation and adoption.
The experimental evaluation is robust and comprehensive, providing strong evidence for dOPSD's effectiveness. The choice of two distinct, instruction-tuned dLLM backbones (Dream-7B-Instruct and LLaDA-8B-Instruct) demonstrates the method's generalizability. Training on a mathematical reasoning corpus (MixChain-Z-PRM12K) and evaluating on both in-domain (GSM8K, MATH500) and out-of-domain (HumanEval, MBPP for code generation) tasks is a rigorous test of performance and generalization. The comparison against a diverse set of baselines—SFT, GRPO (an RLVR adaptation for dLLMs), and two OPSD variants (answer-only and full-solution)—is particularly insightful. The results unequivocally show dOPSD outperforming all baselines and the untuned base models across all metrics, often by substantial margins. Crucially, most baselines either fail to improve or actively *degrade* performance, validating the paper's analysis of their limitations for dLLMs. The ablation studies are well-designed, systematically exploring key hyperparameter choices (KL divergence, teacher horizon, mask threshold) and the impact of rollout verification. These studies not only optimize the method but also provide valuable insights into its underlying mechanisms. The qualitative examples further enhance the evaluation by concretely illustrating how dOPSD avoids specific reasoning pitfalls that trap other methods. The detailed reporting of LoRA settings, training parameters, and inference hyperparameters (including diffusion steps, temperature, top-p, and block length for LLaDA) is commendable and aids in reproducibility.
The paper provides a high level of detail regarding the experimental setup, which should facilitate reproducibility. It specifies the base models, training dataset, evaluation benchmarks, LoRA parameters (rank, alpha, target modules), optimization details (AdamW, learning rate, epochs, batch size), and all critical inference hyperparameters for each model and benchmark (max new tokens, diffusion steps, temperature, top-p, top-k, block length for LLaDA). The algorithm for dOPSD training is clearly outlined. The only minor missing piece for full reproducibility would be the exact code implementation or a link to a repository, but given the detailed descriptions, a competent researcher should be able to replicate the core findings.
One potential limitation, though partially addressed, is the reliance on a "rollout verifier" that uses a gold answer. While the paper demonstrates that dOPSD still provides gains without verification, the best performance is achieved with it. This implies that for datasets where *no* final answer is available for verification, the method's full potential might not be realized. The method is inherently tied to the iterative denoising process of diffusion models, meaning its core "trajectory-derived privilege" mechanism is not directly transferable to autoregressive models without significant adaptation or a re-framing of their generation process. While the paper shows strong generalization to out-of-domain code generation, further exploration of its performance on other complex reasoning tasks beyond math (e.g., scientific reasoning, logical puzzles) would be beneficial to fully characterize its capabilities. The computational cost of the teacher passes (one teacher forward per distinct remaining step) is mentioned, but a more detailed analysis of its overhead compared to baselines or standard training might be useful.
dOPSD offers a significant advancement in post-training techniques for diffusion language models, particularly for enhancing their reasoning capabilities. By providing a dense, on-policy, and self-generated supervision signal, it opens new avenues for improving dLLMs without relying on costly human annotations for privileged information or complex reinforcement learning setups. This could accelerate the development of more capable and robust dLLMs for tasks requiring multi-step reasoning, such as mathematical problem-solving, code generation, and potentially other scientific domains. The concept of leveraging a model's *own internal generation process* to create a stronger self-supervision signal is a powerful idea that could inspire similar approaches in other generative model paradigms. Its label-efficient nature (only requiring final answers for verification, not full CoT) makes it applicable to a wider range of datasets, potentially democratizing access to advanced post-training for dLLMs. The strong out-of-domain generalization also suggests that the method fosters more fundamental reasoning skills rather than just memorization, which is a crucial step towards more intelligent AI systems. This paper introduces dOPSD, a novel on-policy self-distillation method for diffusion language models that derives privileged information from the model's own denoising trajectory, significantly enhancing reasoning and generalization capabilities. The work presents a highly innovative approach to post-training dLLMs, cleverly leveraging their iterative decoding process to generate a dense, on-policy, and self-generated supervision signal. By addressing the fundamental limitations of existing self-distillation and reinforcement learning methods for dLLMs, dOPSD achieves substantial and consistent performance improvements across both in-domain mathematical reasoning and out-of-domain code generation, outperforming all baselines. The methodology is elegant, well-supported by comprehensive experiments, thorough ablation studies, and insightful qualitative examples, making it a significant technical contribution with strong potential for field-wide impact in the development of more capable and robust diffusion-based generative models.
Recent advances in multimodal foundation models and agent systems have driven GUI agents from single-platform task execution toward cross-platform interaction. However, building multi-platform GUI agents remains challenging. On one hand, high-quality and executable cross-platform interaction trajectories are still scarce, and existing data often suffer from limited platform coverage. On the other hand, different platforms exhibit distinct interaction conventions, making joint or continual training prone to behavioral pattern mixing, platform-specific capability degradation, and catastrophic forgetting. To address these challenges, we construct Uni-GUI, a high-quality cross-platform GUI interaction dataset, and propose UI-MOPD, the first method that incorporates multi-teacher on-policy distillation into continual learning for GUI agents. UI-MOPD dynamically selects a platform-specific teacher according to the current environment and transfers platform-specific behavioral priors to a shared policy through platform-conditioned distillation, enabling adaptation to new platforms while preserving capabilities on existing ones. Experiments on OSWorld and MobileWorld show that UI-MOPD achieves task success rates of 38.2% and 12.0%, respectively, demonstrating its effectiveness in balancing cross-platform capability retention and new-platform adaptation. Project page: https://elispectre.github.io/UI-MOPD/.
Primary: Tsinghua University
All Institutions: Tsinghua University, Tsinghua Shenzhen International Graduate School
The paper presents a technically solid and relevant contribution to the field of GUI agents by addressing the critical challenges of continual learning and cross-platform generalization through a novel multi-teacher distillation framework, although the absolute performance gains are moderate and the method's scalability depends on the availability of high-quality platform-specific data.
The paper proposes UI-MOPD, a framework for continual learning in multi-platform GUI agents. The core innovation lies in combining multi-teacher on-policy distillation with continual learning to mitigate catastrophic forgetting and platform-specific capability degradation. The method dynamically selects a platform-specific teacher based on the current environment and transfers behavioral priors to a shared policy via platform-conditioned distillation. This approach addresses the scarcity of high-quality cross-platform trajectories and the challenge of mixing distinct interaction conventions. The methodology is technically sound, leveraging established concepts from reinforcement learning (on-policy distillation) and continual learning (experience replay, regularization) but applying them in a novel architectural configuration for GUI agents.
The authors evaluate UI-MOPD on two major benchmarks: OSWorld and MobileWorld. The reported task success rates are 38.2% on OSWorld and 12.0% on MobileWorld. While these numbers demonstrate effectiveness in balancing retention and adaptation, the absolute performance, particularly on MobileWorld, is modest. The evaluation compares against baselines, presumably single-platform or non-continual multi-platform methods. The results suggest that the proposed method outperforms baselines in terms of stability and cross-platform generalization, but the gap in absolute performance might not be transformative. The experiments are relevant to the current state-of-the-art in GUI agents but do not shatter existing records.
The paper provides a project page and claims to construct a new dataset, Uni-GUI. However, the project URL provided does not explicitly link to a public code repository in the extracted metadata, and the abstract-only score suggests limited prior visibility. Reproducibility will depend heavily on the release of the Uni-GUI dataset and the training code. Given the complexity of multimodal agent training, detailed hyperparameters and environment setups are crucial. The paper likely contains these details, but without public code, full reproducibility is currently unverified.
The primary limitation is the modest absolute performance on MobileWorld (12.0%), which suggests that the method may still struggle with the complexity of mobile GUI interactions or that the dataset quality/coverage is a bottleneck. The dynamic teacher selection mechanism adds computational overhead and complexity to the training loop. Furthermore, the reliance on platform-specific teachers implies that the method's performance is bounded by the quality of the individual platform-specific models, which themselves may suffer from limited data. The "catastrophic forgetting" mitigation is effective but comes at the cost of potential interference between platforms if the distillation is not perfectly calibrated.
This work contributes to the development of more robust and generalizable AI agents capable of operating across diverse digital environments. By addressing the critical issues of data scarcity and catastrophic forgetting in multi-platform settings, it paves the way for more versatile personal assistants. However, the ethical implications of automated GUI interaction, including privacy and security concerns, remain relevant. The construction of Uni-GUI, if released responsibly, could accelerate research in this area by providing a standardized benchmark. The paper presents a technically solid and relevant contribution to the field of GUI agents by addressing the critical challenges of continual learning and cross-platform generalization through a novel multi-teacher distillation framework, although the absolute performance gains are moderate and the method's scalability depends on the availability of high-quality platform-specific data.
We present Wan-Streamer v0.2, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model. v0.2 keeps the v0.1 modeling formulation, but raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS. The higher-resolution stream supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation. To support the larger visual stream without adding user-visible delay, v0.2 keeps the thinker as a single-GPU low-latency path for streaming perception, the short language/state Transformer pass that builds the generation cache, and final decoding. The performer becomes a multi-GPU Ulysses-style context-parallel group for the expensive next-unit latent generation. Each performer rank writes incoming K/V into a pre-sharded local cache. The long high-resolution latent video sequence is split across ranks for denoising and gathered through Ulysses communication, while the much shorter audio latent sequence is generated without sequence sharding. In this split, the thinker's language/state computation reaches the performer only as K/V conditioning, so no separate language sequence has to be communicated inside the performer group. This concentrates additional hardware on visual generation while preserving the compact thinker-performer boundary, keeping total remote interaction latency at approximately 550 ms when a 350 ms bidirectional network budget is included.
Primary: Alibaba Group
All Institutions: Alibaba Group
Wan-Streamer v0.2 achieves a significant systems-level advancement by integrating Ulysses-style context parallelism into a real-time audio-visual streaming framework, enabling a jump to 640p resolution while maintaining sub-200ms model-side latency. This represents a solid engineering contribution to the field of multimodal generation, demonstrating how distributed computing techniques can be adapted to strict latency constraints, although the lack of rigorous quantitative evaluation limits its immediate scientific impact compared to foundational algorithmic papers.
The paper proposes a system-level architectural upgrade to the Wan-Streamer v0.1 model, specifically focusing on increasing visual resolution from 192x336 to 640x368 while maintaining real-time latency constraints (~200ms model-side). The core methodological contribution is a "thinker-performer" split: the "thinker" (single GPU) handles perception, language/state updates, and KV-cache construction, while the "performer" (multi-GPU group) handles the computationally expensive latent video generation using Ulysses-style context parallelism. This approach leverages sequence parallelism for the high-resolution video tokens while keeping audio and language processing unsharded to minimize communication overhead. The novelty lies in the specific application of context parallelism to the denoising step of a real-time streaming diffusion/flow-matching model, effectively decoupling the latency-critical control path from the throughput-heavy generation path. While the parallelism technique itself (Ulysses) is not new, its adaptation to the strict causal, low-latency constraints of end-to-end audio-visual streaming agents is a non-trivial engineering and systems contribution.
The evaluation is primarily qualitative and system-level. The authors report that the model achieves the target latency of ~200ms model-side and ~550ms total remote interaction latency. They provide qualitative visual observations of "scene-grounded mid-shot agents," highlighting improvements in legibility of posture, gaze, and objects compared to v0.1. However, the paper lacks rigorous quantitative benchmarks on visual quality (e.g., FID, CLIP-I, or human preference studies) or dialogue quality metrics. The comparison is largely descriptive ("clearer close-up calls") rather than statistically significant. The absence of ablation studies isolating the impact of the parallelism strategy on latency vs. quality makes it difficult to assess the true efficiency gains beyond the stated latency budget. The experiments are sufficient to demonstrate feasibility but weak in providing comprehensive empirical validation of the quality improvements.
The paper provides a clear description of the serving topology, the thinker-performer split, and the use of Ulysses parallelism. However, it lacks specific implementation details regarding the diffusion/flow-matching model architecture, the exact tokenization scheme for the higher resolution, and the specific hyperparameters used in the parallel communication. Without code or more detailed architectural specifications, reproducing the exact latency and quality results would be challenging. The reliance on a specific "native-streaming formulation" inherited from v0.1 also requires access to the previous version's codebase for full reproducibility.
The primary limitation is the lack of quantitative evaluation. The claims of improved "legibility" and "fidelity" are subjective and not backed by standard metrics. The paper does not discuss the scalability of the performer group (e.g., how latency scales with more GPUs or higher resolutions) or the communication overhead in detail. Furthermore, the "thinker-performer" split introduces a dependency on high-bandwidth, low-latency interconnects between the thinker and performer GPUs, which may limit deployment on standard single-node setups without specialized hardware. The paper also does not address potential degradation in temporal consistency or identity preservation at the higher resolution, which are critical challenges in video generation.
This work contributes to the field of real-time multimodal agents, specifically in making them more visually immersive and interactive. By enabling higher-resolution, scene-grounded interactions, it pushes the boundary of what is possible for digital humans and embodied AI in real-time settings. This has implications for virtual assistants, telepresence, and interactive entertainment. However, the increased computational requirements (multi-GPU setup) may limit accessibility for smaller research groups or edge deployments. The focus on visual fidelity also raises questions about the balance between visual quality and computational efficiency in future real-time systems. Wan-Streamer v0.2 achieves a significant systems-level advancement by integrating Ulysses-style context parallelism into a real-time audio-visual streaming framework, enabling a jump to 640p resolution while maintaining sub-200ms model-side latency. This represents a solid engineering contribution to the field of multimodal generation, demonstrating how distributed computing techniques can be adapted to strict latency constraints, although the lack of rigorous quantitative evaluation limits its immediate scientific impact compared to foundational algorithmic papers.
Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining, yet their generalization remains far more fragile than that of LLMs and VLMs. The prevailing remedy, post-training via supervised fine-tuning or reinforcement learning, improves task-specific performance but narrows the generalist capability that makes pretraining valuable. We identify a key bottleneck: VLA failures stem not only from action generation but also from action evaluation. A diagnostic pass@k study confirms that frozen VLAs already contain competent behaviors in their output distribution, with overall success rates rising from 33% at pass@1 to 92% at pass@32. Inspired by this, we propose SVA (Search, Value, and Act), a simple framework that equips frozen VLA policies with long-term consequence awareness. SVA first uses Monte-Carlo tree search in simulation to fully explore the VLA's output distribution and collect diverse trajectories annotated with empirical returns; this knowledge is then distilled into a lightweight Q-value model that predicts the expected consequence of candidate actions; at deployment, the frozen VLA proposes multiple candidates and the evaluator selects the one with the highest uncertainty-regularized Q-value, requiring no simulator access. By decoupling action proposal from consequence evaluation, SVA preserves the generalization capacity of the VLA backbone while substantially improving task success rates. Experiments across embodied benchmarks show that SVA consistently improves generalization on unseen tasks and exhibits strong test-time scaling behavior. Strikingly, SVA enables a 9B VLA to outperform a 27B VLA by 7 points at 27% lower inference latency, suggesting that scaling test-time evaluation is more cost-effective than scaling model size.
Primary: Unknown
All Institutions: Unknown
SVA has significant positive broader impacts. It offers a practical and cost-effective method to enhance the reliability and generalization of VLA models without requiring expensive fine-tuning of multi-billion-parameter backbones. This can accelerate the deployment of more capable and robust generalist robotic agents in various applications, from household assistance to industrial automation. By reframing VLA failures as an evaluation bottleneck, it shifts research focus towards more efficient test-time scaling strategies, potentially leading to more resource-efficient AI development in robotics. The method itself does not present obvious ethical concerns beyond the general considerations for advanced AI and robotics. This paper presents SVA, a novel framework that significantly enhances the generalization and success rates of frozen Vision-Language-Action (VLA) models by distilling Monte-Carlo tree search knowledge into a lightweight, uncertainty-regularized Q-value model for test-time action evaluation. The work provides a compelling diagnostic study identifying an "evaluation bottleneck" in VLAs, proposes a practical and model-agnostic solution, and rigorously demonstrates consistent performance gains across diverse embodied benchmarks and VLA backbones, notably showing that scaling test-time evaluation can be more cost-effective than scaling model size.
The paper proposes SVA (Search, Value, and Act), a three-stage framework to improve the performance of frozen Vision-Language-Action (VLA) models by addressing an identified "action evaluation bottleneck." The methodology is well-structured and practical. 1. **Search**: This stage utilizes Monte-Carlo Tree Search (MCTS) in simulation to explore the frozen VLA's output distribution. MCTS is a well-established technique, and its application here to generate diverse trajectories annotated with empirical returns for *training an evaluator* (rather than directly acting) is a clever and effective use. It efficiently mines long-term consequence signals. 2. **Value**: The knowledge from MCTS is distilled into a lightweight Q-value model. This model, built on a small VLM backbone (e.g., Qwen3.5-0.8B) with LoRA adapters and an ensemble of MLP value heads, predicts the expected consequence of candidate actions. This distillation is crucial for real-time deployment without simulator access. The use of a special `
The experimental evaluation is exceptionally thorough and rigorous, contributing significantly to the paper's impact. 1. **Diagnostic Pass@k Study**: The initial pass@k study is a strong diagnostic, empirically confirming that frozen VLAs often contain competent actions in their output distribution but struggle with selection. This observation directly motivates SVA and provides a clear problem statement. 2. **Benchmarks**: Evaluation spans a diverse set of embodied benchmarks: EmbodiedBench (EB-Habitat, EB-Navigation for embodied reasoning), SimplerEnv (WidowX manipulation), and RoboTwin 2.0 (bimanual manipulation). This breadth demonstrates the generalizability of SVA across different task structures and action granularities. 3. **VLA Backbones**: SVA is tested with a wide range of VLA backbones, including proprietary (GPT-4o), open-source (Qwen3.5-4B/9B/27B, Gemma-4-E4B-it), and state-of-the-art real-robot policies ($\pi_0$, $\pi_{0.5}$, OpenVLA). This model-agnostic evaluation is crucial and shows SVA's broad applicability. 4. **Results**: SVA consistently delivers substantial performance gains across all benchmarks and backbones (e.g., +15.4 on EB-Habitat, +13.2 on EB-Navigation, +26.4 on Stack Cubes). These improvements are significant and demonstrate the effectiveness of the approach. 5. **Ablation Studies**: Comprehensive ablations confirm the necessity of each SVA component (MCTS, Q-model, multi-candidate selection), providing strong evidence for the design choices. 6. **Scaling Behavior and Cost-Effectiveness**: The analysis of test-time scaling with varying numbers of candidates ($N$) is highly impactful. It shows monotonic gains in success rate with sub-linear growth in inference latency. Crucially, the finding that a 9B VLA with SVA outperforms a 27B VLA by 7 points at 27% lower inference latency is a striking result, suggesting that scaling test-time evaluation is more cost-effective than scaling model size. This is a significant insight for the field. 7. **Qualitative Case Studies**: The case studies effectively illustrate how SVA's Q-model can override myopic policy preferences, leading to more robust and goal-directed behavior in the presence of distractors or complex spatial relations. 8. **Real-Robot Relevance**: The appendix provides a strong argument for the real-robot relevance of the simulation study, citing benchmark design, use of real-robot policies, and simulator-free deployment.
The paper provides excellent details in the appendix to support reproducibility. * **Q-Model Architecture**: Specifics on the Qwen3.5-0.8B initialization, special `
The authors openly acknowledge several limitations: 1. **Decoupled Search and Value Learning**: The current pipeline is staged, meaning MCTS is blind to the evolving Q-model, and the policy doesn't benefit from improved values beyond test-time reranking. This limits the full potential of an online search-and-learning loop. 2. **Reliance on Resettable Simulators**: The Search stage requires a resettable simulator with task-success signals, which restricts applicability to domains without high-fidelity simulation or reward functions. 3. **Sim-Only Evaluation**: All experiments are conducted in simulation, and the calibration of the learned Q-model on physical robots remains untested. This is a common but critical limitation for embodied AI research.
SVA has significant positive broader impacts. It offers a practical and cost-effective method to enhance the reliability and generalization of VLA models without requiring expensive fine-tuning of multi-billion-parameter backbones. This can accelerate the deployment of more capable and robust generalist robotic agents in various applications, from household assistance to industrial automation. By reframing VLA failures as an evaluation bottleneck, it shifts research focus towards more efficient test-time scaling strategies, potentially leading to more resource-efficient AI development in robotics. The method itself does not present obvious ethical concerns beyond the general considerations for advanced AI and robotics. This paper presents SVA, a novel framework that significantly enhances the generalization and success rates of frozen Vision-Language-Action (VLA) models by distilling Monte-Carlo tree search knowledge into a lightweight, uncertainty-regularized Q-value model for test-time action evaluation. The work provides a compelling diagnostic study identifying an "evaluation bottleneck" in VLAs, proposes a practical and model-agnostic solution, and rigorously demonstrates consistent performance gains across diverse embodied benchmarks and VLA backbones, notably showing that scaling test-time evaluation can be more cost-effective than scaling model size.
Optimizer selection for large-scale model training has become a system-level design decision constrained jointly by compute, memory, tuning budget, and task diversity, yet the landscape of over one hundred methods remains fragmented. We therefore present OmniOpt, a unified survey and benchmark cookbook of optimizers for the research community. OmniOpt rests on four coupled components. First, we treat every optimizer update as a structured transformation through a five-stage meta-pipeline, and show that most methods engage only one or two of these stages. Second, we use norm-constrained linear minimization oracles (LMOs) to unify different optimizers. Third, these two views ground a dual-dimension taxonomy, one dimension assigning each method to a mechanism family and the other recording the measurable training objectives it aims to improve. Fourth, and at the core of this paper, we instantiate the full taxonomy in a unified cross-domain benchmark spanning representative optimizers, model scales, and training regimes from language model pretraining to image classification, systematically analyzing each method family across multiple effect objectives and laying out their trade-offs. OmniOpt thus supplies the research community with an operational coordinate system for selecting optimizers under explicit mechanism and objective assumptions, and charts a direction for the future development of the optimizer community.
Primary: Shanghai AI Laboratory
All Institutions: Shanghai AI Laboratory, Alibaba Group, Tencent AI Lab, Baidu, ByteDance, Alibaba DAMO Academy, Shanghai Jiao Tong University
OmniOpt presents a comprehensive taxonomy and benchmark for modern optimizers, offering a unified geometric perspective and extensive empirical evaluation that serves as a vital reference for the machine learning community.
The paper proposes "OmniOpt," a unified framework for understanding and benchmarking optimizers. The core methodological contribution is a "five-stage meta-pipeline" that decomposes optimizer updates into structured transformations, arguing that most modern optimizers only engage one or two of these stages. It further employs norm-constrained linear minimization oracles (LMOs) to provide a geometric unification of disparate methods. This is followed by a dual-dimension taxonomy classifying methods by mechanism family and training objectives. While the theoretical unification via LMOs is mathematically sound and offers a fresh perspective on optimizer geometry, the approach is largely analytical and taxonomic rather than introducing a new, superior optimization algorithm itself. The novelty lies in the synthesis and categorization rather than a breakthrough in optimization dynamics.
The empirical contribution is a large-scale, cross-domain benchmark spanning language model pretraining (C4, FineWeb-Edu) and image classification. The study evaluates representative optimizers across multiple model scales (340M to 1B parameters) and training regimes. The results provide a systematic analysis of trade-offs between convergence speed, final performance, memory usage, and per-step runtime. The breadth of the evaluation is impressive, covering long-context training and commonsense reasoning tasks. However, as a benchmarking study, it does not introduce a new SOTA method but rather ranks existing ones. The value is in the comprehensive data and the "cookbook" nature of the results, which helps practitioners make informed choices. The experiments are rigorous but do not demonstrate a surprising new capability; they confirm known trends with greater scale and detail.
The paper includes an appendix with detailed hyperparameter configurations for the main experiments, including learning rates, momentum coefficients, and stability constants. This level of detail significantly aids reproducibility. The authors provide a unified codebase (implied by "benchmark cookbook"), which is crucial for fair comparison. The use of standard datasets (C4, FineWeb-Edu, ImageNet) ensures that results can be verified by the community. The documentation of the "five-stage pipeline" implementation also supports reproducibility of the analytical framework.
The primary limitation is that the paper is a survey and benchmark, not a methodological breakthrough. The "five-stage pipeline" is a descriptive framework rather than a prescriptive one that leads to a new, better optimizer. The geometric unification via LMOs, while elegant, may not translate to practical improvements for all model architectures or data distributions. The benchmark, while extensive, is limited to the optimizers and tasks chosen by the authors; it may not cover emerging domains like reinforcement learning from human feedback (RLHF) or diffusion models in sufficient depth. Additionally, the "taxonomy" is subjective in its classification of methods into families, which may be debated by researchers who view optimizers through different lenses.
OmniOpt provides a valuable resource for the ML community by organizing the fragmented landscape of optimizers. It helps practitioners select appropriate optimizers based on specific constraints (compute, memory, task). The framework could guide future research by highlighting under-explored stages in the optimization pipeline or objectives that are currently neglected. By establishing a common coordinate system, it facilitates more rigorous comparison of future optimizer proposals. However, it does not directly impact societal outcomes or safety, as it is a technical tool for model training. OmniOpt presents a comprehensive taxonomy and benchmark for modern optimizers, offering a unified geometric perspective and extensive empirical evaluation that serves as a vital reference for the machine learning community.
Unified multi-modal models (UMMs) have shown promising interleaved text-image reasoning capabilities, yet effectively optimizing such multi-turn generation via reinforcement learning (RL) remains an open challenge. Existing approaches apply RL exclusively to text steps, relegating image generation to supervised surrogates, preventing policy gradients from propagating through the full interleaved trajectory across heterogeneous modalities. This leaves the potential of RL for UMMs largely untapped. In the paper, we introduce BRAID (Bridging inteRleAved multI-modal reasoning as a unified Decision process), a simple framework that casts multi-turn text-image-text reasoning as a unified Markov decision process (MDP), enabling joint optimization of textual and visual generation via a single, principled RL objective. BRAID computes a shared trajectory-level advantage and propagates it coherently into both text tokens and image denoising paths, each optimized through its modality-native policy gradient mechanism. To further address long-horizon credit assignment, BRAID employs a vision-language model (VLM) judge that scores each intermediate image on its reasoning utility, supplying dense turn-level feedback to sharpen learning at critical visual branches. Experiments on spatial reasoning and visual perception benchmarks show that BRAID consistently outperforms various baselines, confirming that a unified MDP formulation with vision-thinking guidance is essential for effective multi-modal reasoning.
Primary: Tencent Youtu Lab
All Institutions: Tencent Youtu Lab
This paper presents a significant methodological advance in Unified Multi-Modal Models by applying Reinforcement Learning to joint text-image generation, moving beyond supervised surrogates for visual steps. By framing interleaved reasoning as a unified MDP and utilizing VLM-based dense feedback, BRAID addresses the critical challenge of credit assignment across heterogeneous modalities, offering a more principled and potentially more capable approach to multi-turn visual reasoning.
The paper proposes BRAID, a framework that formulates interleaved text-image reasoning as a unified Markov Decision Process (MDP). The core technical contribution is the application of Reinforcement Learning (RL) to the image generation steps within a Unified Multi-Modal Model (UMM), which are typically handled via supervised fine-tuning (SFT). The authors introduce a shared trajectory-level advantage that is backpropagated into both text tokens and image denoising paths using modality-native policy gradients. A key component is the use of a Vision-Language Model (VLM) judge to provide dense, turn-level feedback on intermediate images, addressing the long-horizon credit assignment problem inherent in multi-turn reasoning. The methodology is theoretically sound, extending standard PPO-like objectives to continuous visual spaces via score-based gradients or similar mechanisms compatible with diffusion models.
The experiments focus on spatial reasoning and visual perception benchmarks. The authors compare BRAID against baselines that likely include SFT-only approaches and potentially other RL methods that do not jointly optimize visual steps. The results indicate consistent improvements over baselines, supporting the claim that joint optimization is beneficial. However, the abstract and limited context suggest the benchmarks may be standard ones (e.g., MMMU, MathVista subsets, or specific spatial reasoning tasks). The improvement magnitude is described as "consistent," but without specific delta values in the abstract, the significance is moderate. The use of a VLM judge for evaluation introduces potential bias, as the judge is part of the optimization loop, which is a common but critical point of scrutiny in this domain.
The paper mentions the framework is "simple" and provides a principled RL objective. Assuming the authors release code (standard for arXiv submissions in this era), reproducibility should be high. The use of standard VLM judges and diffusion models for image generation means the components are well-understood. However, the specific implementation details of propagating gradients through the image denoising path (e.g., using score matching or latent space gradients) need to be clearly defined in the full text for exact replication.
The primary limitation is the reliance on a VLM judge for dense feedback. This creates a potential loop where the model optimizes for the judge's preferences rather than ground-truth reasoning, potentially leading to reward hacking or mode collapse in visual generation. Furthermore, RL training for diffusion models is computationally expensive and unstable; the paper must demonstrate that BRAID is stable enough for practical use. The "unified" nature might also introduce complexity in balancing the learning rates and gradients between text and image modalities, which is a known challenge in multi-modal RL.
This work contributes to the field of Multimodal Large Language Models (MLLMs) by providing a pathway to more robust, reasoning-capable models that can generate visual content as part of their thought process. This has implications for autonomous agents, educational tools, and complex problem-solving systems. However, the increased computational cost of RL training and the potential for generating misleading visual content (if the judge is biased) are important societal considerations. This paper presents a significant methodological advance in Unified Multi-Modal Models by applying Reinforcement Learning to joint text-image generation, moving beyond supervised surrogates for visual steps. By framing interleaved reasoning as a unified MDP and utilizing VLM-based dense feedback, BRAID addresses the critical challenge of credit assignment across heterogeneous modalities, offering a more principled and potentially more capable approach to multi-turn visual reasoning.
Medicine is inherently multimodal, requiring clinicians to synthesize information across diverse data streams. Yet the development of multimodal foundation models is constrained by limited access to large-scale, high-quality clinical data. Although PubMed Central (PMC) offers a complementary source of expert-authored image-text data, existing PMC-derived resources remain limited in fidelity, reproducibility, and clinical validation. We introduce MedPMC, an automated, continuously updatable framework that transforms permissively licensed literature into high-fidelity infrastructure for medical multimodal models. Applied to 6.1 million PMC articles, MedPMC curated 11 million medical image-text pairs. Component evaluations showed strong performance for initial screening (F1 = 93.2), multi-panel figure detection (F1 = 96.5), figure separation (mAP = 89.8), caption separation and alignment (F1 = 81.4; ROUGE-L = 85.3), and medical figure classification (F1 = 96.5). Manual review by five annotators, three with medical training, found 95.3% of MedPMC images medically relevant, versus 19.7% in a prior PMC-derived dataset. Across 26 benchmarks spanning 11 specialties, a MedPMC-trained CLIP-style model improved average zero-shot AUC by 7.1 percentage points over the strongest architecture-matched biomedical CLIP baseline despite using fewer than half as many image-text pairs. As the vision encoder in a multimodal large language model, it improved medical visual question-answering by 1.9 and 16.9 percentage points across two benchmarks. In 10,524 Yale New Haven Health System dermatology photographs, it improved morphology-to-image retrieval Recall@5 by 11.7 percentage points. These findings show that high-fidelity literature curation strengthens medical multimodal foundation models across benchmark and clinical settings. We publicly release the framework, corpus, benchmarks, and pretrained models.
Primary: Yale University
All Institutions: Yale University, University of Illinois Urbana-Champaign, Microsoft Research, Amazon
Yale University's MedPMC framework represents a landmark contribution to medical multimodal learning by systematically curating a massive, high-fidelity dataset from open-access literature, demonstrating that data quality is as critical as quantity for training effective foundation models, and providing a reproducible pipeline and resources that will serve as a new standard for the field.
The paper introduces MedPMC, a systematic, automated pipeline for curating high-fidelity image-text pairs from PubMed Central (PMC). The methodology addresses a critical bottleneck in medical multimodal learning: the scarcity of large-scale, high-quality, and clinically relevant data. The pipeline involves several sophisticated components: (1) filtering 6.1 million articles for permissive licenses; (2) detecting multi-panel figures; (3) separating individual figures from panels; (4) aligning figures with their corresponding captions using natural language processing techniques; and (5) classifying figures for medical relevance. The authors report high performance on these component tasks (e.g., F1=93.2 for screening, mAP=89.8 for figure separation), indicating a robust and well-engineered curation process. The novelty lies not in a single algorithmic breakthrough but in the systematic integration and scaling of these components to create a massive, high-quality dataset, coupled with rigorous validation against prior, lower-fidelity datasets.
The evaluation is comprehensive and compelling. First, the authors validate the quality of the curated data through manual review by five annotators (three with medical training), showing a significant improvement in medical relevance (95.3% vs. 19.7% in a prior dataset). Second, they train a CLIP-style model on the MedPMC corpus and evaluate it on 26 benchmarks across 11 medical specialties. The MedPMC-trained model outperforms the strongest architecture-matched biomedical CLIP baseline by 7.1 percentage points in average zero-shot AUC, despite using fewer than half the image-text pairs. This demonstrates the high signal-to-noise ratio of the MedPMC data. Third, they integrate the vision encoder into a multimodal large language model (MLLM) and show improvements in medical visual question-answering. Finally, they demonstrate clinical utility by improving morphology-to-image retrieval on a real-world dermatology dataset from Yale New Haven Health System. The results are statistically significant and practically meaningful.
The authors provide extensive resources for reproducibility. The code for the curation pipeline is publicly available on GitHub. The MedPMC corpus, component-level benchmark resources, pretrained checkpoints, and metadata are released on Hugging Face. The data release includes versioning by article cutoff date, source-license filters, and processing configuration, allowing users to reproduce specific snapshots. They also provide clear instructions for license-aware filtering. The only non-public component is the clinical dermatology data, which is subject to privacy restrictions, but aggregate results and processing procedures are described. This level of transparency is exemplary.
The primary limitation is the reliance on open-access literature, which may introduce selection bias towards certain types of studies or institutions. The pipeline's performance, while high, is not perfect (e.g., F1=81.4 for caption separation), meaning some noise remains in the dataset. The clinical evaluation is limited to dermatology retrieval, and while promising, it does not cover the full breadth of medical specialties. The authors acknowledge that some records are derived from articles with non-commercial licenses (CC BY-NC), which restricts commercial use of the dataset. Additionally, the study focuses on image-text pairs; the extension to other modalities (e.g., audio, video, time-series) is not addressed.
The MedPMC framework has significant potential to accelerate the development of medical multimodal foundation models. By providing a large-scale, high-quality, and openly accessible dataset, it lowers the barrier to entry for researchers and clinicians working in this domain. The improved performance of models trained on MedPMC suggests that high-fidelity data curation is a key lever for improving model capabilities. The release of the code and benchmarks encourages further research in data curation and multimodal learning. However, the non-commercial license restrictions for some data may limit its impact in commercial healthcare applications. The authors have responsibly addressed privacy concerns by not releasing patient data and by providing clear licensing information. Yale University's MedPMC framework represents a landmark contribution to medical multimodal learning by systematically curating a massive, high-fidelity dataset from open-access literature, demonstrating that data quality is as critical as quantity for training effective foundation models, and providing a reproducible pipeline and resources that will serve as a new standard for the field.
Structure-property relationships are foundational to biology, chemistry and materials science, where function, reactivity and physical response emerge from spatial, chemical and periodic organization. Mechanistically explaining these relationships requires interpreting structural evidence through scientific principles and physical constraints, from stereochemistry and bonding to symmetry, energetics and periodic order. However, applying artificial intelligence to this process presents a joint challenge of representation and reasoning: models must preserve domain-native structural information while showing how specific evidence supports predictions under these constraints. Here we introduce SciReasoner, a multimodal scientific foundation model for native structural reasoning across proteins, small molecules and inorganic crystals. SciReasoner discretizes coordinates, topologies and periodic connectivities into a unified structure-aware vocabulary, treating structural tokens as addressable evidence units during reasoning. In homology-controlled Gene Ontology prediction, SciReasoner improves Cellular Component annotation for low-homology and orphan-like proteins, increasing $F_{\max}$ from 0.42 to 0.55. In chemistry, it raises single-step retrosynthesis accuracy from 0.63 to 0.72 while generating fragment-level disconnection and precursor-verification traces. In materials science, its representations separate elemental and compound phases and resolve high- and low-band-gap regimes. Across 86 benchmarks, SciReasoner achieves state-of-the-art performance on 67 tasks. Double-blind expert evaluation rates its reasoning traces as preferred or at least comparable to those of a frontier large language model in 98% of cases. By making structure an inspectable substrate for reasoning under scientific constraints, SciReasoner connects accurate prediction with interpretable scientific inference.
Primary: Shanghai Artificial Intelligence Laboratory
All Institutions: Shanghai Artificial Intelligence Laboratory, The Chinese University of Hong Kong, Shanghai Jiao Tong University, Fudan University, University of Sydney, Nanjing University, University of Oxford, The University of Science and Technology of China, Drug Discovery and Design Center, State Key Laboratory of Drug Research, Shanghai Institute of Materia Medica, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Stanford University
SciReasoner represents a significant leap in scientific machine learning by unifying structural reasoning across biology, chemistry, and materials science through a novel tokenization and autoregressive framework, achieving state-of-the-art results and offering interpretable insights that bridge the gap between prediction and mechanistic understanding.
The paper introduces "SciReasoner," a multimodal foundation model designed for "native structural reasoning" across three distinct scientific domains: proteins, small molecules, and inorganic crystals. The core methodological innovation lies in the discretization of continuous structural data (coordinates, topologies, periodic connectivities) into a unified, structure-aware vocabulary. By treating structural tokens as addressable evidence units within an autoregressive reasoning trajectory, the model attempts to bridge the gap between geometric representation and logical inference. This approach moves beyond standard graph neural networks or transformer-based sequence models by explicitly integrating physical constraints (stereochemistry, bonding, symmetry) into the tokenization and reasoning process. The architecture appears to leverage a large language model backbone adapted for these specialized structural tokens, allowing for chain-of-thought style reasoning where the model generates explanations alongside predictions. The unification of these three disparate fields under a single "structure-property" framework is ambitious and represents a significant conceptual shift from domain-specific models to a general scientific reasoning engine.
The evaluation is extensive, covering 86 benchmarks across biology, chemistry, and materials science. Key results include: 1. **Biology:** Improvement in Gene Ontology prediction for low-homology proteins (Cellular Component $F_{max}$ from 0.42 to 0.55), demonstrating the model's ability to generalize beyond sequence similarity to structural function. 2. **Chemistry:** Single-step retrosynthesis accuracy increased from 0.63 to 0.72, with the added capability of generating fragment-level disconnection and precursor-verification traces. This interpretability is a key selling point. 3. **Materials Science:** The model successfully separates elemental and compound phases and resolves band-gap regimes, indicating strong representation learning capabilities for periodic structures. 4. **General Performance:** State-of-the-art performance on 67 out of 86 tasks. 5. **Human Evaluation:** Double-blind expert evaluation rated the reasoning traces as preferred or comparable to frontier LLMs in 98% of cases. The breadth of the evaluation is impressive, but the reliance on "homology-controlled" settings for biology suggests the model's true value is in the "long-tail" or zero-shot scenarios where traditional methods fail. The jump in retrosynthesis accuracy is significant in a field where gains are often marginal.
The paper is published on arXiv. While the abstract and introduction provide a high-level overview, the full text provided in the prompt is truncated (sections are listed but content is missing). However, based on the abstract's description of "discretizing coordinates... into a unified structure-aware vocabulary," the method implies a specific tokenization scheme that, if detailed in the full paper (likely in the Method section), would be reproducible. The claim of 86 benchmarks suggests a comprehensive suite, which aids reproducibility if the benchmark definitions are standardized. The lack of a provided code URL in the prompt is a negative, but top-tier scientific ML papers increasingly open-source code; the high institutional backing (Shanghai AI Lab, CUHK, etc.) suggests code release is likely or imminent.
1. **Computational Cost:** Training a multimodal foundation model on three complex scientific domains with a unified vocabulary is likely extremely computationally intensive. The inference cost for generating "reasoning traces" may be prohibitive for high-throughput screening. 2. **Domain Specificity of Tokenization:** While the "unified vocabulary" is a strong claim, the physical constraints of proteins (flexible, solvent-exposed) differ vastly from inorganic crystals (rigid, periodic). It is unclear how well a single tokenizer handles the topological diversity without significant information loss or ambiguity. 3. **Evaluation Bias:** The "expert evaluation" of reasoning traces, while promising, is subjective. The 98% preference rate is exceptionally high and may suffer from bias if the traces are generated by a model known to be high-performing. 4. **Data Quality:** The performance is heavily dependent on the quality of the training data (PDB, ZINC, Materials Project, etc.). Errors in structural data or labeling in these databases will propagate into the model's "reasoning."
This work has profound implications for scientific discovery. By providing interpretable, structure-grounded reasoning, it addresses the "black box" problem in AI for science. This could accelerate drug discovery (by explaining *why* a molecule is active), materials design (by explaining phase stability), and synthetic biology. The ability to handle low-homology proteins could aid in understanding orphan diseases. However, the potential for misuse in designing harmful biological agents or novel toxins via automated retrosynthesis must be considered, necessitating robust safety guardrails in the reasoning traces. SciReasoner represents a significant leap in scientific machine learning by unifying structural reasoning across biology, chemistry, and materials science through a novel tokenization and autoregressive framework, achieving state-of-the-art results and offering interpretable insights that bridge the gap between prediction and mechanistic understanding.
Major cloud data platforms now expose large language model capabilities as native SQL functions, enabling analysts to perform classification, filtering, sentiment analysis, extraction, similarity search, and aggregation within ordinary SQL queries. Yet existing text-to-SQL benchmarks evaluate only conventional SQL and provide no signal on whether models can generate such AI-native SQL. We introduce Spider 2.0-AIFunc, a benchmark of 465 verified instances across 125 real-world databases covering six types of AI functions on the Snowflake platform. Starting from an existing enterprise text-to-SQL benchmark, we construct Spider 2.0-AIFunc through an agent-based pipeline that rewrites source tasks into AI-native form, simultaneously transforming target queries and refining natural language instructions to make the intended AI-native solution explicit and reduce ambiguity. All instances pass a multi-round repeated execution protocol across temporally separated windows to confirm result stability before release. Evaluating ten state-of-the-art language models, we find that the strongest proprietary models reach 67-70% execution accuracy while the best open-source model achieves 58.1%, a gap driven primarily by errors in predicate specification, schema grounding, and AI function parameterization. Agent frameworks designed for traditional text-to-SQL challenges, such as schema retrieval and relevant table selection, do not transfer effectively to AI-native SQL: a minimal agent setup consistently matches or outperforms more elaborate alternatives, suggesting that the strategies these frameworks employ are less critical in this setting. Data are available at https://github.com/Leolty/Spider2-AIFunc .
Primary: University of Hong Kong
All Institutions: University of Hong Kong, Snowflake
Spider 2.0-AIFunc introduces the first benchmark for evaluating text-to-SQL systems on AI-native SQL workflows, revealing that current models struggle with semantic parameterization and that complex agent frameworks offer no advantage over minimal setups, signaling a shift in the bottleneck of enterprise text-to-SQL from planning to semantic execution. The paper provides a rigorous evaluation framework and detailed error analysis that will serve as a foundational resource for the community as the industry transitions to AI-integrated database systems.
The paper introduces a novel benchmark construction methodology for "AI-Native SQL," a new paradigm where LLM capabilities are exposed as native SQL functions (e.g., `AI_CLASSIFY`, `AI_SENTIMENT`). The core methodological contribution is an agent-based pipeline that rewrites existing enterprise text-to-SQL tasks (from Spider2-Snow) into AI-native forms. This involves not just transforming the SQL query but also refining the natural language instructions to ensure specification determinism (explicitly defining AI function parameters) and verifying execution determinism (handling stochasticity in AI function outputs via multi-round verification). The approach addresses a critical gap in current benchmarks which ignore the shift towards semantic operators in SQL engines.
The evaluation is rigorous and comprehensive. The authors evaluate 10 state-of-the-art models (proprietary and open-source) using a minimal agent framework to isolate model capability from framework complexity. They provide detailed error analysis stratified by model performance tiers, identifying specific failure modes such as schema grounding, predicate specification, and AI function parameterization. The finding that complex agent frameworks (AutoLink, ReFoRCE, DSR-SQL) do not outperform a minimal setup is a significant and surprising empirical result, suggesting that the bottleneck in AI-Native SQL is primarily model capability (semantic understanding and parameterization) rather than retrieval or planning complexity. The benchmark includes 465 verified instances across 125 databases, providing a robust testbed.
The paper provides a detailed description of the construction pipeline, verification protocols, and evaluation metrics. The code and data are made publicly available on GitHub. The multi-round execution verification protocol ensures that the benchmark instances are stable, which is crucial for reproducibility in the context of stochastic AI functions. The evaluation setup is clearly defined, including the handling of timeouts and the specific comparison logic for results.
The benchmark is currently scoped to Snowflake and its specific set of Cortex AI functions, limiting generalizability to other platforms (BigQuery, Databricks) or function types. The construction relies on a single strong model (Claude Opus 4.5) for rewriting, which may introduce biases in instruction wording or task formulation. The evaluation uses a single agent trajectory per model, which does not capture the variance of stochastic agents or allow for pass@k analysis. Additionally, the benchmark relies on the assumption that the underlying AI functions remain stable over time, which may not hold if the cloud provider updates the backend models.
This paper has significant broader impact as it establishes the first standardized benchmark for evaluating text-to-SQL systems on AI-native workflows. As cloud data platforms increasingly integrate LLMs directly into their query engines, this benchmark will guide the development of more capable and reliable text-to-SQL systems. It highlights the unique challenges of AI-Native SQL, such as parameterization and semantic grounding, which are distinct from traditional text-to-SQL challenges. The findings will influence how researchers design agents and models for enterprise data analysis, potentially accelerating the adoption of AI-native SQL in industry. Spider 2.0-AIFunc introduces the first benchmark for evaluating text-to-SQL systems on AI-native SQL workflows, revealing that current models struggle with semantic parameterization and that complex agent frameworks offer no advantage over minimal setups, signaling a shift in the bottleneck of enterprise text-to-SQL from planning to semantic execution. The paper provides a rigorous evaluation framework and detailed error analysis that will serve as a foundational resource for the community as the industry transitions to AI-integrated database systems.
Uncertainty estimation (UE) enables LLM-powered systems to recognize when to abstain, yet existing research has predominantly focused on English. We present the first large-scale evaluation of UE methods across 22 languages, spanning high-, mid-, and low-resource settings. Using two human-curated Q\&A datasets, we compare open and closed box UE methods (nine in total) across different model sizes and architectures while eliciting long-form reasoning, avoiding LLM-as-a-judge and embedding-based scoring, which can introduce evaluation noise. We report three main actionable findings. First, we find that prompting models to reason in English while keeping questions in low-resource languages substantially improves UE performance, suggesting that comprehension of low-resource languages is largely intact, and that the reliability bottleneck lies in generation rather than understanding. Second, prompting models to reason in English closes the UE performance gap between low and high-resource languages, demonstrating that generation language matters more than the question language. Third, the choice of UE method should depend on model scale: at smaller scales, open-box probability-based methods outperform alternatives; at larger scales, closed-box self-verbalized uncertainty becomes superior. Finally, we provide an analysis of threshold selection for selective prediction, offering guidance on calibrating abstention in multilingual settings.
Primary: Amazon
All Institutions: Amazon
This paper presents a comprehensive and methodologically sound large-scale evaluation of uncertainty estimation in multilingual LLMs, providing crucial insights into the interplay between language resource levels, model scale, and uncertainty signaling mechanisms.
The paper proposes a rigorous evaluation framework for Uncertainty Estimation (UE) in Large Language Models (LLMs) across 22 languages. The core methodological innovation lies in the evaluation setup: using human-curated Multiple-Choice Question Answering (MCQA) datasets with elicited long-form reasoning to avoid the noise associated with LLM-as-a-judge or embedding-based correctness metrics. This allows for a clean, label-grounded assessment of UE methods (both open-box and closed-box) on the reasoning traces rather than just the final answer. The study systematically compares nine UE methods across varying model scales (270M to 235B) and resource levels. The approach is sound, addressing a critical gap in multilingual trustworthiness evaluation by isolating the uncertainty signal from the correctness signal more effectively than prior work.
The experimental design is comprehensive and large-scale. The authors evaluate 9 models and 9 UE methods across 22 languages, covering high, mid, and low-resource settings. Key findings include: (1) English reasoning significantly boosts UE performance for low-resource languages, suggesting the bottleneck is generation, not comprehension; (2) Self-Verbalized uncertainty outperforms other methods at large scales (235B), while open-box methods are better for smaller models; (3) Sampling-based methods fail on low-resource languages due to lack of diversity signal. The results are robust, supported by statistical significance testing and confidence intervals. The analysis of cross-lingual answer options and threshold calibration adds practical value. The use of parallel datasets (Global-MMLU and MMLU-ProX) ensures comparability.
The paper provides detailed descriptions of the datasets, models, prompts, and UE methods. It mentions the use of LM-Polygraph for implementation. The hardware infrastructure is specified. However, the specific versions of the models (e.g., "Claude 4.5 Sonnet" which appears to be a future/hypothetical name or typo for a current model, and "Gemma3" which is also not yet publicly released as of mid-2024, suggesting this is a very recent or forward-looking paper) and the exact random seeds are not fully detailed in the text provided, though the appendix references suggest they are available. The code release is mentioned but the URL is not provided in the text. The methodology is clear enough for replication if the models and datasets are accessible.
The study is limited to MCQA tasks, which may not fully generalize to open-ended generation where correctness is harder to define. The reliance on specific datasets (Global-MMLU, MMLU-ProX) means results might vary with other benchmarks. The "Claude 4.5 Sonnet" and "Gemma3" references are unusual and might indicate the paper is from the future or uses internal/unreleased models, which could limit immediate reproducibility for the broader community. The study does not include training-based UE methods.
This work has significant implications for deploying trustworthy LLMs in multilingual contexts. By demonstrating that generation language matters more than question language for UE, it provides actionable guidance for system designers (e.g., using English reasoning traces for low-resource languages). It also clarifies the trade-offs between open-box and closed-box methods based on model scale, helping practitioners choose appropriate UE strategies. The findings challenge the assumption that low-resource language performance is solely due to comprehension deficits, highlighting generation quality as a key factor. This paper presents a comprehensive and methodologically sound large-scale evaluation of uncertainty estimation in multilingual LLMs, providing crucial insights into the interplay between language resource levels, model scale, and uncertainty signaling mechanisms.