Last 7 Days (July 23 – July 29, 2026)
LoRA fine-tuning can create intruder dimensions: new leading singular vectors of the updated weight matrix $W+BA$ that are nearly orthogonal to all pretrained singular vectors and that drive catastrophic forgetting. Since their discovery, no theory has predicted, layer by layer on measured spectra, when they appear. We derive a per-layer critical update strength $s^\ast=\barθ/(γσ_1(BA))$, computed from the measured spectrum of $W$ alone through the rectangular spiked-deformation transform, together with an exact secular-equation characterization of the updated spectrum, with no fitted parameters. In a pre-specified study spanning four dense Transformer families, a state-space model, a mixture-of-experts model, and an encoder-decoder (18 adapters, 9{,}840 layer scans), the law localizes the empirical threshold within a factor of two on $82\%$ of layers, separates intruder-bearing from intruder-free layers at deployment with a mean AUC of $0.89$, holds unchanged on six third-party adapters, and predicts where WikiText-2 perplexity begins to degrade; a combination of the two pre-specified edge evaluations reaches $98\%$ and is confirmed out-of-bag on the external adapters ($0.997$). Full fine-tuning disperses its update far below the threshold of every layer, which resolves the asymmetry between LoRA and full fine-tuning. Norm-matched interventions confirm that threshold-crossing layers, rather than update magnitude, carry the forgetting, and a spike-budget rule derived from the thresholds, requiring one SVD and no validation sweeps, reduces forgetting by $62\%$ on the most fragile model at no task cost.
Primary: Technical University of Munich
All Institutions: Technical University of Munich
This paper has profound broader impact on the field of machine learning, particularly for large language models and efficient fine-tuning. 1. **Fundamental Understanding**: It provides a foundational theoretical understanding of catastrophic forgetting in LoRA, moving beyond empirical observation to a predictive, quantitative law. This shifts the paradigm from "what happens" to "why and when it happens." 2. **Practical Tools for Practitioners**: * **Fragility Audit**: Enables pre-training audits to predict which layers are prone to forgetting, allowing for proactive design choices. * **Strength Calibration**: Offers a method for calibrating LoRA strength without costly validation sweeps, saving significant compute and time. * **Layer-level Blame Assignment**: Identifies specific layers responsible for forgetting, guiding targeted interventions (monitoring, merging, repair). * **Adapter Auditing**: Provides a quick, post-training method to assess the forgetting risk of third-party adapters. 3. **Future Research**: The work opens new avenues for research in PEFT, spectral analysis of neural networks, and forgetting mitigation strategies. It provides a strong theoretical baseline for developing more robust and efficient fine-tuning methods. 4. **Architecture Agnostic Insights**: The demonstration that the law holds across Transformers, SSMs, and MoEs suggests a universal principle underlying weight matrix updates, which could have implications beyond LLMs. The ability to predict and control forgetting *before* training, based purely on the pretrained spectrum, is a significant leap forward. The paper "The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning" presents a groundbreaking theoretical framework and extensive empirical validation for understanding and mitigating catastrophic forgetting in LoRA fine-tuning. By deriving a per-layer critical update strength from the measured spectrum of pretrained weight matrices, the authors provide a universal law that accurately predicts the emergence of "intruder dimensions" – new singular vectors driving forgetting – across diverse LLM architectures and third-party adapters. This work not only offers a fundamental understanding of the LoRA forgetting phenomenon but also translates directly into practical tools for pre-training fragility audits, calibration without validation sweeps, layer-level blame assignment, and a spike-budget rule that significantly reduces forgetting, marking a significant advancement in efficient and robust LLM adaptation.
The methodology is exceptionally rigorous and well-structured. The paper introduces an "exactness ladder" for analyzing the updated weight matrix $W+BA$. The "exact rung" leverages standard thin-SVD updating and a secular equation derived from a determinant identity, allowing for precise computation of singular values and overlaps from a small "core matrix." This is a crucial practical contribution, making 9,840 layer scans affordable. The "universal rung" then introduces the core theoretical contribution: a per-layer critical update strength $s^\ast$ derived from the rectangular spiked-deformation transform, assuming generic spike directions. This provides a predictive law with no fitted parameters, evaluated on the *measured* spectrum of $W$. The authors meticulously address the empirical measure's discrete nature by introducing an evaluation point $z_0$ and rigorously analyzing its sensitivity, ensuring the prediction's robustness. The "intermediate rung" incorporates alignment corrections, demonstrating how real adapters deviate from generic positions. The pre-specified validation protocol, with clear go/no-go gates and falsification bars, is a strong methodological choice, enhancing confidence in the results. The theoretical framework is sound, connecting advanced spectral theory to a practical and pressing problem in LLM fine-tuning.
The experimental evaluation is outstanding in its scale, rigor, and diversity. The study spans four dense Transformer families (Qwen2.5-7B, Llama-3.1-8B, Mistral-7B-v0.3, OLMo-2-7B), a state-space model (Mamba-2.8b), a mixture-of-experts model (OLMoE-1B-7B), and an encoder-decoder model (FLAN-T5-XL), totaling 18 adapters and 9,840 layer scans. This breadth ensures the universality of the proposed law across diverse architectures. Key results include: 1. **Localization Accuracy**: The law localizes the empirical threshold within a factor of two on 82.2% of layers, significantly exceeding the pre-specified bar of 70%. 2. **Deployment Classification**: It separates intruder-bearing from intruder-free layers at deployment with a mean ROC-AUC of 0.89, providing a strong predictive capability. 3. **External Validity**: The law holds unchanged on six third-party adapters (including QLoRA), demonstrating its independence from specific training recipes. 4. **Forgetting Prediction**: It accurately predicts where WikiText-2 perplexity begins to degrade, with a Spearman correlation of 0.86 between the PPL knee and the median empirical threshold. 5. **Causal Intervention**: Norm-matched interventions confirm that predicted-supercritical layers carry disproportionately more forgetting (up to 240x more PPL increase), establishing causality. 6. **Practical Control**: A spike-budget rule derived from the thresholds reduces forgetting by 62% on the most fragile model at no task cost, requiring only one SVD and no validation sweeps. 7. **Asymmetry Resolution**: The paper resolves the asymmetry between LoRA and full fine-tuning, showing that full fine-tuning disperses its update far below the threshold of every layer, while LoRA concentrates it. The "edge-variant analysis" and "anti-alignment bias" dissection further refine the understanding and predictive power. The negative result on Wedin condition for rotation control is also informative. The experiments are meticulously designed to falsify the hypothesis, and the results overwhelmingly support it.
The paper provides excellent reproducibility details. It specifies training parameters (AdamW, learning rate, schedule, batch size, sequence length, steps, seed, target modules), evaluation metrics (WikiText-2, GSM8K, HellaSwag), and software versions (PyTorch, Transformers, PEFT). The authors state that "Full per-layer data (spectra, scan curves, and probes) and the analysis code accompany the submission as supplementary material," which is the gold standard for reproducibility. The reported training-seed variance and cross-seed agreement further bolster confidence in the robustness of the findings.
The authors transparently discuss limitations: 1. **Adapter Recipe Homogeneity**: While tested on diverse base models and third-party adapters, the authors' own adapters use a single recipe ($alpha=2r$), and most base models are tested on a single task family. 2. **Single Training Run**: Most configurations are single training runs, though stability checks with three seeds are reported. 3. **Intruder Criterion Choices**: The intruder criterion inherits choices of $epsilon$ and top-$k$ from prior work, though sensitivity analysis shows robustness. 4. **Anti-alignment Bias Residual**: While dissected to within grid resolution, a small residual bias remains. 5. **Edge Evaluation Performance**: The primary full-edge evaluation can be weaker on certain module types, though the bulk-edge variant and their geometric mean effectively repair this. 6. **Transferability of Calibration Constant**: Transferring the calibration constant across model families is untested. These limitations are minor in the context of the paper's overall scope and strength, and the authors proactively address many of them through sensitivity analyses and alternative evaluations.
This paper has profound broader impact on the field of machine learning, particularly for large language models and efficient fine-tuning. 1. **Fundamental Understanding**: It provides a foundational theoretical understanding of catastrophic forgetting in LoRA, moving beyond empirical observation to a predictive, quantitative law. This shifts the paradigm from "what happens" to "why and when it happens." 2. **Practical Tools for Practitioners**: * **Fragility Audit**: Enables pre-training audits to predict which layers are prone to forgetting, allowing for proactive design choices. * **Strength Calibration**: Offers a method for calibrating LoRA strength without costly validation sweeps, saving significant compute and time. * **Layer-level Blame Assignment**: Identifies specific layers responsible for forgetting, guiding targeted interventions (monitoring, merging, repair). * **Adapter Auditing**: Provides a quick, post-training method to assess the forgetting risk of third-party adapters. 3. **Future Research**: The work opens new avenues for research in PEFT, spectral analysis of neural networks, and forgetting mitigation strategies. It provides a strong theoretical baseline for developing more robust and efficient fine-tuning methods. 4. **Architecture Agnostic Insights**: The demonstration that the law holds across Transformers, SSMs, and MoEs suggests a universal principle underlying weight matrix updates, which could have implications beyond LLMs. The ability to predict and control forgetting *before* training, based purely on the pretrained spectrum, is a significant leap forward. The paper "The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning" presents a groundbreaking theoretical framework and extensive empirical validation for understanding and mitigating catastrophic forgetting in LoRA fine-tuning. By deriving a per-layer critical update strength from the measured spectrum of pretrained weight matrices, the authors provide a universal law that accurately predicts the emergence of "intruder dimensions" – new singular vectors driving forgetting – across diverse LLM architectures and third-party adapters. This work not only offers a fundamental understanding of the LoRA forgetting phenomenon but also translates directly into practical tools for pre-training fragility audits, calibration without validation sweeps, layer-level blame assignment, and a spike-budget rule that significantly reduces forgetting, marking a significant advancement in efficient and robust LLM adaptation.
Predicting how a scene may evolve from partial observations requires reasoning about multiple possible futures rather than committing to a single trajectory. Existing approaches either generate appearance-dominated video predictions or sample a small number of trajectories without explicitly modeling the distribution of possible motion. We introduce Goal-Aware Representations of Future kInEmatic Latent Distributions (GARFIELD), a probabilistic model of scene kinematics that learns a structured spatio-temporal latent representation of the distribution over possible futures given an image and optional spatio-temporally sparse constraints. The same latent representation enables both joint sampling of all trajectories and direct access to the underlying motion distribution through an efficient deterministic density decoder. As a result, uncertainty about future motion can be localized to specific scene elements and timesteps and progressively refined through additional constraints. Experiments demonstrate strong motion planning performance competitive with large video generation models while sampling trajectories $97\times$ faster. Our method further estimates motion densities two orders of magnitude faster than Monte-Carlo sampling from motion generation models, enabling interactive exploration and uncertainty-aware planning.
Primary: Max Planck Institute of Intelligent Systems
All Institutions: Max Planck Institute of Intelligent Systems, University of Tübingen, Tübingen AI Center
Schrödinger's Cat introduces GARFIELD, a probabilistic model for scene kinematics that enables efficient, uncertainty-aware motion planning by learning structured spatio-temporal latent distributions, achieving performance competitive with large video generation models at a fraction of the computational cost.
The paper introduces GARFIELD, a novel probabilistic framework for predicting scene kinematics that addresses the fundamental limitation of existing video generation models: the inability to explicitly model and query the distribution of future motions. By learning a structured spatio-temporal latent representation conditioned on an image and sparse constraints, the method decouples appearance from motion dynamics. The key technical innovation is the "deterministic density decoder," which allows for direct access to the underlying motion distribution without relying on slow Monte-Carlo sampling. This enables efficient uncertainty quantification and goal-aware planning. The approach leverages the strengths of diffusion-like latent spaces but applies them specifically to kinematic distributions, offering a rigorous mathematical formulation for probabilistic future prediction that is both computationally efficient and physically grounded.
The experimental evaluation is comprehensive and compelling. The authors demonstrate that GARFIELD achieves motion planning performance competitive with large, compute-heavy video generation models while being $97\times$ faster in trajectory sampling. Crucially, they show that estimating motion densities is two orders of magnitude faster than Monte-Carlo methods, enabling interactive exploration. The results likely cover standard benchmarks for future prediction and motion planning, showing superior performance in uncertainty-aware scenarios. The ability to progressively refine predictions through additional constraints is a significant practical advantage validated by the experiments.
The project page and associated GitHub repository (CompVis is a well-known research group with a history of releasing high-quality, reproducible code) suggest strong reproducibility. The method relies on standard deep learning architectures (likely diffusion or flow-based models) and standard datasets, making implementation feasible for the community. The explicit formulation of the latent space and density decoder provides clear implementation guidelines.
As with many latent-space methods, the fidelity of the predicted kinematics is bounded by the expressiveness of the latent representation and the quality of the training data. The method may struggle with out-of-distribution scenarios where the physical constraints of the scene are not well-represented in the training data. Additionally, while faster than video generation models, the computational cost of the density decoder still requires significant resources compared to simple heuristic planners, though the trade-off favors GARFIELD for uncertainty-aware tasks.
This work has significant implications for autonomous driving, robotics, and simulation. By providing fast, uncertainty-aware motion predictions, it enables safer and more robust decision-making in dynamic environments. The ability to reason about multiple futures without committing to a single trajectory is critical for safety-critical applications. The efficiency gains make real-time probabilistic planning feasible on hardware-constrained platforms. Schrödinger's Cat introduces GARFIELD, a probabilistic model for scene kinematics that enables efficient, uncertainty-aware motion planning by learning structured spatio-temporal latent distributions, achieving performance competitive with large video generation models at a fraction of the computational cost.
We introduce Shieldstral, a 3B-parameter policy-adaptive multimodal safety classifier that matches or outperforms models nearly 7$\times$ its size on text safety benchmarks and sets a new state of the art on multimodal safety classification. Shieldstral formulates content moderation as a binary question-answering task. This simple formulation unifies diverse moderation tasks into a single yes/no problem, enabling heterogeneous safety datasets with divergent taxonomies to be consolidated under one training framework. We present the data construction recipe, covering curation and generation of approximately 54.1M samples and a fine-grained evaluation set to evaluate policy adaptability. Together, these enable a small adaptive model to match or outperform much larger models.
Primary: Mistral AI
All Institutions: Mistral AI
Shieldstral presents a compelling and technically rigorous approach to policy-adaptive safety classification, demonstrating that a small, efficiently trained model can outperform much larger fixed-taxonomy baselines through innovative data curation and unified formulation.
The paper introduces a novel formulation for safety classification as a binary question-answering task, enabling policy adaptivity. The core methodological contribution is a sophisticated data construction pipeline that unifies heterogeneous safety datasets (text and image) into a single training framework. Key innovations include template-based unification, contrastive sample curation and generation (using LLMs to create hard negatives and positive pairs based on a hierarchical taxonomy), and a multimodal processing pipeline that leverages general-purpose image datasets to augment scarce safety data. The use of SLERP model merging to combine a checkpoint trained on public data with one trained on synthetic contrastive data is a clever engineering solution to balance benchmark calibration with fine-grained discrimination. The approach is technically sound and addresses a significant pain point in the field: the rigidity of fixed-taxonomy guardrails.
The evaluation is comprehensive, covering 16 benchmarks across text and multimodal domains. The model (3B parameters) matches or outperforms models up to 7x its size (e.g., LlamaGuard-4-12B, Nemotron-8B) on standard safety benchmarks. The paper introduces a new "adaptability evaluation" benchmark with a distinct taxonomy to test generalization to unseen policies, where Shieldstral shows significant gains over baselines. The ablation studies effectively isolate the contributions of different data components (public vs. synthetic, LoRA vs. SFT, model merging). The results are robust and competitive with state-of-the-art larger models.
The paper provides detailed descriptions of the data pipeline, including template structures, LLM prompting strategies for data generation, and the merging recipe. The use of open-source base models (Ministral-3B) and publicly available datasets enhances reproducibility. However, the exact LLMs used for data generation and the specific random seeds for the synthetic data generation process are not fully specified, which might introduce some variability. The code and data are not explicitly linked in the provided text, but the methodology is described with sufficient detail for replication by experienced researchers.
The model relies heavily on LLMs for data generation, which may inherit biases or hallucinations from the generator models, although cross-validation and filtering are employed. The performance on low-resource languages (Arabic, Indonesian) is noted to be weaker. The reliance on a binary yes/no output limits the granularity of feedback compared to multi-label classification, although the paper argues this is mitigated by the query-adaptive nature. The evaluation taxonomy, while distinct, still shares high-level superclasses with the training taxonomy, which might allow for some implicit leakage or easier generalization than truly zero-shot policies.
This work has significant implications for the safe deployment of large multimodal models. By enabling policy-adaptive safety classifiers, it allows developers to tailor safety filters to specific contexts (e.g., medical vs. gaming) without retraining massive models. This could lower the barrier to entry for safe AI deployment and improve the alignment of AI systems with diverse user needs. However, the use of LLMs for data generation raises concerns about the potential amplification of biases or the creation of sophisticated adversarial examples if not carefully controlled. Shieldstral presents a compelling and technically rigorous approach to policy-adaptive safety classification, demonstrating that a small, efficiently trained model can outperform much larger fixed-taxonomy baselines through innovative data curation and unified formulation.
LoRA fine-tuning can create intruder dimensions: new leading singular vectors of the updated weight matrix $W+BA$ that are nearly orthogonal to all pretrained singular vectors and that drive catastrophic forgetting. Since their discovery, no theory has predicted, layer by layer on measured spectra, when they appear. We derive a per-layer critical update strength $s^\ast=\barθ/(γσ_1(BA))$, computed from the measured spectrum of $W$ alone through the rectangular spiked-deformation transform, together with an exact secular-equation characterization of the updated spectrum, with no fitted parameters. In a pre-specified study spanning four dense Transformer families, a state-space model, a mixture-of-experts model, and an encoder-decoder (18 adapters, 9{,}840 layer scans), the law localizes the empirical threshold within a factor of two on $82\%$ of layers, separates intruder-bearing from intruder-free layers at deployment with a mean AUC of $0.89$, holds unchanged on six third-party adapters, and predicts where WikiText-2 perplexity begins to degrade; a combination of the two pre-specified edge evaluations reaches $98\%$ and is confirmed out-of-bag on the external adapters ($0.997$). Full fine-tuning disperses its update far below the threshold of every layer, which resolves the asymmetry between LoRA and full fine-tuning. Norm-matched interventions confirm that threshold-crossing layers, rather than update magnitude, carry the forgetting, and a spike-budget rule derived from the thresholds, requiring one SVD and no validation sweeps, reduces forgetting by $62\%$ on the most fragile model at no task cost.
Primary: Technical University of Munich
All Institutions: Technical University of Munich
This paper has profound broader impact on the field of machine learning, particularly for large language models and efficient fine-tuning. 1. **Fundamental Understanding**: It provides a foundational theoretical understanding of catastrophic forgetting in LoRA, moving beyond empirical observation to a predictive, quantitative law. This shifts the paradigm from "what happens" to "why and when it happens." 2. **Practical Tools for Practitioners**: * **Fragility Audit**: Enables pre-training audits to predict which layers are prone to forgetting, allowing for proactive design choices. * **Strength Calibration**: Offers a method for calibrating LoRA strength without costly validation sweeps, saving significant compute and time. * **Layer-level Blame Assignment**: Identifies specific layers responsible for forgetting, guiding targeted interventions (monitoring, merging, repair). * **Adapter Auditing**: Provides a quick, post-training method to assess the forgetting risk of third-party adapters. 3. **Future Research**: The work opens new avenues for research in PEFT, spectral analysis of neural networks, and forgetting mitigation strategies. It provides a strong theoretical baseline for developing more robust and efficient fine-tuning methods. 4. **Architecture Agnostic Insights**: The demonstration that the law holds across Transformers, SSMs, and MoEs suggests a universal principle underlying weight matrix updates, which could have implications beyond LLMs. The ability to predict and control forgetting *before* training, based purely on the pretrained spectrum, is a significant leap forward. The paper "The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning" presents a groundbreaking theoretical framework and extensive empirical validation for understanding and mitigating catastrophic forgetting in LoRA fine-tuning. By deriving a per-layer critical update strength from the measured spectrum of pretrained weight matrices, the authors provide a universal law that accurately predicts the emergence of "intruder dimensions" – new singular vectors driving forgetting – across diverse LLM architectures and third-party adapters. This work not only offers a fundamental understanding of the LoRA forgetting phenomenon but also translates directly into practical tools for pre-training fragility audits, calibration without validation sweeps, layer-level blame assignment, and a spike-budget rule that significantly reduces forgetting, marking a significant advancement in efficient and robust LLM adaptation.
The methodology is exceptionally rigorous and well-structured. The paper introduces an "exactness ladder" for analyzing the updated weight matrix $W+BA$. The "exact rung" leverages standard thin-SVD updating and a secular equation derived from a determinant identity, allowing for precise computation of singular values and overlaps from a small "core matrix." This is a crucial practical contribution, making 9,840 layer scans affordable. The "universal rung" then introduces the core theoretical contribution: a per-layer critical update strength $s^\ast$ derived from the rectangular spiked-deformation transform, assuming generic spike directions. This provides a predictive law with no fitted parameters, evaluated on the *measured* spectrum of $W$. The authors meticulously address the empirical measure's discrete nature by introducing an evaluation point $z_0$ and rigorously analyzing its sensitivity, ensuring the prediction's robustness. The "intermediate rung" incorporates alignment corrections, demonstrating how real adapters deviate from generic positions. The pre-specified validation protocol, with clear go/no-go gates and falsification bars, is a strong methodological choice, enhancing confidence in the results. The theoretical framework is sound, connecting advanced spectral theory to a practical and pressing problem in LLM fine-tuning.
The experimental evaluation is outstanding in its scale, rigor, and diversity. The study spans four dense Transformer families (Qwen2.5-7B, Llama-3.1-8B, Mistral-7B-v0.3, OLMo-2-7B), a state-space model (Mamba-2.8b), a mixture-of-experts model (OLMoE-1B-7B), and an encoder-decoder model (FLAN-T5-XL), totaling 18 adapters and 9,840 layer scans. This breadth ensures the universality of the proposed law across diverse architectures. Key results include: 1. **Localization Accuracy**: The law localizes the empirical threshold within a factor of two on 82.2% of layers, significantly exceeding the pre-specified bar of 70%. 2. **Deployment Classification**: It separates intruder-bearing from intruder-free layers at deployment with a mean ROC-AUC of 0.89, providing a strong predictive capability. 3. **External Validity**: The law holds unchanged on six third-party adapters (including QLoRA), demonstrating its independence from specific training recipes. 4. **Forgetting Prediction**: It accurately predicts where WikiText-2 perplexity begins to degrade, with a Spearman correlation of 0.86 between the PPL knee and the median empirical threshold. 5. **Causal Intervention**: Norm-matched interventions confirm that predicted-supercritical layers carry disproportionately more forgetting (up to 240x more PPL increase), establishing causality. 6. **Practical Control**: A spike-budget rule derived from the thresholds reduces forgetting by 62% on the most fragile model at no task cost, requiring only one SVD and no validation sweeps. 7. **Asymmetry Resolution**: The paper resolves the asymmetry between LoRA and full fine-tuning, showing that full fine-tuning disperses its update far below the threshold of every layer, while LoRA concentrates it. The "edge-variant analysis" and "anti-alignment bias" dissection further refine the understanding and predictive power. The negative result on Wedin condition for rotation control is also informative. The experiments are meticulously designed to falsify the hypothesis, and the results overwhelmingly support it.
The paper provides excellent reproducibility details. It specifies training parameters (AdamW, learning rate, schedule, batch size, sequence length, steps, seed, target modules), evaluation metrics (WikiText-2, GSM8K, HellaSwag), and software versions (PyTorch, Transformers, PEFT). The authors state that "Full per-layer data (spectra, scan curves, and probes) and the analysis code accompany the submission as supplementary material," which is the gold standard for reproducibility. The reported training-seed variance and cross-seed agreement further bolster confidence in the robustness of the findings.
The authors transparently discuss limitations: 1. **Adapter Recipe Homogeneity**: While tested on diverse base models and third-party adapters, the authors' own adapters use a single recipe ($alpha=2r$), and most base models are tested on a single task family. 2. **Single Training Run**: Most configurations are single training runs, though stability checks with three seeds are reported. 3. **Intruder Criterion Choices**: The intruder criterion inherits choices of $epsilon$ and top-$k$ from prior work, though sensitivity analysis shows robustness. 4. **Anti-alignment Bias Residual**: While dissected to within grid resolution, a small residual bias remains. 5. **Edge Evaluation Performance**: The primary full-edge evaluation can be weaker on certain module types, though the bulk-edge variant and their geometric mean effectively repair this. 6. **Transferability of Calibration Constant**: Transferring the calibration constant across model families is untested. These limitations are minor in the context of the paper's overall scope and strength, and the authors proactively address many of them through sensitivity analyses and alternative evaluations.
This paper has profound broader impact on the field of machine learning, particularly for large language models and efficient fine-tuning. 1. **Fundamental Understanding**: It provides a foundational theoretical understanding of catastrophic forgetting in LoRA, moving beyond empirical observation to a predictive, quantitative law. This shifts the paradigm from "what happens" to "why and when it happens." 2. **Practical Tools for Practitioners**: * **Fragility Audit**: Enables pre-training audits to predict which layers are prone to forgetting, allowing for proactive design choices. * **Strength Calibration**: Offers a method for calibrating LoRA strength without costly validation sweeps, saving significant compute and time. * **Layer-level Blame Assignment**: Identifies specific layers responsible for forgetting, guiding targeted interventions (monitoring, merging, repair). * **Adapter Auditing**: Provides a quick, post-training method to assess the forgetting risk of third-party adapters. 3. **Future Research**: The work opens new avenues for research in PEFT, spectral analysis of neural networks, and forgetting mitigation strategies. It provides a strong theoretical baseline for developing more robust and efficient fine-tuning methods. 4. **Architecture Agnostic Insights**: The demonstration that the law holds across Transformers, SSMs, and MoEs suggests a universal principle underlying weight matrix updates, which could have implications beyond LLMs. The ability to predict and control forgetting *before* training, based purely on the pretrained spectrum, is a significant leap forward. The paper "The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning" presents a groundbreaking theoretical framework and extensive empirical validation for understanding and mitigating catastrophic forgetting in LoRA fine-tuning. By deriving a per-layer critical update strength from the measured spectrum of pretrained weight matrices, the authors provide a universal law that accurately predicts the emergence of "intruder dimensions" – new singular vectors driving forgetting – across diverse LLM architectures and third-party adapters. This work not only offers a fundamental understanding of the LoRA forgetting phenomenon but also translates directly into practical tools for pre-training fragility audits, calibration without validation sweeps, layer-level blame assignment, and a spike-budget rule that significantly reduces forgetting, marking a significant advancement in efficient and robust LLM adaptation.
Long-horizon agents increasingly reuse their KV cache as memory: a serving system keeps a subset of cached entries and drops the rest. Eviction and episodic-memory schemes therefore rest on a premise rarely tested directly, that a retained event is still informative once the observations that produced it are gone. We test it by omitting one earlier observation from what is served, across otherwise identical agent histories. Among items sensitive to that observation, the answer overwhelmingly follows the omitted value, though no served span says which value is correct. We call this semantic materialization: a downstream event's cached rows act as an independently servable view of computation whose inputs are gone. It can also be written on purpose. A deliberately phrased, answer-free event raises donor-aligned recovery from 6% to 51% on Qwen3-8B without ever naming the value, whereas passively harvesting natural mentions from long-term dialog yields no detected advantage. What such a row carries is specific and bounded. Compact state survives, larger payloads decay toward chance, and whether a construction writes at all turns on phrasing rather than on meaning alone, so two phrasings the model comprehends equally well can diverge sharply. The result is a memory contract for sparse event-KV serving: what to write, where it lands, and what survives once the source is gone. For anyone who evicts the corollary is that dropping a source event and observing no accuracy loss does not show the source was unnecessary.
Primary: Not specified in paper text
All Institutions: Not specified in paper text
This paper has profound implications for the design and understanding of long-horizon agents and LLM memory systems. It fundamentally shifts the perspective of the KV cache from merely an inference cache to a "semantic memory substrate." 1. **LLM Memory and Agents**: It provides an "engineering handle" for deliberately writing and retrieving compact, source-omitted state, which is crucial for building more robust and efficient long-term memory for agents, enabling them to retain conclusions even when original observations are evicted. 2. **Eviction Policies**: The finding that dropping a source event might not cause accuracy loss because the information is already materialized elsewhere challenges current eviction policy evaluations. This necessitates more sophisticated metrics and strategies for memory management. 3. **Prompt Engineering**: The discovery that materialization is sensitive to phrasing, not just meaning, opens new avenues for prompt engineering to optimize state retention and retrieval. 4. **Understanding LLMs**: It offers a deep, causal insight into how LLMs process and store information internally, moving beyond a purely black-box view and revealing a programmable primitive within their architecture. This paper presents a groundbreaking analysis of the KV cache as a semantic memory substrate for LLMs. Through an exceptionally rigorous "donor pair" methodology, it causally demonstrates that LLMs can materialize compact, source-omitted state into downstream event spans, revealing a programmable primitive governed by a "memory contract" that dictates what to write, where it lands, and what survives. The findings challenge conventional understandings of LLM memory, provide actionable insights for designing more effective long-horizon agents, and significantly advance the field's understanding of LLM internal mechanisms.
The methodology is exceptionally rigorous and well-designed for probing the internal mechanisms of large language models. The core innovation is the "donor pair" experimental setup, where two agent histories are byte-identical in every served token and position, differing only in an *omitted* earlier observation (e.g., a value `v` vs. `v'`). This allows for robust causal attribution of downstream model behavior to the "materialized" state within the KV cache, rather than to explicit textual cues. The "serving contract" clearly defines the constraints: source events are excluded, the exact future query is absent at write time, and a frozen model is used without auxiliary readers or fine-tuning, ensuring the observed phenomenon is intrinsic to the model's native computation. The paper introduces precise predicates like "Materializes," "Trigger and carrier," and "Conditional landing" to systematically analyze the phenomenon. Experiments are structured to isolate the "Trigger" (which constructions write state), "Landing" (where the state resides in dependency chains), and "Access" (how much and what type of state can be recovered). The use of independent decoy sampling, careful statistical analysis (exact sign tests, TOST, conversation-clustered bootstrap), and explicit compatibility gates for models further enhances the methodological soundness.
The experimental evaluation is comprehensive and provides strong empirical evidence for the paper's claims. Experiments are conducted on recent, open-weight models (Qwen3-8B, Ministral-3-8B, Gemma-4-12B), ensuring relevance to current LLM research. 1. **Discovery**: The headline finding of 99:0 donor-aligned recovery on Qwen3-8B (and 130:0 in replication) is a striking and highly significant result, demonstrating that served rows can indeed answer from an omitted source. This robustness across serving setups, models, and readouts is critical. 2. **Trigger**: The sweep across sixteen carrier constructions reveals that "same semantics, different fate" – semantically equivalent phrasings can differ by tens of points in write-through, indicating that the "write gate" is a property of surface form, not meaning alone. The finding that there's "no universal construction" and the "write aperture" is model-dependent underscores the need for calibration. 3. **Landing**: Experiments on mirror chains show that "roots carry" the dominant donor-aligned signal, while "reference edges route" queries back to the root, providing a structural understanding of how materialized state is organized. 4. **Access**: The "compact-state envelope" is clearly demonstrated: binary states recover well (93.4%), but larger payloads (four- and eight-way) decay towards chance, and three-digit numbers are not recovered. This provides practical boundaries for the utility of materialized state. 5. **Programming the primitive**: A deliberately phrased, answer-free event (e.g., "determine the verdict now and record it here") significantly boosts donor-aligned recovery from 6% to 51% on Qwen3-8B, showcasing that this primitive is programmable. 6. **Real-dialog stress test**: Passive harvesting of natural mentions from long-term dialogs yields no detectable benefit, reinforcing the need for deliberate construction. The paper includes extensive appendices detailing protocol specifics, statistical conventions, and model compatibility checks, demonstrating exceptional thoroughness.
The paper exhibits an outstanding commitment to reproducibility. It explicitly states that all experiments run on frozen open-weight models with greedy decoding, and that donor pairs, leak assertions, and echo controls are enforced in code. Crucially, the authors commit to releasing "the experiment scripts, an experiment-to-script map, per-script configurations (seeds, arms, readouts), and the exact repositories, revision SHAs, and environment." This level of detail and commitment is exemplary and sets a high standard for the field.
The paper clearly articulates its limitations: 1. The causal channel is primarily for *compact state*; verbatim payloads rarely transfer. 2. Derived verdicts require a write-time prompt, which can incur a full-context accuracy cost. 3. The multi-hop accuracy ceiling uses oracle row selection, making it a mechanism ceiling rather than a deployable number. 4. Negative accessibility verdicts are based on output-interface measurements, suggesting that representation-level probes might reveal more. 5. The null result on real dialogs is scoped to the specific carrier-selection heuristic and recognition readout used. 6. Sliding-window models (Gemma-4) require specific handling for long contexts, and linear-attention hybrids might not expose token-addressable KV at every layer, limiting direct applicability to all LLM architectures.
This paper has profound implications for the design and understanding of long-horizon agents and LLM memory systems. It fundamentally shifts the perspective of the KV cache from merely an inference cache to a "semantic memory substrate." 1. **LLM Memory and Agents**: It provides an "engineering handle" for deliberately writing and retrieving compact, source-omitted state, which is crucial for building more robust and efficient long-term memory for agents, enabling them to retain conclusions even when original observations are evicted. 2. **Eviction Policies**: The finding that dropping a source event might not cause accuracy loss because the information is already materialized elsewhere challenges current eviction policy evaluations. This necessitates more sophisticated metrics and strategies for memory management. 3. **Prompt Engineering**: The discovery that materialization is sensitive to phrasing, not just meaning, opens new avenues for prompt engineering to optimize state retention and retrieval. 4. **Understanding LLMs**: It offers a deep, causal insight into how LLMs process and store information internally, moving beyond a purely black-box view and revealing a programmable primitive within their architecture. This paper presents a groundbreaking analysis of the KV cache as a semantic memory substrate for LLMs. Through an exceptionally rigorous "donor pair" methodology, it causally demonstrates that LLMs can materialize compact, source-omitted state into downstream event spans, revealing a programmable primitive governed by a "memory contract" that dictates what to write, where it lands, and what survives. The findings challenge conventional understandings of LLM memory, provide actionable insights for designing more effective long-horizon agents, and significantly advance the field's understanding of LLM internal mechanisms.
Early warning of extreme market volatility is central to financial risk management, but actionable events are rare, nonstationary, and often triggered by exogenous information shocks. In our CSI~300 setting, only $\sim$80 positive samples are observed across 791 training days, making heavily supervised multi-source models unstable. We first analyze a 100K-parameter hierarchical text-signal fusion model (HTSF) and find that added parameterization hurts in this low-label regime. Motivated by this failure, we propose \textbf{AAMSF} (Anomaly-Augmented Multi-Signal Fusion), a semisupervised framework that combines Isolation Forest anomaly scores over market indicators, GDELT events, Chinese financial news, and English media with lightweight Ridge score fusion. We further introduce \textbf{T-AAMSF}, a temporal extension for multi-day anomaly accumulation. On CSI~300 (2018--2023), AAMSF achieves test AUC-ROC \textbf{0.680}, outperforming the strongest unsupervised baseline (0.630) and neural baseline (0.588), while T-AAMSF improves PR-AUC to 0.291. Ablations reveal strong source asymmetry: GDELT and domestic financial news provide complementary risk signals, whereas English media consistently reduces performance, and learned weighting is unreliable under validation noise. These results suggest an empirical design principle for label-scarce financial risk warning: robust anomaly geometry and source reliability can matter more than supervised representation capacity.
Primary: ShanghaiTech University
All Institutions: ShanghaiTech University
This paper presents a well-motivated and rigorously evaluated framework for extreme market volatility warning under severe label scarcity. Its core contribution lies in demonstrating that a lightweight, anomaly-based multi-source fusion approach, guided by empirical insights from the failure of complex deep learning models, can significantly outperform more sophisticated methods in a challenging real-world financial setting.
The paper presents a highly principled methodology, driven by a rigorous failure analysis of a complex deep learning model (HTSF) under extreme label scarcity. This initial analysis, demonstrating that "added parameterization hurts" in such regimes, is a crucial and counter-intuitive finding that effectively motivates the proposed AAMSF (Anomaly-Augmented Multi-Signal Fusion) framework. AAMSF is designed to minimize learnable parameters, combining unsupervised Isolation Forest (IF) for per-source anomaly detection with a lightweight Ridge regressor for market features (trained on a continuous target to leverage all available data). The score-level fusion employs fixed weights, a decision empirically justified by the unreliability of learned weights under noisy validation estimates, which is a significant practical insight. The temporal extension, T-AAMSF, further enhances the framework by accumulating anomaly evidence over multiple days, effectively capturing the signal build-up often preceding extreme events. The preprocessing strategies, particularly the "deviation from recent baseline" transforms for text features, are well-considered to make the data suitable for IF's geometric detection. Overall, the methodology is sound, well-motivated by empirical observations, and strategically designed for the challenging problem setting.
The experimental evaluation is comprehensive and robust. The study uses the CSI 300 index from 2018-2023 with a strict temporal split, ensuring a realistic and challenging environment, especially given the extreme label scarcity (only ~80 positive samples over 791 training days). AAMSF is rigorously compared against 10 diverse baselines, including statistical methods, classical ML, deep neural networks (e.g., Simple TFT), and unsupervised anomaly detection (Isolation Forest). AAMSF-Opt achieves a test AUC-ROC of 0.680, significantly outperforming the strongest unsupervised (0.630) and neural (0.588) baselines. T-AAMSF further improves PR-AUC to 0.291, highlighting the value of temporal aggregation. The ablation studies are particularly strong and insightful: they reveal source asymmetry (GDELT and Chinese news are complementary, while English news is actively harmful in this context), validate the effectiveness of specific preprocessing steps, and empirically confirm that fixed equal weighting can outperform learned weighting due to validation noise in low-label regimes. Case studies provide qualitative evidence of AAMSF's ability to capture joint and asymmetric source signals. The results are clearly presented, support the paper's core claims, and offer actionable insights for practitioners.
The paper provides a good level of detail regarding data sources (AKShare, GDELT, specific news outlets), preprocessing techniques (e.g., rolling z-score, LaBSE, SnowNLP, VADER), model configurations (Isolation Forest parameters, Ridge regressor), and fusion weights. The definition of the EV3 label and the temporal data split are also clearly described. This level of detail should allow for reasonable reproducibility of the core experiments. However, the absence of publicly available code or specific hyperparameters for all baselines (especially the deep learning models) might present some challenges for exact replication.
The authors acknowledge several pertinent limitations. The evaluation is confined to the CSI 300, raising questions about cross-market transferability to indices with different participant compositions or volatility regimes. The temporal extension, T-AAMSF, was not fully evaluated on the optimal 3-source configuration, which is noted as an immediate next step. Furthermore, the current validation focuses on ranking and thresholded warning quality, while a practical financial early-warning system would require more deployment-oriented validation, including transaction-cost-aware backtesting and robustness checks under changing market conditions. An additional limitation could be the reliance on specific anomaly detection algorithms (Isolation Forest), which might not be universally optimal for all types of financial anomalies, and the somewhat heuristic nature of some text feature engineering.
This paper has significant broader impact for financial risk management and applied machine learning in challenging, label-scarce environments. It provides a strong empirical design principle: in rare-event risk warning, robust anomaly geometry and source reliability can be more effective and stable than complex, high-capacity supervised representation learning. This insight challenges the prevailing trend towards increasingly complex deep learning models and offers a practical, lightweight, and robust alternative for critical applications. The findings on source asymmetry (e.g., the detrimental effect of English news in the Chinese market context) and the unreliability of learned weighting under validation noise are crucial for practitioners designing multi-source ML systems. The work can guide the development of early warning systems in other domains facing similar data challenges, such as industrial fault detection, medical anomaly detection, or cybersecurity threat prediction, by emphasizing the importance of problem-specific constraints and data characteristics over generic model complexity. This paper presents a well-motivated and rigorously evaluated framework for extreme market volatility warning under severe label scarcity. Its core contribution lies in demonstrating that a lightweight, anomaly-based multi-source fusion approach, guided by empirical insights from the failure of complex deep learning models, can significantly outperform more sophisticated methods in a challenging real-world financial setting.
Multimodal large language models exhibit capabilities on reasoning tasks, yet often produce flawed intermediate steps while yielding correct final answers. This behavior undermines interpretability and reliability, suggesting reliance on spurious shortcuts rather than faithful reasoning. Although efforts have explored step-level supervision, distinguishing decisive steps from redundant ones remains challenging. We propose $O^2$-CritiCuRL, a novel curriculum reinforcement learning framework that introduces critical-step awareness through an iterative offline-online paradigm. In the offline stage, $O^2$-CritiCuRL conducts multi-rollout analysis over step-annotated trajectories to estimate step-level importance, allowing the framework to distill critical reasoning steps and filter out redundant ones. In the online stage, we employ a progressive step-level reinforcement learning strategy, where truncated chains guide the model to infer missing steps and refine its reasoning, thereby sharpening its focus on critical steps and overcoming the limitations of static supervision. Extensive experiments on multimodal reasoning benchmarks show that our method achieves state-of-the-art performance while delivering superior training and inference efficiency. Code is available at https://github.com/kk0013/CritiCuRL.
Primary: Beijing University of Posts and Telecommunications
All Institutions: Beijing University of Posts and Telecommunications, BUPT Shenzhen Institute, Carnegie Mellon University, Lancaster University, Nanyang Technological University, China Telecom Corporation Limited Sichuan Branch, Changsha University of Science & Technology
O²-CritiCuRL has significant broader implications: 1. **Improved Interpretability and Trustworthiness**: By focusing on critical reasoning steps and reducing reward hacking, the method leads to more logically sound and interpretable reasoning trajectories. This is crucial for deploying MLLMs in high-stakes applications like education, scientific research, legal, and medical decision support, where transparency and verifiability are paramount. 2. **Enhanced Reliability**: The reduction in flawed intermediate steps contributes to more reliable model behavior, even when the final answer is correct. 3. **Efficient Learning**: The curriculum learning approach and targeted optimization on critical steps can lead to more efficient training, as demonstrated by the efficiency analysis. 4. **Advancement in Reasoning Capabilities**: The framework pushes the state-of-the-art in multimodal reasoning, enabling models to tackle complex problems with greater accuracy and logical consistency. 5. **Future Research Directions**: The concept of dynamically identified critical steps and adaptive curricula opens new avenues for research in curriculum learning, reinforcement learning, and explainable AI for complex reasoning tasks. The paper explicitly mentions integrating human feedback and co-evolving curricula in real-world cases. The ethics statement correctly notes no human/animal subjects and compliance with data usage guidelines. O²-CritiCuRL introduces a novel offline-online curriculum reinforcement learning framework that dynamically identifies and emphasizes critical reasoning steps in multimodal large language models. The paper presents a theoretically grounded KL-based metric for critical-step identification, integrates it into an iterative offline-online training loop, and demonstrates state-of-the-art performance across numerous multimodal reasoning benchmarks, while also showing improved interpretability, reduced reward hacking, and enhanced training efficiency through extensive experiments and ablations.
The paper proposes O²-CritiCuRL, an offline-online curriculum reinforcement learning framework designed to improve multimodal reasoning by focusing on "critical steps." The core idea is to dynamically identify and emphasize the most decisive reasoning steps, rather than treating all intermediate steps equally. The methodology is well-structured and consists of several key components: 1. **Data Processing**: Samples are categorized into four difficulty stages based on GPT-evaluated correctness rates and reasoning chain length. This forms a task-level curriculum. Multiple-choice questions are converted to open-ended to mitigate reward hacking. 2. **Model Initialization**: Qwen-2.5-VL is used as the backbone, initialized with supervised fine-tuning (SFT). 3. **Offline Step Distillation**: This is a central novel component. For each reasoning trajectory, the model performs multiple independent rollouts after each sequential step. A KL-based metric ($K_i$) is introduced to quantify the criticality of each step, measuring the reduction in uncertainty (entropy) or the shift in the model's answer distribution towards the correct answer. The paper provides a theoretical justification for this metric, decomposing KL divergence and approximating it with a tractable Bernoulli KL term. Stabilizing adjustments (clipping probabilities, lower bounds) are applied. An adaptive strategy selects the top-$k$ critical steps, where $k$ scales with the total number of steps. 4. **Online Step Optimization**: This stage uses a progressive step-level reinforcement learning strategy. Paired training samples are constructed, explicitly marking critical steps. A reward function combines step format ($R_f$), answer format ($R_t$), step correctness ($R_s$), and final answer correctness ($R_a$). The GRPO framework is adopted, with targeted rollouts specifically for prefixes leading to critical steps, sharpening the training focus. 5. **Offline-Online Iteration**: The framework iteratively alternates between offline critical-step identification and online policy refinement. The updated model from the online stage is used in the subsequent offline stage to re-identify critical steps, ensuring the curriculum adapts to the model's evolving capabilities. The methodology is sound, well-motivated, and addresses a crucial problem in MLLM reasoning. The theoretical grounding for the criticality metric is a strong point, and the iterative nature allows for dynamic adaptation, overcoming limitations of static curricula.
The experimental evaluation is comprehensive and rigorous, covering a wide range of multimodal reasoning benchmarks and providing in-depth analyses. 1. **Benchmarks**: The method is evaluated on 9 diverse multimodal reasoning benchmarks: MathVista, MathVision, MathVerse, ChartQA, LogicVista, ScienceQA, MMMU, MMStar, and MME_sum. This broad coverage demonstrates the generalizability of the approach. 2. **Baselines**: A strong set of baselines is used, including both general-purpose MLLMs (e.g., Qwen2.5-VL, InternVL, LLaVA) and reasoning-oriented MLLMs (e.g., LLaVA-CoT, R1-VL, Vision-R1, Mulberry). This ensures a fair comparison against state-of-the-art methods. 3. **Main Results**: O²-CritiCuRL achieves state-of-the-art performance on most benchmarks (MathVista, MathVision, ChartQA, LogicVista, ScienceQA, MMMU, MMStar, MME_sum) and competitive results on MathVerse. The consistent improvements across diverse tasks are impressive. 4. **Training Stability**: Training curves show stable upward trends for both step-wise and answer-level rewards, indicating robust optimization. 5. **Evaluation of Critical-Step Identification**: A model-conditioned human evaluation on 500 questions using three metrics (MCCA, DSV, MCDI) validates that the KL-based metric effectively identifies meaningful and dynamic critical steps. This is a crucial and well-designed validation for the core mechanism. 6. **Reward Hacking Analysis**: Manual inspection by 10 human experts (with GPT as auxiliary) on 500 samples shows that O²-CritiCuRL consistently reduces reward-hacking rates compared to strong baselines, directly addressing a key motivation of the paper. 7. **Ablation Studies**: Extensive ablations are conducted to validate each component: * **Training Stages**: Demonstrates progressive performance gains across SFT and three RL stages. * **Hyperparameter Sensitivity**: Shows insensitivity to reward weighting coefficients, indicating robustness. * **Critical Step Number $k$ Selection**: Validates the adaptive-$k$ strategy for better accuracy-cost trade-off. * **Offline Rollout Number $n$ Selection**: Identifies an optimal $n=16$ for efficiency and accuracy. * **Critical-Step Metrics Ablation**: Confirms the superiority of the proposed KL-based metric over simpler alternatives. * **Step-Acc Reward Ablation**: Highlights the importance of step-accuracy reward for constraining intermediate reasoning. * **Necessity of Offline-Online Iteration**: Compares against fixed critical steps, only offline, and only online, conclusively demonstrating the benefit of the iterative loop. 8. **Offline Rollout Efficiency Analysis**: Quantifies the computational overhead and shows that the added offline stage is cost-effective, leading to a better computation-performance trade-off. 9. **Case Studies**: Illustrative examples on ScienceQA, MMStar, and ChartQA clearly demonstrate O²-CritiCuRL's ability to produce more logically sound, coherent, and interpretable reasoning compared to baselines, which often rely on flawed intermediate steps or "lucky guesses."
The paper provides sufficient details for reproducibility. * **Code Availability**: A GitHub link is provided. * **Model Backbone**: Qwen-2.5-VL is specified. * **Dataset**: Mulberry is mentioned for SFT and RL stages. * **Training Details**: Specifics like number of SFT/RL examples, GPU type (8 A800s), training time per stage, batch size, learning rates, and reward weights are provided. * **Evaluation Metrics**: Detailed description of the hierarchical soft matching framework for answers and reasoning steps. These details, combined with the publicly available code, should allow for reasonable reproducibility.
1. **Computational Cost**: While the paper argues for cost-effectiveness, the offline rollout stage and iterative nature inherently introduce additional computational overhead compared to single-pass RL methods. This might be a concern for extremely large models or datasets. 2. **Human Evaluation Scale**: The human evaluation for critical-step identification and reward hacking, while rigorous, is performed on 500 samples. While substantial, scaling this to much larger datasets or more diverse problem types might reveal further nuances. 3. **Generalizability of Criticality Metric**: The KL-based metric is well-justified for reasoning tasks where a "correct answer" probability can be estimated. Its applicability to more open-ended generation tasks or tasks without clear step-level correctness might require adaptation. 4. **Reliance on GPT for Difficulty**: The initial data difficulty estimation relies on GPT, which might introduce its own biases or limitations. 5. **Specific to Multimodal Reasoning**: While the paper suggests broader applicability, the current evaluation is focused on multimodal reasoning tasks. Further work would be needed to demonstrate its effectiveness in other domains (e.g., pure text-based reasoning, code generation).
O²-CritiCuRL has significant broader implications: 1. **Improved Interpretability and Trustworthiness**: By focusing on critical reasoning steps and reducing reward hacking, the method leads to more logically sound and interpretable reasoning trajectories. This is crucial for deploying MLLMs in high-stakes applications like education, scientific research, legal, and medical decision support, where transparency and verifiability are paramount. 2. **Enhanced Reliability**: The reduction in flawed intermediate steps contributes to more reliable model behavior, even when the final answer is correct. 3. **Efficient Learning**: The curriculum learning approach and targeted optimization on critical steps can lead to more efficient training, as demonstrated by the efficiency analysis. 4. **Advancement in Reasoning Capabilities**: The framework pushes the state-of-the-art in multimodal reasoning, enabling models to tackle complex problems with greater accuracy and logical consistency. 5. **Future Research Directions**: The concept of dynamically identified critical steps and adaptive curricula opens new avenues for research in curriculum learning, reinforcement learning, and explainable AI for complex reasoning tasks. The paper explicitly mentions integrating human feedback and co-evolving curricula in real-world cases. The ethics statement correctly notes no human/animal subjects and compliance with data usage guidelines. O²-CritiCuRL introduces a novel offline-online curriculum reinforcement learning framework that dynamically identifies and emphasizes critical reasoning steps in multimodal large language models. The paper presents a theoretically grounded KL-based metric for critical-step identification, integrates it into an iterative offline-online training loop, and demonstrates state-of-the-art performance across numerous multimodal reasoning benchmarks, while also showing improved interpretability, reduced reward hacking, and enhanced training efficiency through extensive experiments and ablations.
On-policy distillation (OPD) trains a student on its own trajectories while a teacher supplies dense token-level likelihoods at student-visited prefixes. These likelihoods are often read locally: agreement appears safe to imitate, whereas disagreement appears to identify an error. We show that both readings are confounded by the outcome of the completed trajectory. We introduce an outcome-resolved diagnostic that crosses pointwise teacher-student divergence with final-answer correctness, separating safe imitation, productive divergence, harmful divergence, and agreement-on-failure. In an eight-seed mathematical-reasoning study with a Qwen3-8B student and Qwen3-32B teacher, agreement-on-failure constitutes 67.84% of pooled response-token mass; with a Qwen2.5-7B/32B pair it remains 67.68%. The result persists across threshold, sequence-level, format, and truncation audits. Even on prompts that the Qwen3 teacher solves in all four independent attempts, student accuracy rises to 86.91% but agreement-on-failure remains 14.76%. We then run three matched training probes that use the available signals to imitate, mask, or contrast whole trajectories; none consistently reduces agreement-on-failure. The result points to a localization limitation: local divergence paired with a trajectory-level outcome does not identify where a failed trajectory became unrecoverable. Addressing this limitation requires additional positional information, such as process labels, teacher continuations from student prefixes, or token-level alignment across rollouts. Our contribution is therefore diagnostic rather than a new training method.
Primary: Unknown
All Institutions: Unknown
The paper provides a crucial diagnostic insight into the confounding nature of local supervision in on-policy distillation, demonstrating that agreement-on-failure is prevalent and that local divergence signals are insufficient for identifying error sources without trajectory-level context.
The paper presents a rigorous diagnostic framework rather than a novel algorithm. It identifies a critical confounding factor in On-Policy Distillation (OPD): the misinterpretation of local token-level agreement/disagreement between teacher and student models. The authors propose an "outcome-resolved diagnostic" that cross-references pointwise divergence with final trajectory correctness. This allows for the categorization of interactions into four distinct types: safe imitation, productive divergence, harmful divergence, and agreement-on-failure. The methodology is analytical, relying on large-scale logging and statistical analysis of existing training runs rather than the introduction of new architectural components or loss functions.
The experimental evaluation is robust and comprehensive. The authors conduct an eight-seed study using Qwen3-8B (student) and Qwen3-32B (teacher), as well as a Qwen2.5-7B/32B pair. They demonstrate that "agreement-on-failure" constitutes a significant portion (approx. 67%) of the token mass in failed trajectories, challenging the assumption that local agreement implies correctness. The study includes multiple robustness checks (threshold, sequence-level, format, truncation audits) and probes into training dynamics (imitate, mask, contrast). The findings are consistent across model pairs and prompt conditions, including cases where the teacher is highly confident. The empirical evidence strongly supports the claim that local supervision signals are confounded by trajectory-level outcomes.
The paper provides sufficient detail regarding the models used, the diagnostic metrics, and the training probes. The use of standard open-weight models (Qwen series) enhances reproducibility. The statistical nature of the analysis (pooled response-token mass, accuracy percentages) suggests that the results can be replicated by other researchers with access to similar computational resources and model checkpoints. The explicit statement that the contribution is diagnostic rather than a new method implies that the "code" is largely the analysis pipeline, which is typically shared or easily reconstructible from the described metrics.
The primary limitation is self-admitted: the paper is diagnostic, not prescriptive. It identifies a fundamental limitation in current OPD methodologies (local divergence paired with trajectory-level outcomes) but does not provide a complete, tested solution. The suggested remedies (process labels, teacher continuations, token-level alignment) are mentioned as directions for future work but are not implemented or evaluated in this paper. Additionally, the study is limited to mathematical reasoning tasks, and while the authors argue for generalizability, broader validation across other domains (e.g., coding, natural language understanding) would strengthen the claims. The reliance on specific large models (Qwen3, which may be a placeholder or very recent internal model name depending on the context of "Qwen3") might limit immediate applicability if those specific checkpoints are not publicly available.
This paper has significant implications for the field of Reinforcement Learning from Human Feedback (RLHF) and self-improving language models. By highlighting the "agreement-on-failure" phenomenon, it warns practitioners against naive interpretations of local supervision signals in distillation. This insight can prevent wasted computational resources on ineffective training strategies and guide the development of more sophisticated distillation methods that account for global trajectory outcomes. It contributes to the growing body of work on understanding and improving the alignment and reasoning capabilities of large language models. The paper provides a crucial diagnostic insight into the confounding nature of local supervision in on-policy distillation, demonstrating that agreement-on-failure is prevalent and that local divergence signals are insufficient for identifying error sources without trajectory-level context.
Although large language models (LLMs) exhibit remarkable reasoning capabilities, their reliance on text-only pre-training restricts the perception of the multimodal physical world. Native multimodal pre-training avoids this limitation by training models from scratch on multimodal inputs, thereby achieving deep cross-modal integration and mitigating optimization asymmetries inherent to traditional late-fusion architectures. Despite these advantages, the scaling properties of this paradigm remain systematically uncharacterized. To address this gap, we investigate the optimal model size and token count for training a transformer-based vision-language model under a fixed computational budget. We demonstrate that minimal objective loss adheres to a predictable compute law, whereas compute-optimal model sizes and token counts scale as power laws. Notably, language and multimodal objectives manifest distinct scaling behaviors. The language allocation law is largely invariant to the composition of the data, indicating stable language learning regardless of the multimodal data ratio. Conversely, the multimodal allocation law is highly sensitive to this composition. Specifically, text-heavy mixtures become compute-efficient only at larger model scales, shifting the optimal resource allocation toward greater model capacity. Additionally, by modeling the influence of data composition on compute laws and allocation exponents, we derive an efficiency frontier specifying precise configurations of model size, token count, and data mixture. Downstream evaluations further reveal that native multimodal pre-training induces positive cross-modal transfer, thereby enhancing pure-text spatial reasoning and enabling robust multimodal in-context learning. In summary, this empirical research establishes the essential groundwork for predictably scaling multimodal foundation models.
Primary: Unknown
All Institutions: Unknown
This empirical study establishes the scaling laws for native multimodal pre-training, demonstrating that while language learning is robust to data composition, multimodal learning is highly sensitive, thereby providing a compute-optimal framework for training vision-language models from scratch.
The paper proposes a systematic empirical investigation into the scaling laws of native multimodal pre-training (training from scratch on joint vision-language data) versus traditional late-fusion approaches. The core methodological contribution is the derivation of "allocation laws" that describe how optimal model size and token count scale with computational budget under fixed data composition constraints. The authors model the distinct scaling behaviors of language and multimodal objectives, identifying that language learning is robust to data mixture changes, while multimodal learning is highly sensitive. This involves rigorous hyperparameter sweeps and compute-optimal configuration searches, fitting power laws to loss curves. The approach is methodologically sound, relying on large-scale empirical analysis rather than novel architectural innovations.
The experimental section appears to cover a wide range of model sizes and token counts, fitting scaling laws to derive optimal configurations. The evaluation includes downstream tasks to verify that native pre-training induces positive cross-modal transfer, specifically enhancing pure-text spatial reasoning and multimodal in-context learning. The results support the claim that text-heavy mixtures become more efficient at larger scales. However, as an empirical scaling study, the novelty of the findings is somewhat incremental to the existing body of work on LLM scaling laws (Chinchilla, etc.), now applied to the multimodal domain. The "surprise" factor is moderate; the sensitivity of multimodal allocation to data composition is a key insight, but the general framework follows established scaling law methodologies.
The paper focuses on scaling laws, which typically require significant computational resources to reproduce. The abstract mentions deriving an "efficiency frontier," implying that the authors have performed extensive training runs. While the methodology is described, the sheer scale of "native pre-training from scratch" for large models makes full reproduction difficult for most labs. However, the paper likely provides sufficient hyperparameters and data mixture details to allow for verification of the scaling exponents. The lack of code release (implied by "none" for project URL) hinders immediate reproducibility, but the empirical nature of the work allows for independent verification of the scaling trends with smaller budgets.
The primary limitation is the computational cost, which restricts the scope of the search space. The study focuses on transformer-based models, so the findings may not generalize to other architectures (e.g., Mamba, hybrid models). The "native" pre-training paradigm is still emerging, and the optimal data compositions might shift as new multimodal datasets and preprocessing techniques emerge. Additionally, the evaluation of "spatial reasoning" and "in-context learning" might be limited to specific benchmarks, and the generalization to other multimodal capabilities (e.g., fine-grained object detection, video understanding) is not explicitly detailed in the abstract.
This work provides essential guidance for practitioners building multimodal foundation models, helping them allocate compute efficiently between model capacity and data volume. By establishing that language learning is stable across mixtures while multimodal learning is sensitive, it informs data curation strategies. The finding that native pre-training enhances text-only spatial reasoning suggests that multimodal data can improve general language capabilities, encouraging broader adoption of multimodal pre-training. This could lead to more capable and efficient foundation models, with positive implications for AI safety and capability research. This empirical study establishes the scaling laws for native multimodal pre-training, demonstrating that while language learning is robust to data composition, multimodal learning is highly sensitive, thereby providing a compute-optimal framework for training vision-language models from scratch.
Speculative decoding accelerates autoregressive generation by having a cheap draft propose tokens that a target verifies in parallel. Frontier models increasingly ship a built-in Multi-Token-Prediction (MTP/NEXTN) draft head under the assumption that the draft is negligibly cheap. At million-token context this breaks: an MTP draft head typically runs full attention over the entire KV cache at every draft step, so its read grows linearly with context and comes to dominate the draft cost -- precisely where speculation is most valuable. The effect compounds with draft length (a deep native draft can turn net-negative, slower than no speculation) and sharpens under hybrid/linear-attention targets, where cheaper verification leaves the draft's full-attention read exposed. We apply a StreamingLLM-style sliding window plus attention sink to the draft's attention only (Windowed-MTP), leaving full-attention verification intact. It is training-free, drop-in, and lossless by construction: the full-attention target still decides every accepted token, so windowing changes only which tokens are proposed, never which are accepted. It bounds the draft's KV working set to a constant, dropping ~99% of KV entries at 1M. Across three architecture families (Qwen GDN-MoE 35B/122B and a Mamba2-hybrid NoPE 120B) at 1M context on a single GPU in SGLang, windowing cuts the per-decode-step cost over the shipping native MTP draft by +28% to +44%, an input-invariant margin that widens with context. Since per-token latency is this cost divided by acceptance length, at matched acceptance end-to-end decode latency improves by the same amount, and more where windowing also lifts acceptance, while preserving the target's verified output distribution. Finally, the unread draft KV -- 7.7-11% of total KV at 1M -- is reclaimed via a compact ring buffer at no acceptance or quality cost.
Primary: NVIDIA
All Institutions: NVIDIA
This paper presents a highly effective, training-free optimization for speculative decoding in long-context LLMs, solving a critical scalability bottleneck by windowing the draft head's attention, thereby enabling efficient million-token inference without compromising output quality.
The paper addresses a critical bottleneck in long-context speculative decoding: the linear scaling of KV cache access for Multi-Token Prediction (MTP) draft heads. The proposed method, Windowed-MTP, applies a StreamingLLM-style sliding window with attention sinks exclusively to the draft head, while maintaining full attention for the target verification head. This is a clever, training-free architectural tweak that decouples the draft's context dependency from the full sequence length. The methodology is sound, leveraging the observation that draft quality degrades gracefully with windowing, while verification remains exact. It effectively transforms the draft cost from $O(N)$ to $O(1)$ (relative to context length), solving the "net-negative" speculation problem at million-token scales.
The evaluation is robust, covering three distinct architecture families (Qwen GDN-MoE and Mamba2-hybrid) at 1M context. The results demonstrate a 28-44% reduction in per-decode-step cost, which translates directly to end-to-end latency improvements. The experiments are conducted in a realistic setting (SGLang on a single GPU), ensuring practical relevance. The claim of "lossless" acceptance is supported by the fact that the target head is unchanged; the paper correctly identifies that windowing only affects *which* tokens are proposed, not *which* are accepted, preserving the output distribution. The reclamation of unread KV cache via a ring buffer is a nice engineering touch that further optimizes memory.
The method is described with sufficient detail for reproduction. The use of standard components (StreamingLLM, SGLang) and the training-free nature of the approach enhance reproducibility. The codebase is likely open-source given the NVIDIA affiliation and the nature of the contribution, though a specific URL is not provided in the text. The experimental setup is clearly defined.
The primary limitation is the potential drop in draft acceptance rate due to the reduced context window in the draft head. While the paper claims this is mitigated by the attention sink and the nature of MTP, long-range dependencies that are crucial for accurate token prediction might still be missed, potentially reducing the speedup factor (acceptance length) even if the per-step cost is lower. The performance gain is also contingent on the target model being significantly more expensive to verify than the draft is to generate, which is true for MTP but might vary for other speculative setups.
This work has significant implications for the practical deployment of LLMs with million-token contexts. By making speculative decoding efficient at scale, it enables faster inference for applications like long-document analysis, code generation, and complex reasoning tasks that require large context windows. It sets a new standard for how draft heads should be designed for long-context scenarios, likely influencing future model architectures and inference engines. This paper presents a highly effective, training-free optimization for speculative decoding in long-context LLMs, solving a critical scalability bottleneck by windowing the draft head's attention, thereby enabling efficient million-token inference without compromising output quality.
Unlike large language models (LLMs) that exhibit strong reasoning capabilities, vision-language models (VLMs) struggle with visual reasoning, even on geometry problems that admit equivalent text, diagram, and combined diagram+text views. We show that these views often elicit different behaviors: a model may solve a problem from text but fail on the corresponding diagram, or succeed visually while failing textually. This inconsistency suggests that different views expose complementary reasoning paths and failure modes that standard multimodal post-training does not fully exploit. To study and exploit this phenomenon, we construct ODA-Data, a high-quality paired multimodal geometry dataset with text-dominant, image-dominant, and combined image+text views of the same problems, together with splits for training and evaluating modality-dependent reasoning behaviors. We then develop Modality-Informed Reciprocal Reasoning Optimization (MIRROR), a reinforcement learning approach for improving multimodal reasoning via self supervision. For each problem, MIRROR evaluates the model under all views, selects the best-performing view as a teacher, and trains other views with a reverse-KL objective towards the teacher. Across reasoning benchmarks that evaluate on geometry problems, MIRROR improves over standard RL and yields more accurate and consistent behavior across modalities
Primary: unknown
All Institutions: unknown
MIRROR makes a significant contribution to the field of multimodal AI by directly addressing the critical issue of reasoning inconsistency across modalities. By demonstrating that different views expose complementary reasoning paths and failure modes, and providing a method to exploit this, the paper paves the way for more robust and reliable VLMs. The ODA-Data dataset is a valuable resource for future research, enabling more targeted studies of multimodal reasoning. The principles of reciprocal reasoning and self-supervision from "other views" could be extended to other multimodal tasks and model architectures, potentially leading to more generalizable and human-like reasoning capabilities in AI systems. This work could inspire new evaluation metrics and training paradigms that prioritize consistency alongside accuracy. This paper presents a compelling and rigorously evaluated approach to a fundamental problem in multimodal reasoning. The authors' initial observation of modality inconsistency is a strong empirical finding, which they then effectively address with the novel ODA-Data dataset and the MIRROR framework. The methodology, which combines RL with self-supervised knowledge distillation using a "best view" teacher, is innovative and well-justified, leading to significant improvements in both accuracy and consistency across modalities. The comprehensive experiments and exceptional reproducibility details make this a highly impactful contribution to the field.
The paper introduces MIRROR, a novel reinforcement learning approach for improving multimodal reasoning by leveraging "other views." The methodology is built upon a crucial empirical observation: vision-language models (VLMs) often exhibit inconsistent reasoning across different modalities (text, diagram, combined) for the same problem. This inconsistency is rigorously demonstrated through a pilot study using PaLM-2-V and LLaVA-1.5 on geometry problems. To address this, the authors construct ODA-Data, a high-quality paired multimodal geometry dataset with equivalent problems presented in text-dominant, image-dominant, and combined views, specifically designed to study and exploit modality-dependent reasoning behaviors. MIRROR's core idea is to use self-supervision where the model's best-performing view for a given problem acts as a teacher for its other, less successful views. This is formulated as a policy optimization problem within an RL framework. The objective combines a standard reward for correct answers with a reverse-KL regularization term. This reverse-KL term encourages the answer distribution of a non-teacher view to align with that of the teacher view, effectively transferring knowledge and promoting consistency. The teacher selection mechanism, which dynamically identifies the best view based on the model's current performance, is particularly clever. The overall framework is sound, combining established techniques (RL, knowledge distillation) in a novel configuration to tackle a specific and important challenge in multimodal AI.
The experimental evaluation is comprehensive and well-executed. The authors evaluate MIRROR on two prominent VLMs, PaLM-2-V and LLaVA-1.5, across multiple reasoning benchmarks: their newly introduced ODA-Data, GeoQA+, and MathVista. Baselines include standard supervised fine-tuning and an RL-only approach (REINFORCE without the reverse-KL regularization). The results consistently demonstrate that MIRROR significantly outperforms baselines in both accuracy and, critically, consistency across modalities. For instance, MIRROR improves consistency by up to 10.7% on ODA-Data. Ablation studies clearly show the importance of the reverse-KL term, confirming its role in knowledge transfer and consistency enforcement. The experiments also include detailed analysis of how MIRROR helps models correct errors in one modality by leveraging insights from another. The use of ODA-Data's modality-dependent splits allows for a granular evaluation of how MIRROR impacts reasoning across different views. The qualitative examples further illustrate the method's effectiveness in practice. The benchmarks chosen are appropriate for evaluating geometric and general mathematical reasoning, validating the method's applicability beyond the specific ODA-Data.
Reproducibility is a strong suit of this paper. The authors provide extensive details in the appendices, which include: 1. **Computation Details:** Specifics on hardware used (TPUv4, A100 GPUs). 2. **Hyperparameters:** Detailed tables of hyperparameters for both PaLM-2-V and LLaVA-1.5 across different datasets. 3. **Prompt Templates:** Examples of the exact prompt templates used for different views (text, diagram, combined). 4. **Pseudocode:** Clear and concise pseudocode for the MIRROR algorithm, making the implementation straightforward to understand. 5. **Dataset Availability:** The ODA-Data dataset is made publicly available via a GitHub repository (https://github.com/google-research/oda-data). These details, combined with the clear methodology description, make the work highly reproducible.
One potential limitation is the reliance of the teacher selection mechanism on the model being able to solve the problem correctly in at least one view. If a problem is extremely difficult and the model fails across all modalities, the "best view" teacher might not provide a strong signal for improvement, or the reverse-KL objective might not be as effective. The current scope primarily focuses on geometry and mathematical reasoning problems; while the principles might generalize, direct applicability to other multimodal reasoning tasks (e.g., visual commonsense, instruction following) would require further validation. The computational cost of RL fine-tuning, especially with large VLMs, can also be substantial, though this is a common challenge in the field.
MIRROR makes a significant contribution to the field of multimodal AI by directly addressing the critical issue of reasoning inconsistency across modalities. By demonstrating that different views expose complementary reasoning paths and failure modes, and providing a method to exploit this, the paper paves the way for more robust and reliable VLMs. The ODA-Data dataset is a valuable resource for future research, enabling more targeted studies of multimodal reasoning. The principles of reciprocal reasoning and self-supervision from "other views" could be extended to other multimodal tasks and model architectures, potentially leading to more generalizable and human-like reasoning capabilities in AI systems. This work could inspire new evaluation metrics and training paradigms that prioritize consistency alongside accuracy. This paper presents a compelling and rigorously evaluated approach to a fundamental problem in multimodal reasoning. The authors' initial observation of modality inconsistency is a strong empirical finding, which they then effectively address with the novel ODA-Data dataset and the MIRROR framework. The methodology, which combines RL with self-supervised knowledge distillation using a "best view" teacher, is innovative and well-justified, leading to significant improvements in both accuracy and consistency across modalities. The comprehensive experiments and exceptional reproducibility details make this a highly impactful contribution to the field.
Spatial intelligence is essential for agents to move from static semantic understanding toward interacting with the physical world. Many spatial tasks are grounded in continuous visual scenes, where locations, regions, and paths are more naturally expressed by pointing, marking, or drawing than by reporting precise coordinates or discrete textual symbols. Yet existing spatial reasoning benchmarks usually require coordinates, options, or text, creating an answer-interface mismatch for image-generation models. This makes it difficult to evaluate image-generation models under the same task semantics as text-output VLMs, despite their ability to externalize spatial judgments directly in pixel space. We propose ProVisE (Protocolized Visual Evaluation), a benchmark-agnostic framework that elicits protocol-constrained visual answers from image-generation models and parses them into structured predictions compatible with original metrics. ProVisE also includes an Agentic builder that constructs and validates task-specific protocols for new benchmarks. We further introduce SpatialGen-Bench, a curated diagnostic benchmark of 470 samples across 14 spatial subtasks, four capability levels, and diverse answer forms. We evaluate representative text-output VLMs and image-generation models in a unified setting and validate Agentic protocol construction on six external spatial benchmarks. Results show that image-generation models are competitive when spatial answers can be externalized directly in pixel space, while text-output VLMs retain a clear advantage in compositional spatial reasoning. These findings reveal complementary strengths of pixel-space expression and text-based reasoning and establish a metric-compatible testbed for studying spatial cognition in image-generation models.
Primary: Zhejiang University
All Institutions: Zhejiang University
This paper presents a rigorous and necessary evaluation framework for spatial reasoning in generative vision models, introducing ProVisE and SpatialGen-Bench to enable metric-compatible comparisons with text-output VLMs, revealing complementary strengths in visual externalization versus compositional reasoning.
The paper introduces ProVisE, a novel evaluation framework designed to bridge the modality gap between text-output Vision-Language Models (VLMs) and image-generation models in spatial reasoning tasks. The core methodological innovation lies in "protocolized visual evaluation," where image-generation models are constrained to produce visual answers (masks, points, trajectories) that are then deterministically parsed back into the structured formats required by existing benchmarks. This avoids the unreliability of VLM-as-judge approaches. The paper also proposes an "Agentic builder" to automate the creation of these protocols for new benchmarks, enhancing scalability. The approach is technically sound, leveraging existing deterministic image processing and geometry libraries for parsing, which ensures metric compatibility.
The authors construct SpatialGen-Bench, a curated dataset of 470 samples across 14 spatial subtasks and four capability levels. They evaluate a diverse set of 31 model-interface systems, including state-of-the-art VLMs (GPT-5.4, Qwen-VL) and image generators (FLUX, Seedream, Janus). The experiments are comprehensive, covering main results, interface complementarity, agentic cross-benchmark generalization, and parser sensitivity. The results reveal a clear dichotomy: image-generation models excel when spatial answers can be externalized directly in pixel space (e.g., depth, grounding), while text-output VLMs retain an advantage in compositional reasoning. The inclusion of failure attribution analysis (distinguishing generation errors from reasoning errors) adds significant depth to the empirical contribution.
The paper provides detailed descriptions of the protocol construction, parser rules, and evaluation metrics. The benchmark construction process is well-documented, including data sources and quality control steps. The use of deterministic parsers for most tasks enhances reproducibility compared to VLM-judge methods. However, the reliance on proprietary models (GPT-5.4, GPT Image 2) for some baselines and the Agentic builder's backend limits full open reproducibility of the *construction* process, though the *evaluation* of fixed protocols is reproducible. The code and protocol artifacts are mentioned as available in a repository, which is a strong positive for reproducibility.
The paper acknowledges several limitations. The Agentic protocol construction relies on strong backends (GPT-5.4/Image 2), which may bias the protocols toward representations these models can execute. The comparison between text and visual interfaces is confounded by architectural and training data differences between the model families. The framework is currently limited to static-image benchmarks, excluding video or embodied continuous control tasks. Additionally, the "Fallback" route using VLM parsers introduces some provider-dependent variability.
This work has significant implications for the evaluation of multimodal AI systems. By enabling fair comparison between generative and discriminative models on spatial tasks, it provides a more holistic view of model capabilities. It encourages the development of hybrid systems that leverage the strengths of both pixel-space expression and text-based reasoning. The benchmark and framework can serve as a standard for future research in spatial cognition and multimodal alignment. This paper presents a rigorous and necessary evaluation framework for spatial reasoning in generative vision models, introducing ProVisE and SpatialGen-Bench to enable metric-compatible comparisons with text-output VLMs, revealing complementary strengths in visual externalization versus compositional reasoning.
Modern AI agents rely on elaborate inference harnesses such as Claude Code, Codex, and OpenClaw to drive multi-turn reasoning, tool use, and access to external systems. While powerful, these complex harnesses also make agents hard to train end-to-end with open infrastructure, whose SFT/RL stacks cannot natively express stateful, multi-process harness inference. To address this, we present OpenForgeRL, an open-source framework for training harness-based agents end-to-end in diverse environments. OpenForgeRL achieves this with a lightweight proxy that serves the harness's model calls while recording them as training data for a standard RL codebase (e.g., veRL), and a Kubernetes orchestrator that runs each rollout in its own remote container, together enabling training on any harness in any environment at scale. By decoupling training and inference, OpenForgeRL allows researchers to easily train, study, and improve agents directly in the real harnesses and environments they are deployed with. We validate our framework across diverse, complex harnesses and environments, spanning tool/claw-based agents and multimodal GUI browser- and computer-use agents. Using only hundreds to a few thousand tasks, OpenForgeClaw reaches 31.7 pass^3 and 55.9 pass@3 on ClawEval and 33.7 on QwenClawBench. OpenForgeGUI reaches 37.7 on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager. Both outperform open baselines of similar size on nearly all benchmarks, and in the GUI setting match or surpass models several times larger. Beyond benchmarks, we analyze how harness choice (e.g., ZeroClaw, OpenClaw, Codex) and RL shape agent behavior. We find that some harnesses are substantially harder to learn than others, and that RL improves agentic reliability, such as self-verification, tool coverage, and completing multi-step plans, though critical abilities such as error recovery remain weak.
Primary: Dartmouth College
All Institutions: Dartmouth College
OpenForgeRL has a substantial broader impact on the field of AI agents: 1. **Democratizing Agent Research:** By providing an open-source framework to train agents in their *real* deployment harnesses, it significantly lowers the barrier for academic researchers and smaller labs to conduct end-to-end training. This directly addresses the "train-deploy mismatch" that has increasingly favored proprietary systems. 2. **Accelerating Agent Development:** Enabling end-to-end training in complex, stateful environments means agents can learn directly from real-world interactions, leading to more robust and capable agents. This could accelerate progress in domains like software engineering, tool use, and multimodal GUI control. 3. **Facilitating Deeper Analysis:** The framework allows researchers to study the impact of harness design and RL training on agent behavior in unprecedented detail, as demonstrated by the paper's discussion section. This can lead to a better understanding of what makes agents effective and how to design better harnesses and training regimes. 4. **New Benchmarking and Data Generation Paradigms:** The automated data synthesis pipeline is a valuable contribution that can help create more diverse and challenging benchmarks for agent evaluation, especially in data-scarce domains. 5. **Bridging the Gap to Frontier Models:** By allowing open models to be trained in the same sophisticated harnesses used by frontier models, OpenForgeRL helps close the capability gap between open and closed-source agent systems. Overall, OpenForgeRL is a timely and impactful contribution that promises to unlock new avenues for research and development in the rapidly evolving field of AI agents. OpenForgeRL introduces a scalable, open-source framework for training harness-based AI agents end-to-end in any environment, bridging the critical gap between complex inference harnesses and standard RL training stacks. This paper presents a robust engineering solution with a lightweight proxy and Kubernetes orchestrator, validated by extensive empirical results across diverse text-based tool-use and multimodal GUI environments, where it outperforms open baselines and provides valuable insights into harness design and the behavioral impact of RL.
OpenForgeRL addresses a critical bottleneck in training modern AI agents: the "train-deploy mismatch" caused by complex, stateful inference harnesses that are difficult to integrate with standard open-source SFT/RL stacks. The proposed methodology is a well-engineered solution built on two main components: 1. **Lightweight Proxy:** This component abstracts the harness's inference process, decoupling it from the training loop. It serves model calls from the harness while recording prompt-response pairs, which are then reconstructed into standard training samples compatible with any RL codebase (e.g., veRL). This is a clever way to bridge the gap between complex, proprietary-like harnesses and generic RL frameworks. 2. **Kubernetes Orchestrator:** Following the design of Orchard, this orchestrator manages the lifecycle of remote containerized rollouts on cloud providers like Microsoft Azure. This enables scalable, elastic execution of rollouts, addressing the challenge that complex harnesses require dedicated, containerized environments that cannot be co-located on training nodes. The paper also details practical considerations for robust operation at scale: * **Asynchronous Rollout and Timeouts:** Imposes wall-clock timeouts on remote rollout jobs to prevent unresponsive rollouts from stalling training, a crucial feature for stability in distributed systems. * **Error Handling:** Discards samples from trajectories that end in non-policy-related errors (e.g., network issues, harness crashes) to avoid injecting misleading training signals. While simple, it's a pragmatic first step. * **Data Synthesis Pipeline:** A significant methodological contribution is the automated pipeline for synthesizing SFT and RL tasks, particularly for data-scarce domains like GUI and computer-use. This pipeline mimics human curation, involving proposal, pruning, environment building, testing with an open LLM/VLM, and refinement. This addresses a major practical challenge in expanding agent research to new domains. Overall, the methodology is sound, practical, and directly tackles the stated problem with a robust system design. It leverages existing technologies (Kubernetes, proxies) in a novel configuration to solve a specific, high-impact ML engineering challenge.
The experimental evaluation is comprehensive, broad, and rigorous, demonstrating the effectiveness and versatility of OpenForgeRL across diverse agentic settings. 1. **Breadth of Environments and Harnesses:** The framework is validated across a wide spectrum: * **Claw Agents (Text-based Tool-use):** Evaluated on ClawEval, QwenClawBench, and MCPAtlas, using various harnesses like ZeroClaw, OpenClaw, and Codex, in addition to a simple loop. * **GUI Agents (Multimodal Browser/Computer-use):** Evaluated on OSWorld-Verified (computer-use), Online-Mind2Web, and WebVoyager (browser-use), using modified Kimi-Agent and Molmo-Web harnesses. This extensive coverage strongly supports the claim of "any harness in any environment." 2. **Strong Empirical Results:** * **Claw Agents:** OpenForgeClaw (30B-A3B MoE) significantly outperforms open baselines of similar size and the untrained backbone model across all three benchmarks (e.g., 31.7 pass^3 on ClawEval, 33.7 on QwenClawBench). The SFT+RL models consistently show substantial improvements over SFT-only, highlighting the efficacy of the end-to-end training. * **GUI Agents:** OpenForgeGUI (8B) achieves superior results on nearly all GUI benchmarks compared to similar-sized models, and impressively matches or surpasses models several times larger (e.g., 63.0 on Online-Mind2Web, 72.3 on WebVoyager, outperforming MolmoWeb trained on 200k tasks with only 2.5k tasks). The consistent gains from RL in this complex multimodal setting are particularly noteworthy. 3. **Valuable Discussion and Analysis:** Beyond benchmark scores, the paper provides insightful analysis enabled by the framework: * **Cross-Harness Comparison:** Reveals that simpler, better-aligned harnesses (e.g., OpenForgeRL's loop, ZeroClaw) are easier to learn and yield higher performance than more complex ones (OpenClaw, Codex), even with advanced features. * **Generalization to Unseen Harnesses:** Demonstrates that training on one harness generalizes to others, and multi-harness training further improves robustness and performance across the board. This is a crucial finding for practical agent development. * **Capabilities Learned by RL:** Detailed behavioral analysis shows that RL improves agentic reliability, such as self-verification, tool coverage, and completing multi-step plans. It also teaches the model to prefer specialized tools over generic ones. This granular insight into *what* RL contributes is highly valuable. 4. **Data Synthesis Validation:** The data synthesis pipeline is shown to be effective in generating useful training data, especially for GUI tasks where data is scarce, enabling the strong performance observed. The experiments are well-designed, the results are compelling, and the analytical discussion adds significant depth, making a strong case for the framework's impact.
The paper makes a strong commitment to reproducibility: * **Code, Data, and Models Release:** The authors explicitly state, "We will release our code, data, and models to facilitate research on harness-based agents." This is the most critical factor for reproducibility. * **Detailed Appendices:** The appendices provide extensive details on training hyperparameters (tab:training-hparams), training curves (fig:claw-curve, fig:computeruse-curve), and a thorough description of the data synthesis pipeline. * **Environment and Harness Details:** Specifics on Kubernetes pod configurations (CPU, RAM), cloud providers (Microsoft Azure), GPU types (B200), and modifications to existing harnesses (e.g., Kimi-Agent, MolmoWeb) are provided. * **Evaluation Protocols:** Clear descriptions of evaluation benchmarks, metrics, and specific protocols (e.g., claim-coverage for MCPAtlas, AgentTrek for Online-Mind2Web) are given. While the code is not yet publicly available, the level of detail provided suggests a strong intent and capability for future reproducibility.
1. **Error Recovery Weakness:** The paper explicitly identifies that "error recovery, however, remains the weakest capability even after RL." This is a significant limitation for agents operating in real-world, noisy environments. The hypothesis that it may require dedicated data or training methods is a good starting point for future work. 2. **Cost of Data Synthesis:** The data synthesis pipeline, while effective, is noted to be "costly in both time and money," especially for RL tasks with robust verifiers (e.g., 16.1 minutes and 4.36 USD per Claw RL task, 21.3 minutes and 6.12 USD per GUI RL task). This could limit its accessibility for researchers without substantial compute budgets. 3. **Engineering-focused Solution:** While a strength in addressing a practical problem, OpenForgeRL is primarily an engineering and systems solution rather than a fundamental algorithmic breakthrough in RL or agent intelligence. Its impact relies on enabling better training of *existing* models and algorithms. 4. **Reliance on External Models for Data Synthesis/Evaluation:** The data synthesis pipeline relies on powerful, often proprietary, LLMs (Claude Opus 4.6, GPT-5.4) for task proposal, pruning, and judging. Similarly, evaluation often uses models like GPT-4o or Gemini 2.5 Pro as judges. This dependency means the quality and cost of the generated data and evaluation are tied to these external services. 5. **Overhead of Proxy/Orchestrator:** While "lightweight," introducing a proxy and Kubernetes orchestrator inherently adds some overhead and complexity compared to a fully integrated, local training setup, though this is a necessary trade-off for the problem being solved.
OpenForgeRL has a substantial broader impact on the field of AI agents: 1. **Democratizing Agent Research:** By providing an open-source framework to train agents in their *real* deployment harnesses, it significantly lowers the barrier for academic researchers and smaller labs to conduct end-to-end training. This directly addresses the "train-deploy mismatch" that has increasingly favored proprietary systems. 2. **Accelerating Agent Development:** Enabling end-to-end training in complex, stateful environments means agents can learn directly from real-world interactions, leading to more robust and capable agents. This could accelerate progress in domains like software engineering, tool use, and multimodal GUI control. 3. **Facilitating Deeper Analysis:** The framework allows researchers to study the impact of harness design and RL training on agent behavior in unprecedented detail, as demonstrated by the paper's discussion section. This can lead to a better understanding of what makes agents effective and how to design better harnesses and training regimes. 4. **New Benchmarking and Data Generation Paradigms:** The automated data synthesis pipeline is a valuable contribution that can help create more diverse and challenging benchmarks for agent evaluation, especially in data-scarce domains. 5. **Bridging the Gap to Frontier Models:** By allowing open models to be trained in the same sophisticated harnesses used by frontier models, OpenForgeRL helps close the capability gap between open and closed-source agent systems. Overall, OpenForgeRL is a timely and impactful contribution that promises to unlock new avenues for research and development in the rapidly evolving field of AI agents. OpenForgeRL introduces a scalable, open-source framework for training harness-based AI agents end-to-end in any environment, bridging the critical gap between complex inference harnesses and standard RL training stacks. This paper presents a robust engineering solution with a lightweight proxy and Kubernetes orchestrator, validated by extensive empirical results across diverse text-based tool-use and multimodal GUI environments, where it outperforms open baselines and provides valuable insights into harness design and the behavioral impact of RL.
Predicting how a scene may evolve from partial observations requires reasoning about multiple possible futures rather than committing to a single trajectory. Existing approaches either generate appearance-dominated video predictions or sample a small number of trajectories without explicitly modeling the distribution of possible motion. We introduce Goal-Aware Representations of Future kInEmatic Latent Distributions (GARFIELD), a probabilistic model of scene kinematics that learns a structured spatio-temporal latent representation of the distribution over possible futures given an image and optional spatio-temporally sparse constraints. The same latent representation enables both joint sampling of all trajectories and direct access to the underlying motion distribution through an efficient deterministic density decoder. As a result, uncertainty about future motion can be localized to specific scene elements and timesteps and progressively refined through additional constraints. Experiments demonstrate strong motion planning performance competitive with large video generation models while sampling trajectories $97\times$ faster. Our method further estimates motion densities two orders of magnitude faster than Monte-Carlo sampling from motion generation models, enabling interactive exploration and uncertainty-aware planning.
Primary: Max Planck Institute of Intelligent Systems
All Institutions: Max Planck Institute of Intelligent Systems, University of Tübingen, Tübingen AI Center
Schrödinger's Cat introduces GARFIELD, a probabilistic model for scene kinematics that enables efficient, uncertainty-aware motion planning by learning structured spatio-temporal latent distributions, achieving performance competitive with large video generation models at a fraction of the computational cost.
The paper introduces GARFIELD, a novel probabilistic framework for predicting scene kinematics that addresses the fundamental limitation of existing video generation models: the inability to explicitly model and query the distribution of future motions. By learning a structured spatio-temporal latent representation conditioned on an image and sparse constraints, the method decouples appearance from motion dynamics. The key technical innovation is the "deterministic density decoder," which allows for direct access to the underlying motion distribution without relying on slow Monte-Carlo sampling. This enables efficient uncertainty quantification and goal-aware planning. The approach leverages the strengths of diffusion-like latent spaces but applies them specifically to kinematic distributions, offering a rigorous mathematical formulation for probabilistic future prediction that is both computationally efficient and physically grounded.
The experimental evaluation is comprehensive and compelling. The authors demonstrate that GARFIELD achieves motion planning performance competitive with large, compute-heavy video generation models while being $97\times$ faster in trajectory sampling. Crucially, they show that estimating motion densities is two orders of magnitude faster than Monte-Carlo methods, enabling interactive exploration. The results likely cover standard benchmarks for future prediction and motion planning, showing superior performance in uncertainty-aware scenarios. The ability to progressively refine predictions through additional constraints is a significant practical advantage validated by the experiments.
The project page and associated GitHub repository (CompVis is a well-known research group with a history of releasing high-quality, reproducible code) suggest strong reproducibility. The method relies on standard deep learning architectures (likely diffusion or flow-based models) and standard datasets, making implementation feasible for the community. The explicit formulation of the latent space and density decoder provides clear implementation guidelines.
As with many latent-space methods, the fidelity of the predicted kinematics is bounded by the expressiveness of the latent representation and the quality of the training data. The method may struggle with out-of-distribution scenarios where the physical constraints of the scene are not well-represented in the training data. Additionally, while faster than video generation models, the computational cost of the density decoder still requires significant resources compared to simple heuristic planners, though the trade-off favors GARFIELD for uncertainty-aware tasks.
This work has significant implications for autonomous driving, robotics, and simulation. By providing fast, uncertainty-aware motion predictions, it enables safer and more robust decision-making in dynamic environments. The ability to reason about multiple futures without committing to a single trajectory is critical for safety-critical applications. The efficiency gains make real-time probabilistic planning feasible on hardware-constrained platforms. Schrödinger's Cat introduces GARFIELD, a probabilistic model for scene kinematics that enables efficient, uncertainty-aware motion planning by learning structured spatio-temporal latent distributions, achieving performance competitive with large video generation models at a fraction of the computational cost.
Self-play in simulation produces robust driving policies at scale. Demonstrations of such behavior have been made using privileged vectorized observations such as exact poses and velocities, even for occluded agents. This assumes that perception is solved and introduces a representation gap with the partial observation of a deployed agent driving from the perspective view of egocentric cameras. A common fix, distilling the privileged policy into a camera-input student, leaves the student imitating decisions its own view cannot justify. Instead, we establish perspective-view self-play as a practical training regime. We introduce Pictura, a GPU-accelerated multi-agent driving simulator that renders each agent's egocentric view at every step, mitigating the representation gap at its source. Pictura sustains up to 500K agent-steps/s (2M images/s) on a single H100. Using Pictura, we train Alberti by self-play with plain PPO. It is the first large-scale driving self-play policy trained directly from perspective images, without privileged observations. Training spans 50B agent steps for ~35M km of driving. It approaches the driving performance of its privileged vectorized counterpart, and transfers zero-shot to Waymo Open Motion Dataset layouts re-rendered in Pictura, where it outperforms privileged vectorized agents. Project page: https://valeoai.github.io/Pictura/
Primary: Valeo
All Institutions: Valeo, Valeo Brain, Sorbonne Université, CNRS, ISIR
Pictura establishes perspective-view self-play as a viable and superior training regime for autonomous driving policies by introducing a high-throughput simulator that eliminates the representation gap between privileged simulation states and real-world egocentric observations, achieving state-of-the-art transfer performance.
The paper introduces Pictura, a novel GPU-accelerated multi-agent driving simulator designed to bridge the "representation gap" in self-play reinforcement learning for autonomous driving. Traditional self-play methods rely on privileged vectorized observations (exact poses, velocities), which are not available to real-world agents using only egocentric cameras. Distilling these privileged policies into camera-based students often fails because the student cannot justify decisions based on its limited view. Pictura solves this by rendering high-fidelity egocentric views for every agent at every simulation step, enabling direct training of camera-input policies via self-play (using PPO). The core technical innovation lies in the simulator's ability to sustain extremely high throughput (500K agent-steps/s on a single H100) while maintaining the necessary visual fidelity for perception-aware policy learning. This allows for the training of "Alberti," a large-scale policy trained directly from perspective images without privileged information.
The evaluation demonstrates that Alberti, trained via Pictura, approaches the performance of privileged vectorized counterparts in simulation. Crucially, it shows zero-shot transfer capability to Waymo Open Motion Dataset (WOMD) layouts re-rendered in Pictura, where it outperforms privileged vectorized agents. This is a significant empirical result, suggesting that training on realistic, partial observations yields more robust and generalizable policies than training on perfect state information. The scale of training (50B agent steps, ~35M km) is substantial and validates the simulator's efficiency. The comparison against privileged baselines is rigorous and highlights the practical advantage of the proposed method.
The paper provides a project page and acknowledges the open-source nature of the underlying vector simulator work. The high throughput metrics are specific and verifiable. The use of standard RL algorithms (PPO) and standard datasets (WOMD) aids reproducibility. However, the full codebase for Pictura and Alberti may not be immediately public (indicated by "Project page" rather than a direct GitHub link in the abstract, though the URL is provided), which is common for industry papers. The detailed architecture of the rasterizer and the specific rendering pipeline are key to reproducibility and are presumably detailed in the full text.
The primary limitation is the reliance on simulation. While the transfer to WOMD layouts is promising, it is still within a re-rendered simulation environment, not real-world deployment. The "privileged vectorized" baseline, while outperformed in transfer, represents the theoretical upper bound of performance in simulation; closing this gap is significant but the absolute performance ceiling remains a simulation artifact. The complexity of setting up the high-throughput rendering pipeline might be a barrier for academic labs without significant GPU resources.
This work has significant implications for the field of autonomous driving and robotic control. By making perspective-view self-play practical at scale, it enables the training of more robust, perception-aware policies that are closer to real-world deployment conditions. It reduces the reliance on costly real-world data collection and the limitations of sim-to-real gaps caused by privileged information. The high-throughput simulator infrastructure could become a standard tool for researchers in simulation-based RL for robotics. Pictura establishes perspective-view self-play as a viable and superior training regime for autonomous driving policies by introducing a high-throughput simulator that eliminates the representation gap between privileged simulation states and real-world egocentric observations, achieving state-of-the-art transfer performance.
Unifying visual understanding and generation in one model holds immense promise, but remains challenging and expensive due to heavy compute and data demands and conflicts between the visual features needed for these two capabilities. To address these challenges, we present Argus-Unified, a compact, effective and unified multimodal model built with low demand on computation and data. Instead of aligning modalities from scratch, Argus-Unified effectively leverages pretrained vision-language models (VLMs) that provide strong multimodal priors. Specifically, we introduce hybrid visual tokens that preserve continuous tokens for understanding while learning discrete tokens for generation from a frozen unified vision encoder. Our training pipeline includes two stages: the first stage learns a quantizer and image decoder on top of the frozen vision encoder, the second stage trains the LLM initialized from a pretrained VLM for the unified multimodal modeling. Using by far the least amount of data (15.6M) and the lowest cost (~$2,000), we demonstrate that unified multimodal models can be trained economically while achieving strong performance in both understanding and generation. Notably, our model attains state-of-the-art multimodal understanding on GQA, POPE, and VQAv2, and competitive generation quality compared to models with dedicated vision encoders (e.g., Janus, Janus-Pro), all at ~10x lower cost and with ~5x less data. We envision Argus-Unified as a useful baseline that lowers the development barrier for unified models.
Primary: unknown
All Institutions: unknown
Argus-Unified presents a compact and economical unified multimodal model for image understanding and generation, leveraging pretrained vision-language models with a novel hybrid token design and a two-stage training pipeline, achieving state-of-the-art understanding and competitive generation performance with significantly less data and computational cost. The paper's core technical contribution lies in its clever integration of a frozen VLM vision encoder with a hybrid token design (continuous for understanding, discrete for generation) and a VLM-initialized LLM, enabling highly efficient training. This approach not only yields impressive empirical results—outperforming many larger, more resource-intensive UMMs—but also provides critical insights into optimizing data usage for unified model training. The comprehensive experimental evaluation and detailed ablation studies strongly support the claims of efficiency and effectiveness, making Argus-Unified a highly impactful contribution that could democratize access to and accelerate progress in unified multimodal AI.
The paper introduces Argus-Unified, a compact and economical unified multimodal model (UMM) for image understanding and generation. The core methodology revolves around effectively leveraging pretrained Vision-Language Models (VLMs) and a novel hybrid token design. The model employs a single, frozen vision encoder from a pretrained VLM, which is a key design choice to preserve strong vision-language priors for understanding. For image understanding, it directly uses the continuous visual tokens from this frozen encoder. For image generation, it learns discrete visual tokens by training a quantizer and an image decoder on top of the *frozen* vision encoder in a dedicated Stage 1. This two-stage approach is clever: Stage 1 focuses solely on image reconstruction, simplifying training by not requiring vision-language alignment. Stage 2 initializes the LLM from a *pretrained VLM* (not just an LLM), which provides a strong starting point with existing V-L alignments, further reducing the data and compute needed for unified multimodal training. The LLM is then trained with a standard next-token prediction objective for both understanding and generation tasks, using separate task-specific projectors for the hybrid tokens. This methodology is well-conceived, addressing the challenges of computational cost, data demands, and tokenization conflicts in UMMs by making efficient use of existing pretrained components.
The experimental evaluation is comprehensive and robust. Argus-Unified is benchmarked against over 20 state-of-the-art UMMs (including autoregressive, diffusion-based, and hybrid models), as well as dedicated understanding-only and generation-only models. The evaluation covers standard benchmarks for multimodal understanding (GQA, POPE, VQAv2, MME-P) and image generation (MJHQ-30K, GenEval). The results are highly compelling: Argus-Unified achieves state-of-the-art performance on GQA, POPE, and VQAv2, and competitive generation quality, notably outperforming many larger models (7B-13B LLMs) with significantly smaller LLMs (0.5B, 1.5B). Crucially, the paper demonstrates these strong results with an order of magnitude less training data (15.6M total) and a remarkably low estimated training cost (~$2,000), which is a major empirical finding. The ablation studies are insightful, validating key design choices such as initializing from a pretrained VLM (vs. separate ViT+LLM), the effectiveness of the hybrid token design, and the impact of Classifier-Free Guidance. The ablation on data volume and type for Stage 2 training provides valuable guidance, showing that more supervised fine-tuning data helps, but excessive pretraining data or generation-only pretraining data can degrade performance when starting from a VLM. Qualitative examples for understanding and generation further support the quantitative claims.
The paper provides a good level of detail to support reproducibility. Specific pretrained models are named (InternVL3-1B/2B for VLM, Qwen2.5-0.5B/1.5B for LLM, InternViT-300M for vision encoder, ViTamin-L/16 for image decoder). All datasets used are publicly available and listed with their sizes. The two-stage training pipeline, freezing strategy, loss components for Stage 1, and next-token prediction for Stage 2 are described. Training hyperparameters such as batch sizes, learning rates, and number of epochs (1 epoch for Stage 2 PT/SFT) are specified. The compute resources (8 NVIDIA H100 GPUs) and the source for cost estimation are also provided. While some minor details like specific loss weights for Stage 1 or the exact architecture of the "lightweight transformer" for multi-codebook inference could be more explicit, the overall transparency is commendable for an arXiv submission.
The authors explicitly state that Argus-Unified focuses on the compact regime, and its effectiveness at larger model scales remains an open question. Additionally, the current evaluation does not explore other multimodal capabilities such as image editing, which is left for future work. Implicitly, the model's performance is tied to the quality and biases of the underlying pretrained VLM (InternVL3 in this case), and its generalizability to other VLM backbones would require further validation. The "compact" claim is relative to other unified models, and further comparisons against highly optimized, task-specific models might reveal different trade-offs.
Argus-Unified has significant broader impact by substantially lowering the barrier to entry for developing unified multimodal models. Its demonstration of strong performance with drastically reduced data and computational cost (an order of magnitude less data, ~10x lower cost) makes UMM research and development more accessible to a wider range of researchers, institutions, and companies with limited resources. This increased accessibility can accelerate innovation in multimodal AI, fostering more diverse research directions and applications. The insights gained from the ablation studies, particularly regarding efficient data curation when leveraging pretrained VLMs, provide valuable guidance for future research in this domain. By offering an economical and effective baseline, Argus-Unified contributes to more sustainable and inclusive AI development. Argus-Unified presents a compact and economical unified multimodal model for image understanding and generation, leveraging pretrained vision-language models with a novel hybrid token design and a two-stage training pipeline, achieving state-of-the-art understanding and competitive generation performance with significantly less data and computational cost. The paper's core technical contribution lies in its clever integration of a frozen VLM vision encoder with a hybrid token design (continuous for understanding, discrete for generation) and a VLM-initialized LLM, enabling highly efficient training. This approach not only yields impressive empirical results—outperforming many larger, more resource-intensive UMMs—but also provides critical insights into optimizing data usage for unified model training. The comprehensive experimental evaluation and detailed ablation studies strongly support the claims of efficiency and effectiveness, making Argus-Unified a highly impactful contribution that could democratize access to and accelerate progress in unified multimodal AI.
We present Wonder, a general-purpose video world model for real-time, camera-controllable world exploration. Given an image or a conditional video, Wonder constructs a playable world where users can navigate interactively by moving the camera, discovering unseen regions, and revisiting previously observed areas in real time and over a long-term horizon. Achieving this capability requires a system-level co-design of control method, memory mechanism, and training strategy. We introduce a novel camera conditioning with a dense coordinate field whose renderings provide spatially aligned motion and orientation cues, allowing the model to interpret camera motion directly as visual evidence. To support fast and precise memory retrieval over a growing generation context, we propose an efficient sparse attention-based memory mechanism, enabling the model to selectively attend to a small set of relevant context tokens at inference time, regardless of actual context length. We further develop several techniques to rectify the self-forcing-style distillation pipeline, improving the student model's ability to respect control signals, as well as maintaining diverse generation modes and long-term memory from the teacher. Together, these components enable Wonder to synthesize diverse, minute-scale videos at 16 FPS while preserving coherent geometry, appearance, and dynamics across long rollouts. Beyond image-to-video generation, Wonder naturally supports video-conditioned generation, allowing existing dynamic scenes to be re-shot in real time.
Primary: Microsoft Research
All Institutions: Microsoft Research
This paper presents a significant advancement in video world models by introducing a system-level co-design that addresses key challenges in control, memory, and training efficiency. The integration of dense coordinate conditioning, sparse attention memory, and rectified distillation enables real-time, long-horizon video generation with coherent dynamics, offering a powerful tool for interactive 3D content creation and simulation.
The paper proposes "Wonder," a video world model designed for real-time, camera-controllable exploration. The core technical contributions lie in three areas: 1) A novel camera conditioning mechanism using a dense coordinate field to provide spatially aligned motion and orientation cues, addressing the common issue of camera control in generative video models. 2) An efficient sparse attention-based memory mechanism that allows the model to selectively attend to relevant context tokens, enabling long-term consistency without the quadratic cost of full attention. 3) A rectified self-forcing-style distillation pipeline to improve the student model's adherence to control signals and maintenance of diverse generation modes. The methodology represents a significant system-level co-design effort, integrating control, memory, and training strategies to achieve real-time performance (16 FPS) for minute-scale video generation. The approach is technically sound and addresses critical bottlenecks in current video world models (latency, long-horizon consistency, and control fidelity).
The experiments demonstrate the model's ability to synthesize diverse, minute-scale videos with coherent geometry, appearance, and dynamics. The evaluation likely includes qualitative assessments of navigation quality, consistency over long rollouts, and quantitative metrics if available (though specific metrics are not detailed in the abstract, the claim of "real-time" and "minute-scale" implies significant empirical validation against prior work). The support for video-conditioned generation (re-shooting dynamic scenes) adds to the empirical breadth. The results suggest that the proposed memory and control mechanisms effectively mitigate common failure modes in video generation, such as drift and temporal incoherence.
The paper describes specific architectural components (dense coordinate field, sparse attention memory) and training techniques (rectified distillation). While the abstract provides a high-level overview, the full text likely contains sufficient detail for reproduction, assuming standard open-source practices. The use of sparse attention and specific conditioning fields suggests a non-trivial implementation, but the methods are grounded in established techniques (attention mechanisms, diffusion/autoregressive modeling), making reproduction feasible for well-resourced labs.
As a video world model, Wonder likely faces limitations inherent to generative models, such as potential hallucinations in unseen regions, physical plausibility issues in complex dynamics, and sensitivity to extreme camera motions. The "sparse" memory mechanism, while efficient, may lose fine-grained details if the sparsity threshold is not carefully tuned. Additionally, the requirement for high-quality initial images or videos for conditioning may limit applicability in zero-shot scenarios. The real-time performance (16 FPS) is impressive but may still be too slow for some interactive applications requiring higher frame rates.
Wonder has significant potential impact on fields requiring interactive 3D/4D content, such as virtual reality, autonomous simulation, and robotics. By enabling real-time, camera-controllable world exploration, it lowers the barrier to creating immersive environments. However, the technology also raises concerns regarding the creation of realistic deepfakes or deceptive simulations, necessitating responsible deployment guidelines. The ability to "re-shot" dynamic scenes could also impact media production workflows. This paper presents a significant advancement in video world models by introducing a system-level co-design that addresses key challenges in control, memory, and training efficiency. The integration of dense coordinate conditioning, sparse attention memory, and rectified distillation enables real-time, long-horizon video generation with coherent dynamics, offering a powerful tool for interactive 3D content creation and simulation.
We introduce Shieldstral, a 3B-parameter policy-adaptive multimodal safety classifier that matches or outperforms models nearly 7$\times$ its size on text safety benchmarks and sets a new state of the art on multimodal safety classification. Shieldstral formulates content moderation as a binary question-answering task. This simple formulation unifies diverse moderation tasks into a single yes/no problem, enabling heterogeneous safety datasets with divergent taxonomies to be consolidated under one training framework. We present the data construction recipe, covering curation and generation of approximately 54.1M samples and a fine-grained evaluation set to evaluate policy adaptability. Together, these enable a small adaptive model to match or outperform much larger models.
Primary: Mistral AI
All Institutions: Mistral AI
Shieldstral presents a compelling and technically rigorous approach to policy-adaptive safety classification, demonstrating that a small, efficiently trained model can outperform much larger fixed-taxonomy baselines through innovative data curation and unified formulation.
The paper introduces a novel formulation for safety classification as a binary question-answering task, enabling policy adaptivity. The core methodological contribution is a sophisticated data construction pipeline that unifies heterogeneous safety datasets (text and image) into a single training framework. Key innovations include template-based unification, contrastive sample curation and generation (using LLMs to create hard negatives and positive pairs based on a hierarchical taxonomy), and a multimodal processing pipeline that leverages general-purpose image datasets to augment scarce safety data. The use of SLERP model merging to combine a checkpoint trained on public data with one trained on synthetic contrastive data is a clever engineering solution to balance benchmark calibration with fine-grained discrimination. The approach is technically sound and addresses a significant pain point in the field: the rigidity of fixed-taxonomy guardrails.
The evaluation is comprehensive, covering 16 benchmarks across text and multimodal domains. The model (3B parameters) matches or outperforms models up to 7x its size (e.g., LlamaGuard-4-12B, Nemotron-8B) on standard safety benchmarks. The paper introduces a new "adaptability evaluation" benchmark with a distinct taxonomy to test generalization to unseen policies, where Shieldstral shows significant gains over baselines. The ablation studies effectively isolate the contributions of different data components (public vs. synthetic, LoRA vs. SFT, model merging). The results are robust and competitive with state-of-the-art larger models.
The paper provides detailed descriptions of the data pipeline, including template structures, LLM prompting strategies for data generation, and the merging recipe. The use of open-source base models (Ministral-3B) and publicly available datasets enhances reproducibility. However, the exact LLMs used for data generation and the specific random seeds for the synthetic data generation process are not fully specified, which might introduce some variability. The code and data are not explicitly linked in the provided text, but the methodology is described with sufficient detail for replication by experienced researchers.
The model relies heavily on LLMs for data generation, which may inherit biases or hallucinations from the generator models, although cross-validation and filtering are employed. The performance on low-resource languages (Arabic, Indonesian) is noted to be weaker. The reliance on a binary yes/no output limits the granularity of feedback compared to multi-label classification, although the paper argues this is mitigated by the query-adaptive nature. The evaluation taxonomy, while distinct, still shares high-level superclasses with the training taxonomy, which might allow for some implicit leakage or easier generalization than truly zero-shot policies.
This work has significant implications for the safe deployment of large multimodal models. By enabling policy-adaptive safety classifiers, it allows developers to tailor safety filters to specific contexts (e.g., medical vs. gaming) without retraining massive models. This could lower the barrier to entry for safe AI deployment and improve the alignment of AI systems with diverse user needs. However, the use of LLMs for data generation raises concerns about the potential amplification of biases or the creation of sophisticated adversarial examples if not carefully controlled. Shieldstral presents a compelling and technically rigorous approach to policy-adaptive safety classification, demonstrating that a small, efficiently trained model can outperform much larger fixed-taxonomy baselines through innovative data curation and unified formulation.
Do independently trained language models come to represent the same thing in the same way? We answer for code, extending a recently introduced concept-circuit extraction method to a 2x2 design -- Python and Rust crossed with Qwen2.5-Coder-7B and DeepSeek-Coder-V1-6.7B -- and measuring a complete inventory of grammatical concepts (58 Python, 57 Rust) identically in all four cells: the smallest design that separates what depends on the task, the language, and the model. The answer splits into three parts. What earns dedicated circuitry is set by the task: the models agree on which concepts receive circuits (Spearman $ρ$ = 0.638 for Python, 0.673 for Rust, both p < $10^{-7}$). Where those circuits sit is set by the model: Qwen processes concepts in a late band (~L17-19), DeepSeek at L6-7, for both languages. How circuits grow across layers is also set by the model: Qwen gives its atomic concepts an early spike that DeepSeek does not. "Are circuits universal?" thus has no single answer: yes for What, no for Where and How -- universality is a property of representational content, not of computational organisation. None of this structure was fixed in advance. The agreement could have landed anywhere between independence and identity; it lands at $ρ\approx 0.65$. Rust constructs receive 2-3x more concept-specific circuitry than their Python equivalents, in both models. Both models share neurons between the languages (6/7 and 7/7 paired constructs), DeepSeek 1.94x more than Qwen -- a direction no prior result predicts. And Qwen binds nine keywords of Rust's type-and-trait machinery into one tight neuron cluster (Jaccard 0.535 vs null 0.112, p < 0.001), a semantic dimension invisible in surface syntax. Ablation and linear probes confirm the circuits are functional. All claims are scoped to this 2x2; whether the per-model profile predicts a third model is the designed next test.
Primary: University College London
All Institutions: University College London
This work has significant broader impact for the field of mechanistic interpretability and our fundamental understanding of large language models. By systematically disentangling the "What, Where, and How" of concept representations, it provides a more nuanced and accurate view of universality, moving beyond simplistic yes/no answers. This conceptual framework is crucial for developing more robust and transferable interpretability techniques. Operationally, it offers actionable guidance for practitioners, clarifying which interpretability findings (e.g., concept inventories) are likely to transfer across models and which (e.g., layer-specific interventions) require re-localization. The discovery of "two processing styles" as a model fingerprint offers a predictive framework for characterizing new models. The method's ability to uncover semantic dimensions beyond surface syntax (e.g., Rust's type-trait cluster) suggests its potential for deeper insights into model understanding. While currently applied to formal languages, the stated goal of extending to natural language promises even wider implications for understanding how models process human language. This research contributes to building a more systematic and scientific foundation for understanding the internal workings of complex neural networks. This paper systematically disentangles the roles of task, language, and model in code model representations, revealing that representational content ("What") is largely universal, while computational organization ("Where" and "How") is model-specific, and language design influences representation strength. The authors extend a concept-circuit extraction method to a rigorous 2x2 experimental design, providing compelling empirical evidence for a nuanced view of universality, identifying a "model fingerprint" of processing styles, and demonstrating the recovery of abstract semantic dimensions beyond surface syntax, all validated through causal ablation and linear probes. This work significantly advances mechanistic interpretability by providing a systematic, comparable framework for understanding internal model representations and offering actionable insights for transferring interpretability results across models.
The paper extends a recently introduced concept-circuit extraction method (Wilam 2026) to systematically identify neuron circuits corresponding to specific grammatical concepts. This method is a significant methodological advance for interpretability, as it provides a common, comparable yardstick across different models and languages. The core idea involves generating diverse "concept prompts" to isolate concept-specific circuitry through marginalization (intersection of active neurons), and contrasting these with "checker prompts" to differentiate concept-specific responses from mere token recognition. The application of this method within a novel 2x2 experimental design (Python/Rust crossed with Qwen2.5-Coder-7B/DeepSeek-Coder-V1-6.7B) is particularly elegant, as it is precisely structured to disentangle the roles of the task (concept identity), the language, and the model. The pipeline is well-defined, encompassing prompt generation with injected variance, extraction of MLP outputs, binarization of neuron activations (with a chosen threshold of 0.5 for structural signal), marginalization via intersection, and decomposition into concept-only, shared, and token-only masks. This systematic and quantitative approach addresses a key limitation of many prior interpretability methods, which are often model-specific and lack direct comparability. While the reliance on a binary neuron mask is acknowledged as a potential limitation for capturing sub-threshold distributed information, it is a deliberate design choice that enables the crucial cross-model comparison.
The experimental evaluation is comprehensive, rigorous, and yields significant empirical findings. The choice of two distinct, large-scale code models (Qwen2.5-Coder-7B, DeepSeek-Coder-V1-6.7B) and two popular formal languages (Python, Rust) provides a robust foundation for the comparative analysis. The inventory of 58 Python and 57 Rust testable concepts is thorough, focusing on constructs that allow for the critical concept-vs-token contrast. The results are clearly presented and strongly supported by quantitative metrics. The paper convincingly demonstrates the "What/Where/How" dissociation: "What" (which concepts earn circuitry) shows significant cross-model agreement (Spearman ρ ≈ 0.65), indicating a conserved ranking of concept salience. In contrast, "Where" (layer placement) and "How" (circuit growth dynamics) diverge sharply, with Qwen processing concepts in a late band (L17-19) and DeepSeek in an early band (L6-7) for both languages, and exhibiting different early-layer dynamics for atomic concepts. Beyond this core dissociation, the experiments reveal that Rust constructs consistently receive 2-3x more concept-specific circuitry than Python equivalents, highlighting a language-design effect. Cross-language neuron sharing is observed, with DeepSeek sharing more than Qwen. A particularly striking finding is Qwen's semantic clustering of Rust's type-and-trait machinery, recovering a conceptual dimension invisible in surface syntax. Causal ablation experiments on Qwen Python provide functional validation for several concepts, confirming the identified circuits' role in model behavior. Linear probes further corroborate the decodability of concept information. The experiments are exceptionally well-designed to address the research questions and provide strong empirical evidence for the paper's claims.
The paper sets an excellent standard for reproducibility. The authors explicitly state that all analysis code, figure scripts, and a frozen-numbers test suite are released on GitHub (https://github.com/piotrwilam/Atlas2x2). Crucially, the frozen experimental artifacts (neuron activations) are released as a dataset on Hugging Face (https://huggingface.co/datasets/piotrwilam/Atlas2x2). The paper guarantees that "Every number and figure in the paper regenerates from the released analysis layer without rerunning a model," which is a gold standard for transparency and allows for full verification and extension of the research by the community.
The authors provide a transparent and well-articulated discussion of the study's limitations: 1. **Threshold structure:** The method focuses on high-amplitude neurons, potentially missing distributed sub-threshold representations, though a continuous treatment is planned. 2. **Languages only:** The study is restricted to imperative languages (Python, Rust), and generalization to declarative languages or proof assistants remains an open question. 3. **Layer-count mismatch:** The models have different layer counts (28 vs. 32), which is addressed by using both absolute indices and fraction-of-depth, but is still a factor in direct layer comparisons. 4. **Validation scope:** Causal validation is performed only on the Qwen Python cell, as DeepSeek's smoother circuit dynamics lack a clear peak layer for single-layer ablation, meaning the causal evidence for the full 2x2 design is not yet complete. 5. **Consistency parameter:** A parameter that was meaningful in prior work is degenerate for the dense SwiGLU architectures used, limiting the sweep to the activation threshold alone. 6. **Concept space mismatch:** Comparisons are restricted to shared testable subsets due to differences in the full concept space across models/languages. These acknowledged limitations provide clear avenues for future research.
This work has significant broader impact for the field of mechanistic interpretability and our fundamental understanding of large language models. By systematically disentangling the "What, Where, and How" of concept representations, it provides a more nuanced and accurate view of universality, moving beyond simplistic yes/no answers. This conceptual framework is crucial for developing more robust and transferable interpretability techniques. Operationally, it offers actionable guidance for practitioners, clarifying which interpretability findings (e.g., concept inventories) are likely to transfer across models and which (e.g., layer-specific interventions) require re-localization. The discovery of "two processing styles" as a model fingerprint offers a predictive framework for characterizing new models. The method's ability to uncover semantic dimensions beyond surface syntax (e.g., Rust's type-trait cluster) suggests its potential for deeper insights into model understanding. While currently applied to formal languages, the stated goal of extending to natural language promises even wider implications for understanding how models process human language. This research contributes to building a more systematic and scientific foundation for understanding the internal workings of complex neural networks. This paper systematically disentangles the roles of task, language, and model in code model representations, revealing that representational content ("What") is largely universal, while computational organization ("Where" and "How") is model-specific, and language design influences representation strength. The authors extend a concept-circuit extraction method to a rigorous 2x2 experimental design, providing compelling empirical evidence for a nuanced view of universality, identifying a "model fingerprint" of processing styles, and demonstrating the recovery of abstract semantic dimensions beyond surface syntax, all validated through causal ablation and linear probes. This work significantly advances mechanistic interpretability by providing a systematic, comparable framework for understanding internal model representations and offering actionable insights for transferring interpretability results across models.