Last 7 Days (July 09 – July 15, 2026)
Robotic manipulation is inherently multi-frame: local actions may be simple in an end-effector frame, while transport, upright-object handling, and whole-body coordination are better represented in a base-aligned frame. However, modern diffusion-based visuomotor policies typically commit to a single predefined action frame, forcing one denoiser to model action distributions that are often unnecessarily complex in that frame. We propose Mixture of Frames Policy (MoF), a diffusion policy that performs synchronized action denoising across multiple coordinate frames. MoF maintains a single canonical diffusion state, re-expresses it in several task-relevant frames, applies frame-specialized denoisers, and fuses their noise predictions back in the canonical frame. To make this possible for intermediate noisy diffusion states, we introduce a column-based 6D rotation representation within an SE(3) action parameterization that supports exact, differentiable frame transformations without requiring noisy rotations to lie on the SO(3) manifold. Across nine simulated bimanual manipulation tasks, we show that the best action frame is task-dependent and that MoF improves over oracle frame selection and standard Mixture-of-Experts (MoE) baselines. We further evaluate MoF on two real-world bimanual mobile manipulation tasks, demonstrating that it outperforms all constituent single-frame baselines. Project homepage: https://mofpo.github.io
Primary: Stanford University
All Institutions: Stanford University, Apple, Toyota Research Institute
The paper presents a highly innovative and technically sound approach to improving diffusion-based visuomotor policies by leveraging multi-frame reasoning. The introduction of a column-based 6D rotation representation for exact noisy state transformation is a significant methodological advance. Comprehensive experiments in simulation and on real-world robots demonstrate substantial improvements over strong baselines, including oracle frame selection. The work is well-motivated, rigorously evaluated, and addresses a fundamental limitation in current robotic policy architectures. It is a strong candidate for top-tier publication and will likely influence future research in robotic policy learning.
The paper proposes a novel architecture for diffusion-based visuomotor policies called Mixture of Frames (MoF). The core insight is that robotic manipulation actions are naturally represented in different coordinate frames at different stages of a task (e.g., end-effector for grasping, base for transport). Standard diffusion policies fix a single action frame, forcing the denoiser to model complex, frame-dependent distributions. MoF maintains a single canonical diffusion state but applies frame-specialized denoisers in their respective frames, fusing the noise predictions back into the canonical frame. A critical technical contribution is the introduction of a column-based 6D rotation representation for SE(3) actions. This allows for exact, differentiable frame transformations of *noisy* intermediate states without requiring projection onto the SO(3) manifold, which is non-linear and lossy. This mathematical insight is elegant and solves a significant implementation hurdle for multi-frame reasoning in diffusion policies. The method is generalizable to any diffusion policy and can be implemented as a drop-in replacement or extension.
The evaluation is comprehensive and rigorous. The authors first establish the motivation by showing that action frame choice has a substantial, task-dependent impact on performance in bimanual mobile manipulation, with a 15% gap between the best and worst single frames. They then demonstrate that MoF outperforms oracle frame selection (which knows the best frame per task) and standard Mixture-of-Experts (MoE) baselines that do not operate in multiple action frames. The ablation studies are particularly strong, showing that the proposed column-based representation is essential (orthogonalization leads to collapse in ensemble variants) and that the router effectively switches between frames based on task phase. Real-world experiments on two complex bimanual mobile manipulation tasks (Pouring, Serving) further validate the approach, showing significant improvements over single-frame baselines. The use of simulated benchmarks (BiGym, DexMimicGen) and real-world validation provides a robust evidence base.
The paper provides detailed descriptions of the method, including the specific rotation representation and the fusion mechanism. The project homepage link suggests code availability. The experimental setup is well-described, including dataset sizes, training epochs, and evaluation metrics. The ablation studies provide clear evidence of component contributions. The reliance on standard simulators (BiGym) and real-world hardware (HoMMI) enhances reproducibility.
The method relies on a designer-specified set of candidate frames. While the router learns to weight them, the set must be predefined. The paper acknowledges this and suggests future work on learning or discovering frames. Additionally, the router is supervised via denoising loss, which may not perfectly align with policy performance, though the results suggest it works well in practice. The current experiments focus on smaller diffusion policies; scaling to large VLA models is noted as future work.
This work has significant implications for robotic manipulation, particularly in complex, multi-stage tasks where different coordinate systems are advantageous. By improving the robustness and performance of visuomotor policies, it contributes to the broader goal of autonomous robots capable of operating in unstructured environments. The technical contribution regarding SE(3) transformations is also valuable for the broader machine learning community working with geometric deep learning and diffusion models. The paper presents a highly innovative and technically sound approach to improving diffusion-based visuomotor policies by leveraging multi-frame reasoning. The introduction of a column-based 6D rotation representation for exact noisy state transformation is a significant methodological advance. Comprehensive experiments in simulation and on real-world robots demonstrate substantial improvements over strong baselines, including oracle frame selection. The work is well-motivated, rigorously evaluated, and addresses a fundamental limitation in current robotic policy architectures. It is a strong candidate for top-tier publication and will likely influence future research in robotic policy learning.
Compression is fundamental to intelligence. A model that can represent its training data as a short code has discovered regularities that enable generalization. Large neural networks may learn functions far simpler than their parameter counts suggest, but it is challenging to construct codes that realize this simplicity. Parameter-based methods such as quantization produce code lengths that scale with model size, insensitive to how much information the parameters store. Prequential coding bypasses this issue by compressing the training trajectory, but codes the exact data sequence regardless of how much the model learns, yielding large codes when the data has high entropy. We introduce requential coding, where a teacher model selects training samples drawn from the student's own distribution. The student's code records only these selections, which cost bits only where teacher and student disagree. The resulting code length is independent of parameter count and data entropy, and often orders of magnitude shorter than the prequential counterpart, with an advantage that grows with scale. This compression sheds light on phenomena inaccessible to prior compressors. Holding loss fixed, larger models and ensembles compress to much smaller sizes despite more parameters. Plugged into a PAC-Bayes bound, the requential code yields state-of-the-art generalization guarantees for billion-parameter LLMs, outperforming bounds built on aggressive post-training quantization even granted zero error. The bound tightens with scale in the compute-optimal regime, as models become increasingly compressible relative to dataset size. The same code predicts that models gradually overfit when trained for multiple epochs. It also isolates the learnable information in a dataset from its unpredictable, random content, revealing that lower-entropy text holds far more learnable structure than higher-entropy image data.
Primary: New York University
All Institutions: Carnegie Mellon University, New York University
Requential coding has a profound broader impact on the field of machine learning: 1. **Understanding Generalization**: It provides a rigorous, operational measure of model complexity that directly correlates with generalization performance, especially for large neural networks where traditional measures (like parameter count) are misleading. This is a critical step towards a deeper theoretical understanding of deep learning. 2. **Scaling Laws**: The finding that larger models are more compressible at fixed loss offers crucial insights into the efficiency of scaling, suggesting that larger models indeed learn simpler, more generalizable functions. This can guide future research into model architectures and training paradigms. 3. **PAC-Bayes Bounds**: Achieving state-of-the-art, non-vacuous PAC-Bayes generalization bounds for billion-parameter LLMs is a major theoretical breakthrough, making these bounds relevant for modern large-scale models. 4. **Data Curation and Selection**: The ability to distinguish learnable information from random content in datasets provides a principled metric for data quality and could inform strategies for more effective data curation and selection. 5. **Theoretical Tool**: Requential coding serves as a powerful analytical tool for diagnosing phenomena like overfitting and understanding the information flow during training, opening new avenues for theoretical and empirical investigation in deep learning. Requential coding introduces a novel, highly efficient model compression scheme that significantly advances the understanding of generalization in deep learning by providing a rigorous measure of model complexity independent of parameter count and data entropy. This method yields state-of-the-art PAC-Bayes generalization bounds for billion-parameter LLMs, reveals that larger models are more compressible, and offers new insights into overfitting and the learnable information content of datasets, thus serving as a powerful analytical tool for the field.
The paper introduces "requential coding," a novel and highly efficient method for compressing generative models. It builds upon the principles of prequential coding and relative entropy coding (REC) but addresses their fundamental limitations. Unlike parameter-based methods (e.g., quantization) that scale with model size regardless of information stored, or prequential coding that scales with dataset size and entropy, requential coding decouples code length from both. The core idea is that a "student" model generates candidate training samples from its own distribution, and a stronger "teacher" model selects one of these candidates. The student's code then records only the index of the selected sample, which costs approximately $KL(Q_t\|P_t)$ bits, where $Q_t$ is the teacher and $P_t$ is the student. This effectively distills the teacher's knowledge into the student by efficiently transmitting only the "disagreement" between them. The methodology is rigorously defined, including the encoder and decoder protocols, and a formal bound for the expected code length is derived in the appendix, showing it scales with the cumulative teacher-student KL divergence. Practical improvements like "teacher smoothing" (using an EMA of teacher checkpoints) and "iso-loss projection" (periodically resetting the teacher to the student's performance level) are introduced to further reduce the code length without sacrificing performance. The paper also provides a detailed analysis of the runtime for code length evaluation, encoding, and decoding, acknowledging the computational cost of actual transmission (which can be high due to REC's proposal generation) but emphasizing that evaluation is much more efficient. The use of a universal integer code (Elias delta) for the index ensures the decoder doesn't need prior knowledge of the KL divergence. This is a sophisticated and well-thought-out approach that significantly advances the state of the art in model compression theory.
The experimental evaluation is comprehensive and compelling, demonstrating the power of requential coding across various tasks and scales. 1. **Compression Benchmarking**: Requential coding is benchmarked against prequential coding and idealized 4-bit post-training quantization (PTQ) on autoregressive transformers (100M parameters) trained on OpenWebText, CIFAR-5M, and FineWeb. It achieves one to two orders of magnitude shorter codes than prequential coding, dominating the Pareto frontier of loss vs. code length. The prequential heuristic (a non-rigorous estimate) also falls well above requential coding. 2. **Scaling Insights**: * **Model Size**: Experiments show that larger models (from 10M to 1B parameters) are *more compressible* at a fixed loss, despite having more parameters. This is a crucial finding, as it suggests larger models learn simpler functions relative to their capacity, which parameter-based methods fail to capture. * **Ensembles**: Larger ensembles are also shown to be more compressible, reaching lower loss at a fixed code budget. 3. **Generalization Guarantees (PAC-Bayes Bounds)**: This is a highlight. Plugging the requential code length into a PAC-Bayes bound yields state-of-the-art generalization guarantees for billion-parameter LLMs. These bounds: * Improve with model scale, mirroring the true test loss, for models trained on a fixed amount of data. * Outperform even *idealized lossless 4-bit PTQ bounds* in the compute-optimal scaling regime ($D=20N$), a significant breakthrough as previous non-vacuous bounds for LLMs were based on realistic PTQ. * Tighten with scale, as the code length per parameter decays as a power law, predicting a vanishing generalization gap. 4. **Overfitting Prediction**: The requential code successfully predicts gradual overfitting under data repetition, showing the divergence between training and test loss as the complexity penalty rises. 5. **Learnable Information Content**: The method effectively isolates learnable structure from random information in datasets, revealing that lower-entropy text holds more learnable structure than higher-entropy image data, and distinguishing it from random or trivially repeating strings. This provides a principled way to rank datasets by their "epiplexity." The experiments are well-designed, the results are clearly presented (e.g., Figure 3, 4, 5), and the conclusions drawn are profound, offering new insights into deep learning phenomena that were previously inaccessible to rigorous measurement.
The paper explicitly states that "Code available at https://github.com/shikaiqiu/requential-coding". The methodology is described with sufficient detail, including pseudocode for REC and descriptions of teacher smoothing and iso-loss projection. The appendix provides full experiment details, including how code lengths are computed, datasets used, and per-figure configurations. This commitment to open-sourcing the code and providing detailed experimental setups significantly enhances reproducibility.
The authors acknowledge several limitations: 1. **Lossy Compression of Teacher**: Requential coding provides a lossless compression for the student model, which is a distillation of the teacher. If the goal is to compress a pre-trained model not trained via distillation, it becomes a lossy compression of that original model. 2. **Encoding Runtime**: While code length *evaluation* is efficient, *actually transmitting* a model via requential coding can be prohibitively slow due to the exponential scaling of REC encoding time with KL divergence. The paper primarily presents it as a tool for *evaluation* and *understanding* rather than practical transmission. 3. **Information Forgetting**: The current code length only grows with training steps and doesn't account for information that might be learned early but subsequently forgotten. Leveraging forgotten information could potentially further reduce the code length. 4. **Teacher Optimization**: The choice of teacher sequence is simple, and further gains are expected from optimizing it.
Requential coding has a profound broader impact on the field of machine learning: 1. **Understanding Generalization**: It provides a rigorous, operational measure of model complexity that directly correlates with generalization performance, especially for large neural networks where traditional measures (like parameter count) are misleading. This is a critical step towards a deeper theoretical understanding of deep learning. 2. **Scaling Laws**: The finding that larger models are more compressible at fixed loss offers crucial insights into the efficiency of scaling, suggesting that larger models indeed learn simpler, more generalizable functions. This can guide future research into model architectures and training paradigms. 3. **PAC-Bayes Bounds**: Achieving state-of-the-art, non-vacuous PAC-Bayes generalization bounds for billion-parameter LLMs is a major theoretical breakthrough, making these bounds relevant for modern large-scale models. 4. **Data Curation and Selection**: The ability to distinguish learnable information from random content in datasets provides a principled metric for data quality and could inform strategies for more effective data curation and selection. 5. **Theoretical Tool**: Requential coding serves as a powerful analytical tool for diagnosing phenomena like overfitting and understanding the information flow during training, opening new avenues for theoretical and empirical investigation in deep learning. Requential coding introduces a novel, highly efficient model compression scheme that significantly advances the understanding of generalization in deep learning by providing a rigorous measure of model complexity independent of parameter count and data entropy. This method yields state-of-the-art PAC-Bayes generalization bounds for billion-parameter LLMs, reveals that larger models are more compressible, and offers new insights into overfitting and the learnable information content of datasets, thus serving as a powerful analytical tool for the field.
Existing robotic perception is constrained by sensors that are either robot-mounted or permanently fixed in the environment, locking perception to a limited set of viewpoints. Yet as robots perform increasingly diverse tasks, the most informative viewpoint shifts from one task to the next-often somewhere onboard sensor and static infrastructure can not readily satisfy. To address this gap, we propose SensorPerch, a novel realization of active perception that decouples sensing from both the robot embodiment and the environment by treating sensors as independent physical entities that the robot can autonomously detach and re-attach within the environment. SensorPerch presents one realization of this paradigm: a lightweight, wireless, reconfigurable sensor platform that can perch on diverse surfaces, paired with a viewpoint-selection framework that determines task-optimal sensor placements. Together, these enable robots to construct task-relevant viewpoints on demand, independent of the robot's current position and available fixed infrastructure. We demonstrate the paradigm on two task classes: (i) object-coupled perception, where SensorPerch enables persistent object-state detection beyond the robot's current position, achieving successful event detection even when the robot is not nearby; and (ii) policy-coupled perception, where SensorPerch allows robots to construct diverse, policy-specific viewpoints for various policies, achieving success rates comparable to those obtained using oracle viewpoints.
Primary: Cornell University
All Institutions: Cornell University
SensorPerch represents a significant step towards more versatile and autonomous robotic systems. By decoupling perception from fixed infrastructure and robot embodiment, it enables robots to adapt their sensing capabilities dynamically to task requirements, leading to several broader impacts: 1. **Enhanced Robot Autonomy:** Robots can operate more independently in diverse, unstructured environments without relying on pre-installed cameras or human intervention for viewpoint adjustment. 2. **Improved Task Performance:** The ability to construct task-optimal viewpoints on demand can lead to higher success rates in complex manipulation, monitoring, and interaction tasks, especially in scenarios requiring persistent observation or specific perspectives. 3. **New Active Perception Paradigm:** This work establishes a new paradigm for active perception, shifting from merely adjusting onboard sensors to physically reconfiguring the sensing infrastructure itself. This could inspire future research into modular, reconfigurable robotic components beyond just sensors. 4. **Applications in Diverse Fields:** Beyond household robotics, this technology could be transformative for applications in industrial inspection, disaster response (deploying sensors in hazardous areas), construction, and even scientific exploration, where dynamic, adaptive sensing is crucial. 5. **Reduced Infrastructure Costs:** By allowing a single robot to deploy and manage its own sensing network, it could reduce the need for expensive, permanently installed camera systems or multi-robot coordination for comprehensive coverage. SensorPerch introduces a novel paradigm for active perception by treating sensors as independent, reconfigurable physical entities that robots can autonomously deploy and manage. This comprehensive system, integrating robust hardware with an intelligent software framework for task-conditioned viewpoint selection, significantly enhances robotic perception capabilities, enabling persistent object-state detection and policy-aligned viewpoints in diverse real-world tasks, thereby pushing the boundaries of robot autonomy and adaptability.
The methodology proposed in SensorPerch is a comprehensive and well-integrated system that addresses a critical limitation in robotic perception. The core idea of decoupling sensing from the robot's embodiment and fixed infrastructure by treating sensors as autonomous, reconfigurable physical entities is genuinely novel. The system comprises both robust hardware and an intelligent software framework. The hardware design is modular, lightweight (201.4g), and self-contained, featuring an onboard Raspberry Pi 4B for computation, a 2-DoF motorized gimbal for viewpoint adjustment, a compact battery for independent operation (2.25 hours), and a vacuum-based attachment mechanism for diverse surfaces. The integration of a charging dock on the robot's mobile base for autonomous recharging is a practical and essential detail for long-term operation. The cost-effectiveness ($94) is also a significant advantage. The software framework, "Where to Perch," is equally sophisticated. It leverages a hybrid scene representation combining semantic SLAM for geometric reasoning and mountable surface identification, and a radiance-field model (NeRF) for novel view synthesis. This allows for efficient evaluation of candidate viewpoints in simulation without physical deployment. Viewpoint sampling is intelligently biased towards task-relevant regions using a VLM. The task-conditioned scoring function is a flexible abstraction, instantiated for two critical use cases: object-coupled perception (using VLM queries for object state detection) and policy-coupled perception (using DINOv2 embeddings to match a policy's training distribution). Finally, the system incorporates model-based pose estimation and grasp-and-place primitives for autonomous detachment and reattachment of the sensor platforms. The overall methodology presents a complete, closed-loop system for active, reconfigurable perception.
The experimental evaluation is thorough and compelling, covering platform capabilities, task performance, and system efficiency. 1. **Platform Evaluation:** Quantitative metrics for attachment accuracy (0.42 cm, <1 degree), battery life (2.25 hours), and thermal performance demonstrate the hardware's robustness and suitability for real-world, long-horizon tasks. 2. **Task Evaluation:** This is the strongest part of the evaluation. * **Object-coupled perception:** Tested across five diverse scenarios (pot boiling, human fall, fire hazard, cup dropped, water leakage), SensorPerch consistently achieves high event detection success rates, closely matching oracle performance and significantly outperforming baselines like "wrist camera only" or "random feasible placement." This validates its ability to provide persistent, task-relevant perception beyond the robot's immediate vicinity. * **Policy-coupled perception:** Evaluated on three manipulation tasks (peeling, cutting, pouring), SensorPerch achieves success rates (80-90%) comparable to oracle viewpoints, demonstrating its capability to construct policy-specific views that align with training data distributions. This is a crucial validation for enabling robust policy execution in varied settings. * **Baselines:** The comparison against a comprehensive set of baselines (w/o 2-DoF, random, oracle, wrist-only, fixed third-person) rigorously isolates the contributions of different components and the overall system. * **Multi-platform demonstration:** The real-world demonstration with three platforms simultaneously monitoring different aspects of a breakfast preparation task (pot, cutting, human's cup) is highly impressive and showcases the system's scalability and practical utility in complex, multi-task scenarios. 3. **System Evaluation:** The latency breakdown provides valuable insights into the computational overhead. While initial scene reconstruction is time-consuming (369s), it's a one-time cost. Viewpoint sampling, synthesis, and scoring are performed in seconds (0.91s to 7.95s), and the full detach-reattach loop takes a reasonable 12.53 seconds. Streaming latency (180ms) is suitable for real-time perception, even with multiple platforms. These metrics confirm the system's practical usability.
The paper provides detailed descriptions of the hardware components, including specific models (Raspberry Pi 4B, MG90S servos, RealSense D435i, 18650 battery) and their integration. The software stack outlines the use of semantic SLAM, radiance fields (NeRF), VLM, DINOv2, and ROS. A lightweight ROS package is mentioned, which is a good step towards reproducibility. However, the absence of a public code repository or specific details on the semantic SLAM and NeRF implementation (e.g., which specific open-source projects were used, training data for NeRF, VLM model used) makes full replication challenging without further information. The experimental setup is well-described, but the exact datasets for policy training and VLM queries would be needed.
The authors acknowledge several pertinent limitations: 1. **Surface Dependence:** The vacuum-based mounting mechanism relies on the availability of flat, non-porous, rigid surfaces. This limits deployment in environments with soft, porous, or highly irregular textures (e.g., carpets, fabric, rough stone). 2. **Vacuum Reliability:** The long-term reliability of vacuum seals under various environmental conditions (dust, temperature changes, vibrations) is a potential concern, although the paper demonstrates robust attachment in its experiments. 3. **Robot Motion and Occlusions:** The current framework does not explicitly model how the robot's own motion might create or resolve occlusions, or how it affects viewpoint quality over time. This could lead to suboptimal placements if the robot itself becomes an occluder. Additional limitations could include: 4. **Scalability to Many Sensors:** While a three-sensor demo is shown, managing and coordinating a much larger fleet of reconfigurable sensors could introduce new challenges in terms of communication, power management, and optimal placement strategies. 5. **Initial Scene Reconstruction Time:** The 369-second scene reconstruction time, while a one-time cost, might be prohibitive for highly dynamic environments or those requiring frequent re-mapping.
SensorPerch represents a significant step towards more versatile and autonomous robotic systems. By decoupling perception from fixed infrastructure and robot embodiment, it enables robots to adapt their sensing capabilities dynamically to task requirements, leading to several broader impacts: 1. **Enhanced Robot Autonomy:** Robots can operate more independently in diverse, unstructured environments without relying on pre-installed cameras or human intervention for viewpoint adjustment. 2. **Improved Task Performance:** The ability to construct task-optimal viewpoints on demand can lead to higher success rates in complex manipulation, monitoring, and interaction tasks, especially in scenarios requiring persistent observation or specific perspectives. 3. **New Active Perception Paradigm:** This work establishes a new paradigm for active perception, shifting from merely adjusting onboard sensors to physically reconfiguring the sensing infrastructure itself. This could inspire future research into modular, reconfigurable robotic components beyond just sensors. 4. **Applications in Diverse Fields:** Beyond household robotics, this technology could be transformative for applications in industrial inspection, disaster response (deploying sensors in hazardous areas), construction, and even scientific exploration, where dynamic, adaptive sensing is crucial. 5. **Reduced Infrastructure Costs:** By allowing a single robot to deploy and manage its own sensing network, it could reduce the need for expensive, permanently installed camera systems or multi-robot coordination for comprehensive coverage. SensorPerch introduces a novel paradigm for active perception by treating sensors as independent, reconfigurable physical entities that robots can autonomously deploy and manage. This comprehensive system, integrating robust hardware with an intelligent software framework for task-conditioned viewpoint selection, significantly enhances robotic perception capabilities, enabling persistent object-state detection and policy-aligned viewpoints in diverse real-world tasks, thereby pushing the boundaries of robot autonomy and adaptability.
In this paper, we challenge the prevailing view that information dependency (including rote memorization) drives training data exposure to image reconstruction attacks. We show that extensive exposure can persist without rote memorization and is instead caused by a tunable connection to adversarial robustness. We begin by presenting three surprising results: (1) recent defenses that inhibit reconstruction by Model Inversion Attacks (MIAs), which evaluate leakage under an idealized attacker, do not reduce standard measures of information dependency (HSIC); (2) models that maximally memorize their training datasets remain robust to MIA reconstruction; and (3) models trained without seeing 97% of the training pixels, where recent information-theoretic bounds give arbitrarily strong privacy guarantees under standard assumptions, can still be devastatingly reconstructed by MIA. To explain these findings, we provide causal evidence that privacy under MIA arises from what the adversarial examples literature calls ``non-robust'' features (generalizable but imperceptible and unstable features). We further show that recent MIA defenses obtain their privacy improvements by unintentionally shifting models toward such features. To establish this causal relationship, we introduce Anti Adversarial Training (AT-AT), a training regime that intentionally learns non-robust features to obtain both superior reconstruction defense and higher accuracy than state-of-the-art defenses. Our results revise the prevailing understanding of training data exposure and reveal a new privacy-robustness tradeoff.
This paper fundamentally challenges the prevailing information-theoretic understanding of machine learning privacy by demonstrating that adversarial non-robust features, rather than rote memorization, are the primary driver of training data exposure in Model Inversion Attacks. The work is highly significant for the privacy and robustness communities, offering a causal explanation for why standard defenses work and introducing a novel training regime (AT-AT) that leverages this insight to improve both privacy and accuracy. While the findings are surprising and likely to reshape theoretical discussions on the relationship between robustness and privacy, the immediate impact on general ML practitioners is somewhat niche compared to foundational architectural or scaling breakthroughs, placing it in the "strong paper, likely cited widely" category rather than a field-defining landmark.
The key-value (KV) cache has become the dominant memory cost of transformer inference. It grows with batch size, context length, and depth, and at long context it, rather than the model weights, sets the ceiling on throughput. Two families of methods reduce it. Low-rank methods factor two-dimensional slices of the cache, either per-head matrices or cross-layer feature blocks, and quantization methods lower the bit-width of every entry. Neither family exploits the fact that the cache at a layer is naturally a third-order tensor whose three axes, the heads, the tokens, and the features, carry very different amounts of redundancy. We take this tensor view directly. Our method, JoLT, applies a partial Tucker decomposition that compresses only the token and feature axes while leaving the head and layer axes intact, and then restores the energy that truncation discards with a Johnson-Lindenstrauss (JL) rotated low-bit residual. A single Lagrangian dual allocates the Tucker ranks and the residual bit-widths together, per layer group and separately for keys and values, under one byte budget. The result is a near-lossless 2-3x compression: perplexity, GSM8K accuracy, and RULER needle-in-a-haystack retrieval all stay at or within statistical noise of the uncompressed baseline on both a grouped-query-attention model (Mistral-7B-v0.3) and a multi-head-attention model (LLaMA-2-13B). At 2x, JoLT reconstructs the cache to relative Frobenius error 0.009 (K) and 0.006 (V) on both architectures, roughly an order of magnitude below cross-layer SVD and 4-bit quantization. A randomized-SVD variant, FlashJoLT, delivers a 5-13x compression-time speedup at matched quality.
This paper introduces JoLT, a novel KV cache compression method that leverages tensor decomposition and Johnson-Lindenstrauss residual allocation to achieve near-lossless compression with significantly lower reconstruction error than existing low-rank or quantization baselines. The work is technically sound and addresses a critical bottleneck in LLM inference, offering a meaningful engineering advance that balances compression ratio with accuracy preservation, though it operates within the established paradigm of cache optimization rather than introducing a new foundational architecture or training paradigm.
Compression is fundamental to intelligence. A model that can represent its training data as a short code has discovered regularities that enable generalization. Large neural networks may learn functions far simpler than their parameter counts suggest, but it is challenging to construct codes that realize this simplicity. Parameter-based methods such as quantization produce code lengths that scale with model size, insensitive to how much information the parameters store. Prequential coding bypasses this issue by compressing the training trajectory, but codes the exact data sequence regardless of how much the model learns, yielding large codes when the data has high entropy. We introduce requential coding, where a teacher model selects training samples drawn from the student's own distribution. The student's code records only these selections, which cost bits only where teacher and student disagree. The resulting code length is independent of parameter count and data entropy, and often orders of magnitude shorter than the prequential counterpart, with an advantage that grows with scale. This compression sheds light on phenomena inaccessible to prior compressors. Holding loss fixed, larger models and ensembles compress to much smaller sizes despite more parameters. Plugged into a PAC-Bayes bound, the requential code yields state-of-the-art generalization guarantees for billion-parameter LLMs, outperforming bounds built on aggressive post-training quantization even granted zero error. The bound tightens with scale in the compute-optimal regime, as models become increasingly compressible relative to dataset size. The same code predicts that models gradually overfit when trained for multiple epochs. It also isolates the learnable information in a dataset from its unpredictable, random content, revealing that lower-entropy text holds far more learnable structure than higher-entropy image data.
Primary: New York University
All Institutions: Carnegie Mellon University, New York University
Requential coding has a profound broader impact on the field of machine learning: 1. **Understanding Generalization**: It provides a rigorous, operational measure of model complexity that directly correlates with generalization performance, especially for large neural networks where traditional measures (like parameter count) are misleading. This is a critical step towards a deeper theoretical understanding of deep learning. 2. **Scaling Laws**: The finding that larger models are more compressible at fixed loss offers crucial insights into the efficiency of scaling, suggesting that larger models indeed learn simpler, more generalizable functions. This can guide future research into model architectures and training paradigms. 3. **PAC-Bayes Bounds**: Achieving state-of-the-art, non-vacuous PAC-Bayes generalization bounds for billion-parameter LLMs is a major theoretical breakthrough, making these bounds relevant for modern large-scale models. 4. **Data Curation and Selection**: The ability to distinguish learnable information from random content in datasets provides a principled metric for data quality and could inform strategies for more effective data curation and selection. 5. **Theoretical Tool**: Requential coding serves as a powerful analytical tool for diagnosing phenomena like overfitting and understanding the information flow during training, opening new avenues for theoretical and empirical investigation in deep learning. Requential coding introduces a novel, highly efficient model compression scheme that significantly advances the understanding of generalization in deep learning by providing a rigorous measure of model complexity independent of parameter count and data entropy. This method yields state-of-the-art PAC-Bayes generalization bounds for billion-parameter LLMs, reveals that larger models are more compressible, and offers new insights into overfitting and the learnable information content of datasets, thus serving as a powerful analytical tool for the field.
The paper introduces "requential coding," a novel and highly efficient method for compressing generative models. It builds upon the principles of prequential coding and relative entropy coding (REC) but addresses their fundamental limitations. Unlike parameter-based methods (e.g., quantization) that scale with model size regardless of information stored, or prequential coding that scales with dataset size and entropy, requential coding decouples code length from both. The core idea is that a "student" model generates candidate training samples from its own distribution, and a stronger "teacher" model selects one of these candidates. The student's code then records only the index of the selected sample, which costs approximately $KL(Q_t\|P_t)$ bits, where $Q_t$ is the teacher and $P_t$ is the student. This effectively distills the teacher's knowledge into the student by efficiently transmitting only the "disagreement" between them. The methodology is rigorously defined, including the encoder and decoder protocols, and a formal bound for the expected code length is derived in the appendix, showing it scales with the cumulative teacher-student KL divergence. Practical improvements like "teacher smoothing" (using an EMA of teacher checkpoints) and "iso-loss projection" (periodically resetting the teacher to the student's performance level) are introduced to further reduce the code length without sacrificing performance. The paper also provides a detailed analysis of the runtime for code length evaluation, encoding, and decoding, acknowledging the computational cost of actual transmission (which can be high due to REC's proposal generation) but emphasizing that evaluation is much more efficient. The use of a universal integer code (Elias delta) for the index ensures the decoder doesn't need prior knowledge of the KL divergence. This is a sophisticated and well-thought-out approach that significantly advances the state of the art in model compression theory.
The experimental evaluation is comprehensive and compelling, demonstrating the power of requential coding across various tasks and scales. 1. **Compression Benchmarking**: Requential coding is benchmarked against prequential coding and idealized 4-bit post-training quantization (PTQ) on autoregressive transformers (100M parameters) trained on OpenWebText, CIFAR-5M, and FineWeb. It achieves one to two orders of magnitude shorter codes than prequential coding, dominating the Pareto frontier of loss vs. code length. The prequential heuristic (a non-rigorous estimate) also falls well above requential coding. 2. **Scaling Insights**: * **Model Size**: Experiments show that larger models (from 10M to 1B parameters) are *more compressible* at a fixed loss, despite having more parameters. This is a crucial finding, as it suggests larger models learn simpler functions relative to their capacity, which parameter-based methods fail to capture. * **Ensembles**: Larger ensembles are also shown to be more compressible, reaching lower loss at a fixed code budget. 3. **Generalization Guarantees (PAC-Bayes Bounds)**: This is a highlight. Plugging the requential code length into a PAC-Bayes bound yields state-of-the-art generalization guarantees for billion-parameter LLMs. These bounds: * Improve with model scale, mirroring the true test loss, for models trained on a fixed amount of data. * Outperform even *idealized lossless 4-bit PTQ bounds* in the compute-optimal scaling regime ($D=20N$), a significant breakthrough as previous non-vacuous bounds for LLMs were based on realistic PTQ. * Tighten with scale, as the code length per parameter decays as a power law, predicting a vanishing generalization gap. 4. **Overfitting Prediction**: The requential code successfully predicts gradual overfitting under data repetition, showing the divergence between training and test loss as the complexity penalty rises. 5. **Learnable Information Content**: The method effectively isolates learnable structure from random information in datasets, revealing that lower-entropy text holds more learnable structure than higher-entropy image data, and distinguishing it from random or trivially repeating strings. This provides a principled way to rank datasets by their "epiplexity." The experiments are well-designed, the results are clearly presented (e.g., Figure 3, 4, 5), and the conclusions drawn are profound, offering new insights into deep learning phenomena that were previously inaccessible to rigorous measurement.
The paper explicitly states that "Code available at https://github.com/shikaiqiu/requential-coding". The methodology is described with sufficient detail, including pseudocode for REC and descriptions of teacher smoothing and iso-loss projection. The appendix provides full experiment details, including how code lengths are computed, datasets used, and per-figure configurations. This commitment to open-sourcing the code and providing detailed experimental setups significantly enhances reproducibility.
The authors acknowledge several limitations: 1. **Lossy Compression of Teacher**: Requential coding provides a lossless compression for the student model, which is a distillation of the teacher. If the goal is to compress a pre-trained model not trained via distillation, it becomes a lossy compression of that original model. 2. **Encoding Runtime**: While code length *evaluation* is efficient, *actually transmitting* a model via requential coding can be prohibitively slow due to the exponential scaling of REC encoding time with KL divergence. The paper primarily presents it as a tool for *evaluation* and *understanding* rather than practical transmission. 3. **Information Forgetting**: The current code length only grows with training steps and doesn't account for information that might be learned early but subsequently forgotten. Leveraging forgotten information could potentially further reduce the code length. 4. **Teacher Optimization**: The choice of teacher sequence is simple, and further gains are expected from optimizing it.
Requential coding has a profound broader impact on the field of machine learning: 1. **Understanding Generalization**: It provides a rigorous, operational measure of model complexity that directly correlates with generalization performance, especially for large neural networks where traditional measures (like parameter count) are misleading. This is a critical step towards a deeper theoretical understanding of deep learning. 2. **Scaling Laws**: The finding that larger models are more compressible at fixed loss offers crucial insights into the efficiency of scaling, suggesting that larger models indeed learn simpler, more generalizable functions. This can guide future research into model architectures and training paradigms. 3. **PAC-Bayes Bounds**: Achieving state-of-the-art, non-vacuous PAC-Bayes generalization bounds for billion-parameter LLMs is a major theoretical breakthrough, making these bounds relevant for modern large-scale models. 4. **Data Curation and Selection**: The ability to distinguish learnable information from random content in datasets provides a principled metric for data quality and could inform strategies for more effective data curation and selection. 5. **Theoretical Tool**: Requential coding serves as a powerful analytical tool for diagnosing phenomena like overfitting and understanding the information flow during training, opening new avenues for theoretical and empirical investigation in deep learning. Requential coding introduces a novel, highly efficient model compression scheme that significantly advances the understanding of generalization in deep learning by providing a rigorous measure of model complexity independent of parameter count and data entropy. This method yields state-of-the-art PAC-Bayes generalization bounds for billion-parameter LLMs, reveals that larger models are more compressible, and offers new insights into overfitting and the learnable information content of datasets, thus serving as a powerful analytical tool for the field.
We present a theoretical framework to explain the emergence of inductive reasoning abilities in Transformer language models. While previous works on Transformer learning dynamics have so far been mostly tied to specific tasks, we study a generalized class of inductive tasks that unifies several synthetic tasks known in the literature, including in-context n-grams and multi-hop reasoning. In this class, we theoretically prove that the training dynamics of attention models can be confined to a highly interpretable, low-dimensional invariant manifold. On this manifold, the learning dynamics are captured by a handful of interpretable coordinates rather than millions of parameters, making both theoretical and empirical analysis more tractable. Using this framework, we characterize how data statistics govern the competition between in-context and in-weights learning, we study how random initializations determine the `winning' circuit when multiple solutions are possible, and we demonstrate that the coordinate frame associated with the manifold can be used to automatically detect which circuits have been learned in trained models. By casting circuit formation as a low-dimensional dynamical phenomenon, we take a step toward a predictive theory of how Transformers learn.
Primary: unknown
All Institutions: unknown
This work has significant broader impact potential for several areas of machine learning: 1. **Mechanistic Interpretability:** By providing a low-dimensional, interpretable coordinate system for circuit formation, the IMIR framework offers a powerful new tool for understanding the internal mechanisms of Transformers. This could lead to more systematic and granular circuit discovery. 2. **Predictive Theory of Learning:** The paper takes a concrete step towards a "predictive theory of how Transformers learn," which is a grand challenge in AI. Such a theory could enable better model development, debugging, and safety analysis. 3. **Model Design and Optimization:** A deeper understanding of circuit emergence could inform the design of more efficient architectures, training algorithms, and pruning strategies. Identifying and pruning "useless circuits" or describing useful ones with fewer parameters could reduce memory footprint and inference latency. 4. **Accelerating Research:** By allowing training to be constrained to the IMIR or its subsets, the framework could significantly accelerate experimentation and reduce computational demands for studying learning dynamics, making it easier to uncover novel phenomena. 5. **Generalization of Invariant Manifolds:** The success in identifying an invariant manifold for inductive tasks suggests that similar low-dimensional structures might exist for other tasks, opening new avenues for theoretical research into deep learning dynamics. This paper presents a groundbreaking theoretical framework, the Invariant Manifold of Inductive Reasoning (IMIR), that rigorously explains the emergence of inductive abilities in Transformers by confining their learning dynamics to a low-dimensional, interpretable subspace. The work provides a powerful new lens for mechanistic interpretability, offering a predictive theory for how Transformers learn specific reasoning circuits, and empirically demonstrates its utility in understanding circuit competition, data influence, and initialization effects on synthetic tasks.
This paper introduces a highly novel and rigorous theoretical framework, the Invariant Manifold of Inductive Reasoning (IMIR), to explain the emergence of inductive reasoning abilities in Transformers. The core methodological contribution is the theoretical proof that the training dynamics of attention models, when trained on a generalized class of "block-list tasks," can be confined to a low-dimensional invariant manifold. This manifold is constructed using interpretable "selection, writing, and action bases" that represent elementary operations within the Transformer architecture. The block-list task class itself is a valuable generalization, unifying several synthetic tasks like in-context n-grams and multi-hop reasoning. The paper makes several key assumptions (orthonormal centered embeddings, merged key-query parameterization, fixed orthogonal output-value maps, and specific data symmetries like token invariance and independent positional offsets) to achieve tractability. While these assumptions simplify the model (e.g., attention-only Transformers, fixed value projections), they are clearly stated and motivated, and the paper discusses potential extensions. The decomposition of learning dynamics into interpretable coordinates on this manifold is a significant step towards a more predictive and mechanistic understanding of Transformer learning.
The experimental evaluation, while conducted on synthetic block-list tasks rather than large-scale natural language datasets, effectively demonstrates the utility and explanatory power of the IMIR framework. The paper uses the framework to study three puzzling phenomena: 1. **Circuit Competition:** It analyzes the competition between in-context learning (ICL) and in-weights learning (IWL) in a 2-layer, single-head Transformer. The IMIR allows for an additive decomposition of learned weights into interpretable directions, showing how IWL emerges first and suppresses ICL gradients, and how data properties (rare tokens, burstiness) influence this competition. Theorems are provided to quantify these effects. 2. **Role of Initialization:** The framework is applied to understand how random initializations determine which of several possible induction circuits (e.g., using different layer pairs) "wins" in a multi-layer Transformer. The results show sharp phase transitions in initialization space, suggesting complex interactions beyond simple initial weight magnitudes. 3. **Automated Circuit Detection:** The interpretable nature and low dimensionality of the IMIR are leveraged for automated detection of inductive circuits in trained models. By projecting learned parameters onto the IMIR and ablating dimensions, the paper identifies not only the minimal necessary directions but also "aiding" directions that improve performance, a level of granularity often missed by existing methods. The experiments are well-designed to validate the theoretical claims and provide concrete examples of the framework's application. The figures (e.g., fig:icl-vs-iwl, fig:icl-competition, fig:autointerp) clearly illustrate the findings.
The paper provides a strong foundation for reproducibility. The theoretical framework is rigorously defined with formal assumptions, theorems, and proof sketches (with full proofs promised in appendices). The task class is precisely described. For the empirical evaluations, the paper refers to appendices for "full training details in sec:competing-induction-heads-training" and "complete algorithm in sec:autointerp-algo," which suggests that sufficient detail is provided for others to replicate the experiments. The architecture (attention-only Transformer) and training setup (cross-entropy loss, tied unembedding) are standard. The reliance on synthetic tasks also simplifies reproducibility compared to large-scale natural language experiments.
1. **Simplifying Assumptions:** The core theoretical results rely on several strong assumptions, including attention-only Transformers, merged key-query products, and fixed orthogonal output-value maps. While the paper discusses extensions, the full theoretical treatment for standard Transformers with FFNs, LayerNorms, and learnable value projections is not fully presented. 2. **Synthetic Tasks:** The empirical validation is exclusively on synthetic block-list tasks. While appropriate for foundational theoretical work, it remains to be seen how directly the IMIR framework and its specific basis constructions translate to the complexities of real-world natural language tasks and large-scale LLMs. 3. **Attractiveness of IMIR:** The paper proves the IMIR is invariant (trajectories initialized on it remain on it) but does not formally prove it is *attractive* (trajectories initialized off it converge to it), only showing this empirically. 4. **Scope of Inductive Reasoning:** While the "block-list task class" is a good generalization, it still represents a specific type of inductive reasoning. The framework's applicability to other forms of reasoning (e.g., logical deduction, planning) is not explored.
This work has significant broader impact potential for several areas of machine learning: 1. **Mechanistic Interpretability:** By providing a low-dimensional, interpretable coordinate system for circuit formation, the IMIR framework offers a powerful new tool for understanding the internal mechanisms of Transformers. This could lead to more systematic and granular circuit discovery. 2. **Predictive Theory of Learning:** The paper takes a concrete step towards a "predictive theory of how Transformers learn," which is a grand challenge in AI. Such a theory could enable better model development, debugging, and safety analysis. 3. **Model Design and Optimization:** A deeper understanding of circuit emergence could inform the design of more efficient architectures, training algorithms, and pruning strategies. Identifying and pruning "useless circuits" or describing useful ones with fewer parameters could reduce memory footprint and inference latency. 4. **Accelerating Research:** By allowing training to be constrained to the IMIR or its subsets, the framework could significantly accelerate experimentation and reduce computational demands for studying learning dynamics, making it easier to uncover novel phenomena. 5. **Generalization of Invariant Manifolds:** The success in identifying an invariant manifold for inductive tasks suggests that similar low-dimensional structures might exist for other tasks, opening new avenues for theoretical research into deep learning dynamics. This paper presents a groundbreaking theoretical framework, the Invariant Manifold of Inductive Reasoning (IMIR), that rigorously explains the emergence of inductive abilities in Transformers by confining their learning dynamics to a low-dimensional, interpretable subspace. The work provides a powerful new lens for mechanistic interpretability, offering a predictive theory for how Transformers learn specific reasoning circuits, and empirically demonstrates its utility in understanding circuit competition, data influence, and initialization effects on synthetic tasks.
We show how an adversarial model trainer can plant backdoors in a large class of deep, feedforward neural networks. These backdoors are statistically undetectable in the white-box setting, meaning that the backdoored and honestly trained models are close in total variation distance, even given the full descriptions of the models (e.g., all of the weights). The backdoor provides access to invariance-based adversarial examples for every input, mapping distant inputs to unusually close outputs. However, without the backdoor, it is provably impossible (under standard cryptographic assumptions) to generate any such adversarial examples in polynomial time. Our theoretical and preliminary empirical findings demonstrate a fundamental power asymmetry between model trainers and model users.
Primary: University of Ottawa
All Institutions: University of Ottawa, Bocconi University, Massachusetts Institute of Technology
This paper provides a groundbreaking theoretical proof that statistically undetectable backdoors can be planted in deep neural networks, establishing a fundamental power asymmetry between trainers and users under standard cryptographic assumptions. The rigorous connection between the Johnson-Lindenstrauss lemma, lattice-based hardness, and statistical indistinguishability offers a novel perspective on the intersection of cryptography and machine learning security, with profound implications for the trustworthiness of outsourced model training.
The paper presents a rigorous theoretical construction for planting "statistically undetectable" backdoors in deep neural networks. The core method involves modifying the distribution of the first layer's weights (a Gaussian projection matrix) via rejection sampling conditioned on the existence of a specific "backdoor" input vector that maps to a small norm. The authors leverage the Johnson-Lindenstrauss lemma and properties of the Symmetric Bounded Perceptron problem to prove that while the backdoored weight distribution is statistically close (in Total Variation Distance) to a truly random Gaussian distribution, the ability to find inputs that collide (map to similar outputs) is exponentially easier for the holder of the backdoor than for any efficient adversary. The methodology is mathematically sophisticated, relying on second-moment analysis of the number of solutions to linear constraints over discrete domains to bound the statistical distance. It successfully bridges computational complexity theory (lattice hardness) with statistical learning theory.
The experimental component is preliminary and illustrative rather than comprehensive. The authors provide a single example on Fashion-MNIST to demonstrate the concept of invariance-based adversarial examples (colliding inputs). The primary validation is theoretical, with proofs provided in the appendix. While the experiments confirm the feasibility of the construction in a low-dimensional setting, they do not scale to large modern architectures or datasets, which is consistent with the paper's focus on theoretical guarantees rather than empirical performance benchmarks. The lack of extensive empirical evaluation on larger models is a limitation for assessing practical exploitability, though not for the theoretical claim.
The paper includes a reproducibility statement and mentions source code in supplementary materials. The mathematical proofs are detailed and self-contained. The construction of the backdoor sampler is algorithmically defined. However, the "preliminary" nature of the experiments and the specific parameter regimes required for the theoretical bounds (often involving high compression ratios or specific discrete input domains) may limit direct reproducibility of the *effect* in standard deep learning pipelines without careful tuning. The theoretical results are reproducible in the sense that they are provable theorems.
The primary limitation is the theoretical nature of the work; it does not demonstrate a break in state-of-the-art models (e.g., ResNet, Transformers) on large-scale datasets. The attack relies on specific architectural constraints (frozen Gaussian first layer, bi-Lipschitz subsequent layers) which, while common in theory, may not perfectly capture all practical training dynamics. Furthermore, the attack requires influence over the randomness generation during training, which is a specific threat model (MLaaS). The "undetectability" is statistical, meaning that with infinite data or computational power, detection is possible, albeit with vanishingly small advantage for efficient algorithms.
This work has significant implications for the trustworthiness of Machine-Learning-as-a-Service (MLaaS) and foundation models. It proves a fundamental power asymmetry: a malicious trainer can embed capabilities (or vulnerabilities) that are statistically invisible to users but computationally exploitable by the trainer. This challenges the assumption that white-box access to model weights allows users to fully verify model integrity. It underscores the critical importance of secure randomness generation and trusted execution environments in the ML supply chain. The dual-use nature (potential for authentication/watermarking vs. malicious backdooring) highlights the need for robust auditing mechanisms. This paper provides a groundbreaking theoretical proof that statistically undetectable backdoors can be planted in deep neural networks, establishing a fundamental power asymmetry between trainers and users under standard cryptographic assumptions. The rigorous connection between the Johnson-Lindenstrauss lemma, lattice-based hardness, and statistical indistinguishability offers a novel perspective on the intersection of cryptography and machine learning security, with profound implications for the trustworthiness of outsourced model training.
Many real-world processes can be represented as compositions of functions along a directed acyclic graph (DAG). In causal modelling, these correspond to the underlying mechanisms; in engineering, to multiple fidelity levels; and in gene-regulatory networks, to transcription factors. These functions are partially observed across the DAG, with noisy and heterogeneously sampled measurements, posing significant challenges for reconstruction, uncertainty propagation, and inference. To tackle these challenges, we place priors over functions and naturally arrive at Deep Gaussian Processes over DAGs. We theoretically study their prior-collapse behaviour, and the effect of graph topology and intermediate observations on the preservation of information. We obtain almost-sure lower bounds on the asymptotic frequency of depths at which the distinction between inputs is preserved, identify broad kernel classes for which these hold, and prove an observation by \cite{dunlop2018} on the role of input connections. We offer a structured variational approximation that retains graph dependencies, preserves compositional uncertainty, and captures the explaining-away behaviour of colliders. Finally, we empirically validate our theoretical results and our methodology, and model a latent-collider DAG, a protein signalling network, and a multi-fidelity heavy-ion collision emulation task, attaining state-of-the-art performance while recovering low-fidelity contributions and yielding interpretability of the simulator hierarchy.
Primary: University of Cambridge (inferred from UKRI grant EP/Y014650/1 and ERC Synergy project OCEAN, typically associated with DAMTP or Statistics Dept at Cambridge)
All Institutions: University of Cambridge, UKRI
The paper introduces a rigorous theoretical and practical framework for Deep Gaussian Processes on Directed Acyclic Graphs, effectively unifying multi-fidelity modeling and causal latent variable models while providing novel variational inference techniques to handle complex posterior dependencies.
The paper proposes Deep Gaussian Processes on Directed Acyclic Graphs (DAG-DGPs), a significant generalization of standard Deep Gaussian Processes (DGPs) which are typically restricted to chain architectures. The core methodological innovation lies in defining GP priors over latent functions at each node of a DAG, allowing for multi-parent dependencies via flexible fusion kernels (additive, product, ANOVA). Crucially, the authors develop a structured variational inference scheme (DAG-SVI) that respects the moralized graph of the DAG to capture "explaining-away" dependencies between co-parents, which mean-field approximations fail to do. They also provide rigorous theoretical analysis on prior collapse, proving that intermediate observations and specific graph topologies (high indegree/outdegree) can prevent the vanishing contrast problem inherent in deep compositions. This is a theoretically sound and methodologically sophisticated extension of Bayesian non-parametrics.
The empirical evaluation is strong and diverse, covering synthetic tasks (latent colliders) and two real-world scientific applications: a protein signaling network (Sachs dataset) and a multi-fidelity heavy-ion collision emulation task. The results demonstrate that DAG-SVI outperforms both mean-field variants and existing multi-fidelity GP baselines (like GMGP) in terms of RMSE and CRPS. The ability to recover low-fidelity contributions via Shapley values in the heavy-ion task adds significant interpretability, which is a key selling point for scientific machine learning. The experiments are well-designed to validate the specific claims about explaining-away and uncertainty propagation.
The paper provides detailed mathematical formulations for the priors, fusion kernels, and variational bounds. The complexity analysis and algorithmic details for sparse Cholesky and Takahashi inversion are included. However, as is common with complex Bayesian models, full reproducibility relies on the specific implementation of the structured variational inference, which is non-trivial. The authors acknowledge the code availability for the heavy-ion data but do not explicitly link a public repository for the DAG-DGP framework itself in the text provided, though the mathematical detail is sufficient for a competent practitioner to implement.
The primary limitation is computational complexity. While sparse approximations are discussed, the cost of maintaining dense covariance structures over cliques in the moralized graph can still be prohibitive for very large or densely connected DAGs. The paper assumes the DAG structure is known or pre-elicited; joint structure learning with the latent functions is noted as future work, which is a significant practical constraint. Additionally, the theoretical bounds are asymptotic and rely on specific kernel classes, limiting their immediate applicability to arbitrary kernel choices without further analysis.
This work has broad implications for causal discovery, multi-fidelity modeling, and systems biology. By providing a unified framework for compositional probabilistic modeling on DAGs, it enables more accurate uncertainty quantification in complex scientific simulations where data is sparse and heterogeneous. The focus on interpretability (recovering low-fidelity contributions) makes it particularly valuable for domains where understanding the mechanism is as important as prediction. The paper introduces a rigorous theoretical and practical framework for Deep Gaussian Processes on Directed Acyclic Graphs, effectively unifying multi-fidelity modeling and causal latent variable models while providing novel variational inference techniques to handle complex posterior dependencies.
Additive activation steering (injecting a scaled residual-stream direction during generation) is calibrated almost entirely in single-turn chat, yet the models it targets are increasingly deployed as tool-using ReAct agents. We present the first systematic chat-to-agent transfer study of additive steering, coupling behavioral measurement with a representation read-out in a matched-information design: the same items rendered as plain chat or as a ReAct tool-use episode, with matched-norm random-direction controls and the transcript re-encoded every turn to exclude KV-cache contamination. Transfer is real but rescaled, and the right description is a dissociation: the injected direction reaches the late layers at near-full strength in every setting and model tested (install-site agent-over-chat ratios 0.83-1.16 across three families), while the behavioral coupling is reset per model and context. On Qwen2.5-7B a refusal bypass vector amplifies in the agent (T = 1.45, CI [1.20, 1.78], N = 300); across a powered uniform-protocol distribution the coupling spans amplification (Gemma-2-9B T = 2.00) to attenuation (Yi-1.5-9B T = 0.43, CI [0.29, 0.60]), with no universal constant and a single clean attenuator against a universal sign. Directional ablation of the same axis does not amplify (T = 0.93, CI including 1) while additive injection amplifies (T = 1.50), a 20.1-point gain difference (CI [13.4, 26.8]) that identifies an additive-specific mechanism. Two pre-registered instruments converge to localize the rescaling to the ReAct format scaffold, before any tool observation, rather than to the observation boundary where a dilution account would predict it. The safety implication is immediate and unpredictable: agentic deployment amplifies steering-based refusal bypass by up to 2.00x on some models while others attenuate, so a deployment cannot assume a given model is safe under additive steering.
Primary: Unknown
All Institutions: Unknown
The safety implications are immediate and profound. The finding that agentic deployment can amplify steering-based refusal bypass by up to 2.00x on some models while attenuating it on others means that chat-calibrated safety evaluations cannot be assumed to transfer to agentic deployments. This unpredictability poses a significant challenge for deploying safe LLM agents. For white-box jailbreaks, additive steering could be strictly more effective against agentic deployments on amplifying models. Conversely, the representational survival result is good news for activation-level safety monitoring, as chat-trained probes might still be effective. However, behavioral control mechanisms calibrated in chat will need recalibration in the deployment context. This work significantly advances our understanding of how LLM internal mechanisms behave in complex, multi-turn agentic settings, highlighting the need for context-aware safety engineering and evaluation. This paper presents the first systematic and highly rigorous study of chat-to-agent transfer for additive activation steering, revealing a critical dissociation where representational steering survives but behavioral coupling is unpredictably rescaled per model and context. Its comprehensive methodology, strong empirical findings, and immediate safety implications make it a significant contribution to the field of LLM safety and interpretability.
The methodology is exceptionally rigorous and well-designed. The paper introduces a "matched-information five-rung ladder" (C0-C4) to systematically vary the deployment wrapper from plain chat to multi-turn ReAct with a real tool, while keeping the harmful instruction byte-identical. This allows for precise isolation of context effects. A setting-invariant parser-based metric is used for behavioral measurement, ensuring comparability across different rungs. Crucially, the paper employs six pre-registered controls: (A) matched-norm random directions to establish direction-specificity, (B) KV-cache recompute to exclude multi-turn degradation mechanisms, (C) capability-matched perturbation to rule out generic residual-stream effects, (D) extraction asymmetry for robustness, (E) phase-restricted injection for localization, and (F) ablation vs. addition to differentiate mechanisms. The use of item-paired bootstrap CIs and pre-registration for every experiment significantly enhances the credibility and scientific rigor of the findings. The "uniform-protocol distribution" across multiple model families further strengthens the generalization claims.
The experimental evaluation is comprehensive and robust, spanning multiple LLM families (Qwen2.5-7B, Llama-3.1-8B, Gemma-2-9B-IT, Yi-1.5-9B, etc.). The headline finding of a "dissociation" is strongly supported: the injected direction survives at near-full or above-chat strength in the late layers (install-site agent-over-chat ratios 0.83-1.16), while the behavioral coupling is reset per model and context. Specific results are striking: refusal bypass amplifies on Qwen2.5-7B (T=1.45) and Gemma-2-9B (T=2.00), while it attenuates on Yi-1.5-9B (T=0.43), demonstrating no universal constant or sign for transfer. The paper rigorously shows that this amplification is specific to the additive mechanism, with a 20.1-point gain difference over directional ablation, which does not amplify. Two pre-registered instruments (nested input-frame ablation and forward-pass phase restriction) converge to localize the behavioral rescaling to the ReAct format scaffold, *before* any tool observation, falsifying a dilution account. The paper also successfully falsifies the output-level baseline-shift account (Bigelow et al., 2025) on refusal induction. The detailed analysis of the "uniform-protocol coupling distribution" across eight families provides a rich empirical landscape of model-specific behaviors.
Reproducibility is a major strength of this paper. Every behavioral experiment was pre-registered, with live/die criteria and verdict-mapping engines committed to a public repository before data collection. Specific SHA revisions for primary model weights are pinned, and additional model runs are logged. Detailed logging of every rollout (prompt, activations, action, observation, parse status, score, seed) is performed. Scripts for reproducing figures and tables, including the coupling estimator and mechanism analyses, are explicitly mentioned as available. The use of greedy decoding ensures deterministic rollouts, with randomness confined to item bootstrapping. This level of transparency and detail in pre-registration and implementation is exemplary.
The paper acknowledges several limitations. Most mechanism experiments are primarily conducted on Qwen2.5-7B, though cross-model replication on Llama-3.1-8B and Gemma-2-9B-IT supports generalization of key findings. The sycophancy attenuation is reported as a point estimate, with its confidence interval spanning 1, meaning it's not statistically significant on its own, though the cross-behavior interaction is. The "no universal sign" claim for coupling distribution is primarily anchored on a single attenuating family (Yi-1.5-9B), which required a finer-ladder pass to measure. Three roster families failed to yield a coupling estimate due to saturation or near-floor/ceiling refusal rates. Finally, the agent evaluation uses a custom deterministic-tool harness, and external validity on benchmarks like tau2-bench or SHADE-Arena is reserved for future work.
The safety implications are immediate and profound. The finding that agentic deployment can amplify steering-based refusal bypass by up to 2.00x on some models while attenuating it on others means that chat-calibrated safety evaluations cannot be assumed to transfer to agentic deployments. This unpredictability poses a significant challenge for deploying safe LLM agents. For white-box jailbreaks, additive steering could be strictly more effective against agentic deployments on amplifying models. Conversely, the representational survival result is good news for activation-level safety monitoring, as chat-trained probes might still be effective. However, behavioral control mechanisms calibrated in chat will need recalibration in the deployment context. This work significantly advances our understanding of how LLM internal mechanisms behave in complex, multi-turn agentic settings, highlighting the need for context-aware safety engineering and evaluation. This paper presents the first systematic and highly rigorous study of chat-to-agent transfer for additive activation steering, revealing a critical dissociation where representational steering survives but behavioral coupling is unpredictably rescaled per model and context. Its comprehensive methodology, strong empirical findings, and immediate safety implications make it a significant contribution to the field of LLM safety and interpretability.
We propose contrastive order learning (ConOrd), a contrastive learning framework for ordinal regression that integrates the strengths of contrastive learning and order learning. While contrastive learning effectively leverages all samples in a batch, it typically ignores the inherent ordering among rank labels. Conversely, order learning explicitly models label ordinality but often relies on local, margin-based comparisons, limiting its ability to capture global ordinal structure. ConOrd addresses these limitations by introducing a contrastive order loss with soft affinity and disparity weights based on rank differences, enabling fine-grained modeling of ordinal relationships across all sample pairs within a batch. Extensive experiments on a range of ordinal regression tasks, including facial age estimation, blind image quality assessment, and blind video quality assessment, demonstrate that ConOrd consistently achieves state-of-the-art performance and generalizes well across diverse ordinal regression scenarios. The source code is available at https://github.com/cwlee00/ConOrd.
Primary: Korea University
All Institutions: Samsung Electronics, Korea University
**METHODOLOGY_ASSESSMENT:** The paper introduces Contrastive Order Learning (ConOrd), a novel and principled framework that elegantly merges the strengths of contrastive learning and order learning for ordinal regression. The core methodological contribution is the "contrastive order loss" ($L_{ConOrd}$), which innovatively assigns soft affinity ($a_{ij}$) and disparity ($b_{ij}$) weights to *all* sample pairs within a batch, based on their rank differences. This design addresses key limitations of prior work: unlike traditional supervised contrastive learning, ConOrd explicitly models the inherent ordering of labels; and unlike margin-based order learning or hard-thresholding contrastive methods (e.g., RnC), it captures fine-grained, global ordinal structure through continuous, rank-dependent interactions. The chosen quadratic forms for affinity and disparity weights, combined with negative squared Euclidean distance for similarity, are justified by rigorous gradient analysis in the appendix, demonstrating how they induce stable, rank-aware attraction and repulsion forces that are localized to informative pairs. The integration of a standard center loss ($L_{center}$) further enhances intra-class compactness, and the use of $k$-NN for inference is a practical and well-established approach in order learning. The overall framework is coherent, theoretically well-supported, and represents a significant advancement in designing loss functions for ordinal regression. **EXPERIMENTAL_EVALUATION:** The experimental evaluation is exceptionally comprehensive and compelling, providing strong evidence for ConOrd's generality and effectiveness. The authors rigorously test ConOrd across a remarkably diverse range of ordinal regression tasks, including facial age estimation (6 datasets), blind image quality assessment (5 datasets), blind video quality assessment (5 datasets), temperature prediction, gaze direction estimation, and deep imbalanced regression (DIR) benchmarks. This breadth of evaluation is a major strength, validating the claim of a "general framework." ConOrd consistently achieves state-of-the-art performance across nearly all benchmarks, often with substantial improvements over strong baselines, including recent transformer-based methods, multi-modal approaches (e.g., OrdinalCLIP, NumCLIP), and imbalance-aware techniques (SLACE). The experiments are conducted under fair conditions, often using identical backbone architectures and evaluation protocols as the baselines. The ablation studies are thorough and insightful, systematically dissecting the contributions of different contrastive learning schemes, the center loss, and various design choices for the affinity/disparity weights and similarity functions. These ablations empirically validate the proposed design and provide a deeper understanding of ConOrd's mechanisms. Visualizations of feature similarity matrices further support that ConOrd learns more faithful and structured ordinal representations. **REPRODUCIBILITY:** The paper explicitly provides a link to the source code (`https://github.com/cwlee00/ConOrd`), which is crucial for reproducibility. Furthermore, the appendix offers extensive details on experimental setups, including specific datasets, backbone architectures, optimizers, learning rate schedules, and data augmentation strategies for each task. This level of detail, coupled with the public code, indicates a strong commitment to reproducibility. **LIMITATIONS:** While ConOrd demonstrates impressive performance, a few potential limitations exist. The "all-pairs" comparison in the loss function, while effective for capturing global ordinal structure, inherently leads to quadratic computational complexity with respect to the batch size. Although this is manageable for typical batch sizes, it could become a bottleneck for extremely large batches or in scenarios requiring massive numbers of pairwise comparisons. Similarly, the $k$-NN inference, while simple and effective, can be computationally intensive for very large training datasets, potentially requiring approximate nearest neighbor search methods for scalability. The paper does not provide a detailed analysis of the computational overhead of training or inference compared to some of the faster baselines. While the paper claims insensitivity to the specific functional form of the weights, the exact values of hyperparameters like the temperature parameter $\tau$, the small constant $\epsilon$ in the affinity weight, and the weight $\lambda$ for the center loss are not extensively ablated, and their optimal tuning might still be task-dependent. **BROADER_IMPACT:** ConOrd offers a robust and general framework for ordinal regression, a fundamental problem with wide-ranging applications in various real-world domains. Its consistent state-of-the-art performance across diverse tasks, from facial age estimation to image/video quality assessment and medical diagnostics (implied by general ordinal regression), suggests significant potential for practical deployment. The method's ability to learn fine-grained ordinal relationships could lead to more accurate, reliable, and nuanced predictive models. The authors have commendably included an "Impact Statement," acknowledging the ethical considerations associated with deploying models involving human attributes, particularly concerning potential biases in training data. Their recommendation for fair evaluations and using the model as a decision-support tool rather than an autonomous system is a responsible and necessary stance for this type of research, promoting ethical AI development. This work provides a flexible and powerful foundation for future advancements in ordinal representation learning. ConOrd introduces a novel contrastive order learning framework that effectively integrates ordinal structure into batch-wise representation learning through a soft, rank-aware, all-pairs contrastive loss. The paper presents a theoretically sound methodology with rigorous gradient analysis and demonstrates consistent state-of-the-art performance across an extensive and diverse set of ordinal regression benchmarks, making a significant contribution to the field of representation learning for structured prediction tasks.
Policy-based approaches to Bayesian experimental design (BED) allow the learning of deep policy networks that adaptively make intelligent design decisions based on previously collected data. However, the training of such policies is often held back by a fundamental challenge: the double intractability of the expected information gain (EIG). This necessitates expensive or complex approximations that restrict the effort one can invest in optimising the policy itself. To address this, we show that the double intractability of the EIG can be isolated from the policy learning by first solving a score matching problem that is independent of the policy used, then using the learned score approximation to train the policy in a singly intractable manner. This turns the key multiplicative cost into an additive one and reduces the computational burden on the policy training itself, making it far cheaper to train the policy multiple times when needed, e.g. for architecture search, hyperparameter tuning, or avoiding local optima. In our experiments we train multiple competitive policies without inducing a multiplicative cost in likelihood evaluations, which can increase performance by allowing us to select the best policy even without performing hyperparameter or architecture searches.
Primary: University of Oxford
All Institutions: University of Oxford
The paper presents a theoretically sound and empirically validated method for decoupling the double intractability of EIG in Bayesian Experimental Design using score matching, enabling efficient policy training and hyperparameter search.
The paper proposes a novel two-stage framework for Bayesian Experimental Design (BED) that decouples the double intractability of the Expected Information Gain (EIG) from policy training. By deriving an exact gradient expression for the EIG, the authors identify that the intractable terms (Stein and Fisher scores of the marginal likelihood) are independent of the policy parameters. They then employ a score matching objective (Marginal Score Matching) to learn these scores upfront using a neural network. This transforms the computationally expensive nested sampling required by methods like PCE into a fixed, additive cost, allowing for efficient policy training and hyperparameter search. The theoretical derivation is rigorous, and the separation of concerns is a significant conceptual contribution to the field of amortized inference and experimental design.
The experimental evaluation covers four distinct tasks: source location finding (2D and 3D) and three dynamical systems (stochastic pendulum, cart-pole, double link). The results demonstrate that the proposed method (SM-BED) achieves competitive or superior performance compared to state-of-the-art baselines including PCE, JointVarPost, and variational marginal/posterior approaches. Crucially, the experiments highlight the efficiency gain: SM-BED allows for the cheap training of multiple policy restarts (P=50), which improves final performance by selecting the best policy, a capability not afforded by baselines with multiplicative costs. The bias-variance analysis further validates the method, showing that gradient bias is a key determinant of performance and that SM-BED effectively manages this bias through score network capacity.
The paper provides detailed algorithmic descriptions, including the specific score matching loss and network architecture (TNP-style with attention). The experimental setup is well-described, with clear definitions of computational budgets (NLE). The inclusion of supplementary material with proofs and additional ablation studies enhances reproducibility. However, the reliance on specific dynamical system simulations and the potential instability in score network training (noted in the dynamical systems experiments) suggests that careful implementation is required.
The method assumes explicit, differentiable likelihoods and continuous design/observation spaces, limiting its applicability to implicit models or discrete designs without further adaptation. The score network training can be unstable, as noted in the dynamical systems experiments, requiring multiple seeds and budget splitting. Additionally, while the method reduces the *multiplicative* cost, the upfront cost of training the score network can still be significant for very high-dimensional problems, although it is amortized across policies.
This work has significant implications for fields requiring efficient, adaptive experimental design, such as active learning, clinical trials, and physical sciences. By making policy-based BED more computationally tractable, it enables the use of more complex, non-myopic policies that were previously infeasible due to computational constraints. This could lead to more data-efficient scientific discovery and better decision-making in resource-constrained environments. The paper presents a theoretically sound and empirically validated method for decoupling the double intractability of EIG in Bayesian Experimental Design using score matching, enabling efficient policy training and hyperparameter search.
Speculative decoding accelerates sampling from an autoregressive LLM by using a faster auxiliary model to draft tokens which are then verified in parallel by the LLM. Standard speculative decoding is lossless: its rejection and resampling steps exactly preserve the LLM's sampling distribution. Recent work argues that relaxing this strict guarantee can yield further speed-ups, controlled capability-speed trade-offs, or even capability gains. We practically investigate training-free relaxed speculative decoding techniques, unify existing approaches within a shared framework, benchmark them on contemporary settings, and distil takeaways and empirical findings for practitioners. Important takeaways include: relaxation can require considerable capability evaluation unlike lossless speculative decoding, and many relaxed approaches rely on a drafter that is a good language model, making them unsuited for lightweight dedicated multi-token-prediction drafters.
Primary: Graphcore
All Institutions: Graphcore
This paper has significant broader impact for the LLM inference community. It provides crucial practical guidance for researchers and practitioners considering relaxed speculative decoding. The findings challenge common assumptions and narratives, particularly regarding the utility of relaxation with lightweight multi-token-prediction (MTP) drafters, which are increasingly prevalent. The emphasis on optimizing draft length, understanding the impact of drafter capability, and accounting for generation length changes will directly inform deployment strategies. The finding that relaxation parameters do not generalize across tasks highlights the need for careful, task-specific evaluation, potentially saving significant engineering and evaluation costs. The paper's unified framework and rigorous evaluation methodology could serve as a template for future empirical investigations in LLM inference optimization. This paper provides a highly practical and rigorous investigation into training-free relaxed speculative decoding, unifying existing approaches, benchmarking them comprehensively, and distilling critical empirical findings for practitioners. The paper's strength lies in its systematic evaluation across diverse drafter types and inference settings, revealing that many relaxed approaches are unsuited for lightweight multi-token-prediction drafters and that relaxation requires considerable capability evaluation, offering invaluable guidance for LLM inference optimization.
The paper presents a robust methodological approach for investigating training-free relaxed speculative decoding techniques. A key strength is the introduction of a unified framework (alg:relaxed-spec-dec) that cleanly describes and compares diverse existing approaches by parameterizing relaxed target distributions for rejection, residual sampling, and bonus sampling. This framework is highly valuable for organizing and understanding the fragmented literature. The paper also provides a clear and practical model for understanding and quantifying speed-up (eq:speedup, eq:speedup-approx, eq:gen-len-speedup), which explicitly accounts for drafter cost, accepted draft length, and crucially, changes in overall generation length due to relaxation—a detail often overlooked in prior work. The decision to focus on training-free methods is a reasonable scope limitation, making the investigation directly applicable without additional training overhead. The detailed discussion of factors influencing speed-up and the emphasis on a flexible speed-up proxy model for practitioners are excellent.
The experimental evaluation is exceptionally thorough and well-designed, directly addressing shortcomings in prior literature. The authors benchmark a range of relaxed speculative decoding approaches on contemporary settings, using three diverse drafter+verifier pairs: a dedicated multi-token-prediction (MTP) module (Qwen3.5 MTP+27B), a weak language model drafter (Qwen3 0.6B+32B), and a strong language model drafter (Qwen3 8B+32B). This choice is critical for understanding the impact of drafter capability. The evaluation spans a wide range of inference-time parameters, including different draft lengths ($N_{draft}$) and relative drafter costs ($c_{rel}$), allowing for a holistic view of speed-up. Importantly, the paper uses modern, challenging reasoning benchmarks (AIME 2024, GPQA Diamond, LiveCodeBench Lite v6), which are more representative of real-world LLM applications than older summarization datasets. The use of repeated measurements to reduce noise and the consideration of standard stochastic sampling parameters (top-p/k) further enhance the rigor. The comparison of empirical results against the narratives of the original papers is a valuable contribution, providing a reality check on theoretical claims.
The paper explicitly states that code is available at `https://github.com/graphcore-research/relaxed-spec-dec-repro`, which is a strong indicator of reproducibility. The methodology and experimental setup are described in sufficient detail within the main text and are promised to be fully elaborated in the appendix (e.g., specific alpha values, evaluation harness). The use of standard models (Qwen, Llama) and benchmarks further aids reproducibility.
The paper acknowledges several limitations, such as focusing only on training-free, per-token relaxation methods and autoregressive drafters. While the speed-up proxy model is valuable, it relies on estimated $c_{rel}$ values rather than direct end-to-end latency measurements for all scenarios, which might be a limitation for practitioners seeking precise real-world numbers. The specific choice of verifier models (Qwen 3.5 27B/32B, Llama 3.1 70B) and benchmarks, while modern, still represents a subset of the vast LLM landscape. The paper also notes that the additional memory overhead for relaxed approaches can become notable for small models with large vocabulary sizes at larger batch sizes, which is a practical consideration.
This paper has significant broader impact for the LLM inference community. It provides crucial practical guidance for researchers and practitioners considering relaxed speculative decoding. The findings challenge common assumptions and narratives, particularly regarding the utility of relaxation with lightweight multi-token-prediction (MTP) drafters, which are increasingly prevalent. The emphasis on optimizing draft length, understanding the impact of drafter capability, and accounting for generation length changes will directly inform deployment strategies. The finding that relaxation parameters do not generalize across tasks highlights the need for careful, task-specific evaluation, potentially saving significant engineering and evaluation costs. The paper's unified framework and rigorous evaluation methodology could serve as a template for future empirical investigations in LLM inference optimization. This paper provides a highly practical and rigorous investigation into training-free relaxed speculative decoding, unifying existing approaches, benchmarking them comprehensively, and distilling critical empirical findings for practitioners. The paper's strength lies in its systematic evaluation across diverse drafter types and inference settings, revealing that many relaxed approaches are unsuited for lightweight multi-token-prediction drafters and that relaxation requires considerable capability evaluation, offering invaluable guidance for LLM inference optimization.
Sparse Mixture-of-Experts (MoE) models have become an important approach for scaling Large Language Models (LLMs), but their inference efficiency depends strongly on expert activation patterns. Speculative decoding (SD) accelerates autoregressive generation by verifying multiple draft tokens in parallel, yet existing draft selection strategies primarily optimize acceptance likelihood. In large-scale MoE models, however, selecting draft tokens also determines the union of experts activated during verification. We observe that confidence-driven SD can introduce \textit{expert scattering}: high-probability draft tokens may route to disjoint experts, increasing expert-weight memory traffic and reducing the speedup from speculation. Motivated by this observation, we revisit draft-tree selection under the non-uniform memory-cost structure of MoE inference. We propose \textsc{EcoSpec}, a cost-aware speculative decoding framework that incorporates predicted marginal expert activation cost into draft selection. With a lightweight expert predictor and a dynamic expert buffer, \textsc{EcoSpec} favors draft paths that preserve high acceptance likelihood while reusing experts already covered by the current verification set, without modifying the target-model verification rule. We evaluate \textsc{EcoSpec} on three large-scale MoE models, including DeepSeek-V3.1 (671B), Qwen3-235B-A22B, and GPT-OSS-120B, across reasoning, coding, question-answering, and dialogue benchmarks. \textsc{EcoSpec} consistently reduces active expert footprints and improves end-to-end decoding speed, achieving up to $1.62\times$ speedup. These results show that accounting for expert activation cost is important for efficient speculative decoding in large-scale MoE models.
This paper introduces EcoSpec, a cost-aware speculative decoding framework that optimizes draft token selection by accounting for expert activation memory traffic in Mixture-of-Experts (MoE) models. The work is significant because it identifies and addresses a specific bottleneck—expert scattering—that limits the efficiency of standard speculative decoding in modern MoE architectures like DeepSeek-V3 and Qwen3. By integrating lightweight expert prediction into the draft tree selection process, it achieves meaningful end-to-end speedups without modifying the target model, offering a practical optimization for the rapidly scaling MoE paradigm. While the contribution is solid and addresses a timely engineering challenge in LLM inference, it remains an optimization layer rather than a fundamental architectural or algorithmic breakthrough, placing it in the upper tier of applied ML research but below the threshold for field-defining significance.
Metacognition is a foundational component of intelligence critical to effective learning, problem solving, decision-making, communication, and more. In recent years, it has become increasingly recognized as a cornerstone of capable, transparent AI systems. Yet while LLMs have made significant progress across diverse real-world tasks, it is not yet clear when, how, or to what extent they can exhibit or be endowed with effective metacognitive abilities, nor how such abilities can be adapted to advance the fundamental capabilities, reliability, and intelligence of AI systems. This paper bridges this gap by presenting the first comprehensive overview of the current state of knowledge on metacognition for LLMs. We analyze and taxonomize the landscape of this emerging field and summarize recent technical advancements, including methods and benchmarks to measure and evaluate LLMs' metacognitive abilities, techniques to elicit, improve, and apply metacognition in LLMs, and findings and implications of ongoing research. We also discuss applications, open questions and challenges, and promising directions for future work. Our aim is to provide a detailed and up-to-date review of this topic and stimulate meaningful research and discussion. An organized list of papers can be found at https://github.com/yale-nlp/LLM-Metacognition.
Primary: Yale University
All Institutions: Yale University
This paper is poised to have a substantial broader impact on the field of machine learning, particularly in NLP and AI safety. By providing the first comprehensive overview and taxonomy of metacognition in LLMs, it serves as a critical unifying force for a fragmented and rapidly emerging research area. Its detailed analysis of foundations, progress, and opportunities will guide future research, helping to clarify definitions, standardize measurement, and identify promising directions. This work is essential for developing more capable, reliable, transparent, and trustworthy AI systems, especially as LLMs are deployed in high-stakes applications. It highlights the role of metacognition in improving task performance, reducing hallucinations, enhancing human-AI collaboration, and increasing interpretability—all crucial aspects for the safe and beneficial development of AI. It will likely become a foundational reference for researchers and practitioners alike, stimulating a more coherent and impactful research agenda in LLM metacognition. The main contribution of this paper is presenting the first comprehensive overview and taxonomy of metacognition in LLMs. The paper meticulously synthesizes a nascent and fragmented field, defining key concepts, categorizing existing measurement techniques and empirical findings, and outlining future research directions, thereby establishing a foundational reference for advancing the development of more capable and reliable AI systems.
The methodology of this paper, as a comprehensive survey, is exemplary in its approach to literature review, synthesis, and organization. The authors begin by establishing a clear foundation of metacognition from cognitive psychology, which is crucial for grounding the subsequent discussion on LLMs. They then critically analyze how the concept is currently being interpreted and applied in the LLM literature, explicitly highlighting the existing lack of consensus and varied instantiations. The paper's structure is highly logical, systematically categorizing existing work into distinct areas: defining metacognition, its importance, methods for measurement and benchmarking, current empirical findings, techniques for eliciting and improving metacognition, applications, open questions, and future directions. This systematic approach provides a coherent framework for understanding a rapidly evolving and fragmented field. The inclusion of a GitHub repository with an organized list of papers further enhances the utility and comprehensiveness of this review.
As a survey paper, this work does not present its own experimental results. Instead, it provides a rigorous and critical evaluation of the experimental findings from the extensive body of literature it surveys. The authors meticulously synthesize diverse empirical observations, highlighting both consistent trends and conflicting results regarding LLMs' metacognitive abilities. For instance, they discuss the mixed findings on metacognitive calibration (e.g., overconfidence vs. underconfidence depending on task), the limitations and dependencies of various measurement techniques (e.g., SDT-based approaches, confidence elicitation methods), and the complex interplay between model size, post-training strategies (like RLHF), and metacognitive performance. This critical synthesis is a significant strength, offering a balanced and nuanced perspective on the current state of LLM metacognition, rather than simply listing results.
The reproducibility of this survey is high, primarily due to its clear and well-structured taxonomy, comprehensive literature review, and the explicit provision of a supplementary resource. The paper's detailed categorization of metacognitive concepts, measurement techniques, empirical findings, and proposed methods makes it straightforward for other researchers to understand the intellectual landscape and trace the arguments. Crucially, the mention of `https://github.com/yale-nlp/LLM-Metacognition` as an "organized list of papers" acts as a dynamic and accessible resource, allowing researchers to easily access the cited works and build upon the review. This commitment to providing an external, curated bibliography significantly enhances the paper's utility and reproducibility as a foundational reference.
The paper itself identifies several key limitations within the nascent field of LLM metacognition, which it aims to address through its comprehensive review. These include: 1. **Lack of Unified Definition**: A significant challenge is the absence of a clear, universally agreed-upon definition of metacognition in the context of LLMs, leading to diverse and sometimes inconsistent operationalizations. 2. **Measurement Challenges**: Current methods for measuring LLM metacognition are often task-specific, sensitive to confidence elicitation techniques, and may lack robust empirical justification, making consistent evaluation difficult. 3. **Superficiality vs. Genuine Metacognition**: There's an ongoing debate about whether LLMs genuinely exhibit metacognitive processes or merely simulate them through pattern replication from training data. 4. **Impact of Post-training**: Post-training procedures like RLHF can significantly alter models' internal uncertainty, complicating the assessment of their true metacognitive capabilities. 5. **Benchmark Gaps**: Higher-order cognitive skills like metacognition are currently under-represented in standard LLM benchmarks. The paper also briefly touches upon potential risks of enhanced metacognitive abilities, such as models strategically modifying outputs to evade oversight.
This paper is poised to have a substantial broader impact on the field of machine learning, particularly in NLP and AI safety. By providing the first comprehensive overview and taxonomy of metacognition in LLMs, it serves as a critical unifying force for a fragmented and rapidly emerging research area. Its detailed analysis of foundations, progress, and opportunities will guide future research, helping to clarify definitions, standardize measurement, and identify promising directions. This work is essential for developing more capable, reliable, transparent, and trustworthy AI systems, especially as LLMs are deployed in high-stakes applications. It highlights the role of metacognition in improving task performance, reducing hallucinations, enhancing human-AI collaboration, and increasing interpretability—all crucial aspects for the safe and beneficial development of AI. It will likely become a foundational reference for researchers and practitioners alike, stimulating a more coherent and impactful research agenda in LLM metacognition. The main contribution of this paper is presenting the first comprehensive overview and taxonomy of metacognition in LLMs. The paper meticulously synthesizes a nascent and fragmented field, defining key concepts, categorizing existing measurement techniques and empirical findings, and outlining future research directions, thereby establishing a foundational reference for advancing the development of more capable and reliable AI systems.
Retrieval-augmented generation evaluation checks whether model claims are factually grounded in retrieved documents. It does not check whether retrieved evidence is attributed to the correct entity. A clinical RAG response can pass every automated check (zero hallucinations, near-perfect faithfulness, real citations) while presenting drug Y's clinical evidence as evidence about queried drug X. We term this deceptive grounding (DG): a failure invisible to faithfulness, hallucination, and citation checks because every claim is sourced from a real document, about the wrong entity. Using a controlled factorial benchmark across 13 models, we find DG rates spanning 8-87% at peak adversarial conditions. Medical and biomedical fine-tuned models reach up to 86.7%; domain specialization amplifies the failure rather than mitigating it. A controlled ablation identifies the mechanism: removing entity-specific clinical evidence from retrieved documents eliminates entity-attribution failure entirely, shifting all failures to confabulation. The two failure modes respond to the same trigger, taking different paths. Production measurement across 740 drug-disease pairs finds 7.8% overall DG in a deployed RAG system, rising to 13.6% for recently approved drugs. Entity-attribution verification (checking that cited evidence applies to the queried entity) detects DG at 97.0% precision and 98.7% DG recall (IPW-adjusted human gold standard); no existing framework implements it.
Primary: Lunit
All Institutions: Lunit, Trillion Labs, SK Biopharmaceuticals, Kakao Healthcare, AIGEN Sciences, D-Circle, Rebellions, Standigm, KAIST, Seoul National University, NHIS Ilsan Hospital, Ewha Womans University Seoul Hospital, Keimyung University Dongsan Medical Center, Konyang University Hospital, Korea University Research & Business Foundation, Kyung Hee University Hospital at Gangdong, Kyung Hee University Medical Center, Pusan National University Yangsan University, Yongin Severance Hospital
[One sentence main contribution]. This paper identifies and quantifies "Deceptive Grounding," a critical failure mode in RAG systems where models faithfully attribute evidence from the wrong entity, and proposes Entity-Attribution Verification as a necessary new evaluation standard. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The technical contribution is significant because it moves beyond surface-level faithfulness to inspect the structural integrity of entity attribution in generated text. The methodology is rigorous, combining large-scale behavioral benchmarking with mechanistic interpretability to explain *why* the failure occurs. The empirical findings are surprising and counter-intuitive, particularly the amplification of DG in domain-specialized models, which has major implications for model development strategies. The introduction of EAV fills a critical gap in the evaluation landscape, providing a tool that can detect errors previously considered undetectable. This work is essential reading for anyone building or evaluating RAG systems, particularly in regulated industries.
The paper introduces a rigorous and novel conceptual framework for evaluating Retrieval-Augmented Generation (RAG) systems, specifically identifying "Deceptive Grounding" (DG). Unlike traditional hallucination detection which looks for factual errors or lack of grounding, DG identifies cases where the model faithfully retrieves evidence for Entity Y but incorrectly attributes it to Query Entity X. The methodology involves a controlled 2D factorial benchmark manipulating retrieval completeness for the queried entity (X) and the alternate entity (Y). It employs advanced mechanistic interpretability techniques, including activation patching and class probing, to isolate the internal representations responsible for this failure mode. The introduction of Entity-Attribution Verification (EAV) as a new evaluation metric is a significant methodological contribution, addressing a blind spot in current LLM evaluation suites.
The experimental design is comprehensive and statistically robust. The authors test 13 models across 264 drug-disease triples under 15 retrieval conditions, providing a granular view of model susceptibility. The results are striking: DG rates range from 8% to 86.7%, with domain-specific fine-tuned models being particularly vulnerable. The ablation studies effectively demonstrate the two-stage mechanism (context priming followed by content-dependent attribution). The production measurement across 740 real-world queries provides crucial external validity, showing that while rates are lower in naturalistic settings (7.8%), they remain clinically significant, especially for new drugs. The use of human gold standard adjudication and IPW-adjusted statistics adds credibility to the findings.
The paper provides extensive details on the benchmark construction, including specific prompt templates for document generation and judgment. The tool schemas and execution protocols are described in the appendix. The compute resources and model versions are clearly stated. While the full codebase is not explicitly linked in the text provided, the detailed description of the synthetic document generation process and the evaluation criteria makes replication feasible. The use of a specific judge model (Kimi-K2.5) for evaluation is noted, and inter-rater reliability with human annotators is provided, ensuring the evaluation metric itself is reproducible and valid.
The primary limitation is the adversarial nature of the controlled benchmark, which may overestimate DG rates compared to naturalistic retrieval scenarios (as evidenced by the 5.5x gap to production rates). The evaluation relies heavily on a single LLM judge (Kimi-K2.5) for the initial large-scale scoring, although human validation is provided for key conclusions. The sample size for human adjudication (n=88) is relatively small, leading to wide confidence intervals for some metrics. Furthermore, the study focuses exclusively on the clinical/pharmacological domain, which, while critical, limits the immediate generalizability of the specific DG rates to other domains, though the mechanism is likely domain-agnostic.
This paper has profound implications for the safety and reliability of LLMs in high-stakes domains like healthcare, law, and finance. By identifying a failure mode that is invisible to current automated evaluation frameworks, it forces a re-evaluation of how we define and measure "groundedness." The proposed Entity-Attribution Verification (EAV) could become a standard component of RAG evaluation pipelines. The finding that domain specialization can exacerbate this failure challenges the assumption that fine-tuning always improves reliability, suggesting that architectural or training interventions specifically targeting entity identity are necessary. This work protects against a class of errors that could lead to severe real-world harm, such as incorrect medical advice. [One sentence main contribution]. This paper identifies and quantifies "Deceptive Grounding," a critical failure mode in RAG systems where models faithfully attribute evidence from the wrong entity, and proposes Entity-Attribution Verification as a necessary new evaluation standard. [Comprehensive analysis of the technical contribution, methodology, and significance to the field]. The technical contribution is significant because it moves beyond surface-level faithfulness to inspect the structural integrity of entity attribution in generated text. The methodology is rigorous, combining large-scale behavioral benchmarking with mechanistic interpretability to explain *why* the failure occurs. The empirical findings are surprising and counter-intuitive, particularly the amplification of DG in domain-specialized models, which has major implications for model development strategies. The introduction of EAV fills a critical gap in the evaluation landscape, providing a tool that can detect errors previously considered undetectable. This work is essential reading for anyone building or evaluating RAG systems, particularly in regulated industries.
LLM-based ASR adapted to regulated domains such as banking is bottlenecked by privacy: real speech is costly and legally constrained to collect, making synthetic text-to-speech (TTS) an attractive substitute. Yet synthetic speech stays acoustically mismatched with real recordings, and work on this gap has stayed within supervised fine-tuning (SFT). We instead turn to reinforcement learning, and show that Group Relative Policy Optimization (GRPO) extracts far more from the same synthetic speech than SFT. Synthetic-only adaptation of the model with GRPO, a critic-free method rewarding low-WER hypotheses, reduces WER by 40\% relative to SFT (36.71\%$\to$22.09\%), and an SFT-then-GRPO combination pushes this further to 45\%. We trace the gain to behavior rather than representation: GRPO reduces insertion errors by improving stopping calibration and speech-to-text alignment by better anchoring attention to audio, leaving early-layer representations intact. When synthetic speech is the main resource, reinforcement learning should be preferred over supervised fine-tuning.
Primary: Idiap Research Institute
All Institutions: Idiap Research Institute, EPFL, Uniphore, Brno University of Technology
This paper presents a compelling application of GRPO to ASR, demonstrating that reinforcement learning can effectively bridge the synthetic-to-real speech gap by improving stopping calibration and reducing hallucinations, offering a practical solution for privacy-constrained domain adaptation.
The paper proposes applying Group Relative Policy Optimization (GRPO), a reinforcement learning method recently popularized in LLM alignment (e.g., DeepSeek-R1), to the domain of Automatic Speech Recognition (ASR) using synthetic speech data. The core methodological innovation is the application of a critic-free RL approach to optimize for Word Error Rate (WER) directly, bypassing the limitations of Supervised Fine-Tuning (SFT) which suffers from token-level cross-entropy losses when acoustic signals are mismatched (synthetic vs. real). The authors adapt the standard GRPO formulation by defining specific reward functions (WER, CER, length penalties) and demonstrate its efficacy on an LLM-based ASR backbone (WavLM + Llama-3.2). The analysis of *why* it works—specifically through improved stopping calibration and attention anchoring rather than representation rewriting—is a significant theoretical contribution to understanding RL in multimodal settings.
The experimental setup is rigorous and well-controlled. The authors use the DefinedAI banking corpus, a realistic high-stakes domain, and compare SFT vs. GRPO under two regimes: full real data access and privacy-constrained synthetic-only access. Key findings include: 1) GRPO significantly outperforms SFT on synthetic-only data (WER 22.09% vs 36.71%). 2) SFT+GRPO provides incremental gains on real data. 3) The benefit is driven by a reduction in insertion errors, attributed to better stopping calibration. 4) Ablations on reward functions and synthetic data selection strategies are thorough. The results are statistically significant and practically relevant for regulated industries. The analysis linking behavioral changes (attention entropy, confidence) to performance gains adds depth beyond simple metric reporting.
The paper provides detailed hyperparameters (LoRA rank, learning rates, KL coefficients, group size). The backbone architecture (WavLM-Large, Llama-3.2-1B) and preprocessing steps (RIR convolution, TTS pipeline) are described. However, the code repository is withheld for blind review, and specific details on the TTS model (Qwen3-TTS) and voice cloning setup are referenced but not fully open-sourced in the text. The use of proprietary or less-common TTS models might pose slight reproducibility hurdles for exact replication, though the methodological framework is clear.
The study is limited to a single domain (banking) and a single LLM-based ASR backbone. While the findings are promising, generalizability to other domains (e.g., medical, legal) or different ASR architectures (e.g., Conformer-LLM hybrids) is not tested. The reliance on synthetic speech introduces potential biases inherent to the TTS system used (Qwen3-TTS), and the paper does not extensively analyze the impact of TTS quality variations on GRPO's robustness. Additionally, the computational cost of GRPO (requiring multiple rollouts per step) is higher than SFT, which is a practical limitation not fully quantified in terms of training time/cost relative to the performance gain.
This work has significant implications for deploying speech AI in privacy-sensitive sectors where data collection is legally restricted (GDPR, HIPAA). By demonstrating that synthetic speech can be effectively leveraged via RL, it offers a viable path for domain adaptation without compromising user privacy. It also bridges the gap between NLP alignment techniques (RLHF/GRPO) and speech processing, potentially accelerating the adoption of RL methods in multimodal ASR research. This paper presents a compelling application of GRPO to ASR, demonstrating that reinforcement learning can effectively bridge the synthetic-to-real speech gap by improving stopping calibration and reducing hallucinations, offering a practical solution for privacy-constrained domain adaptation.
A model should refuse two different things: answers it would get wrong, and questions it should not answer at all, such as unanswerable ones or ones resting on a false premise. The usual recipe thresholds a single confidence score, which cannot tell these apart. Across five instruction-tuned models from three families (2B to 14B), we find they are separate axes. Ordinary answer-confidence tracks whether an answer is right but is nearly blind to whether the question is answerable; a linear probe on hidden states does the reverse. The blind spot does not shrink with scale. It is worst on naturally occurring false-premise questions (CREPE). There, answer-confidence, P(IK), P(True), and even asking the model outright whether a premise is false all stay near chance, while a hidden-state probe reaches 0.69 to 0.77 AUROC: the model represents a problem it will not report. This turns out to be fixable. Instructing a model to check premises backfires, because it then disputes sound and false premises alike (57% false challenges), unable to tell them apart; routing the same instruction with the probe roughly triples challenge precision. We turn the two axes into a calibrated policy that answers only when an answerability score and a correctness score each clear a separately certifies behave differently: the unanswerable-answer rate is controllable at every scale, while the wrong-answer rate is capped by model accuracy, so the guarantee tightens as threshold policy certifies both budgets at 0.75 coverage of correct answers, against 0.31 for a single threshold; at 14B it is the only policy that certifies at all.
Primary: City St George's, University of London
All Institutions: City St George's, University of London
The paper makes a significant contribution by empirically demonstrating and formally exploiting the decoupling of answer correctness and question answerability in LLMs, introducing a factorized abstention policy with rigorous statistical guarantees that outperforms standard single-threshold methods.
The paper proposes a fundamental decomposition of LLM abstention into two orthogonal axes: answer correctness (execution) and question answerability (admissibility). The methodology involves training linear probes on hidden states to detect answerability and comparing them against standard output-confidence metrics (mean token log-probability, entropy). It introduces a factorized abstention policy that requires both high answerability and high correctness scores to emit an answer, calibrated using conformal prediction techniques (Learn-Then-Test framework) to provide rigorous statistical guarantees on error rates. The approach is theoretically grounded in neuro-symbolic reasoning, treating abstention as a three-class selective acceptance problem.
The evaluation is extensive and rigorous, spanning five instruction-tuned models (2B to 14B) from three families (Gemma, Qwen, Llama). It utilizes two distinct benchmarks: SelfAware (constructed, controlled) and CREPE (naturally occurring false presuppositions). The experiments demonstrate a consistent "crossed geometry" where output confidence separates correct from wrong answers but fails to distinguish wrong from unanswerable, while hidden-state probes do the reverse. The paper provides certified risk bounds, showing that the factorized policy significantly outperforms single-threshold baselines, especially at larger scales (8B, 14B), where single-threshold policies fail to certify any coverage. The inclusion of a replication audit over 100 resplits adds substantial empirical weight.
The authors commit to releasing code and score artifacts, with a verification script run in continuous integration. The methodology is clearly described, including specific prompts, model quantization details, and calibration splits. The use of exact binomial bounds and clear split definitions enhances reproducibility. The paper explicitly states that all numbers are regenerated from committed artifacts.
The study is limited to models up to 14B, leaving open whether the pattern holds for frontier models (e.g., 70B+). The SelfAware benchmark has surface confounds (lexical differences between answerable/unanswerable), though the CREPE benchmark mitigates this. The correctness evaluation relies on alias matching, which may be imperfect for long-form answers. The certification guarantees are marginal and depend on exchangeability assumptions. The "probe-routed premise checking" is a pilot study with small sample sizes.
This work has significant implications for the safety and reliability of LLMs. By decoupling the ability to answer from the willingness to answer, it provides a pathway to reduce hallucinations and inappropriate responses in high-stakes applications. The certified abstention framework offers a new standard for evaluating model reliability, moving beyond simple accuracy metrics to risk-controlled deployment. It challenges the prevailing reliance on output confidence for uncertainty estimation. The paper makes a significant contribution by empirically demonstrating and formally exploiting the decoupling of answer correctness and question answerability in LLMs, introducing a factorized abstention policy with rigorous statistical guarantees that outperforms standard single-threshold methods.
Robotic manipulation is inherently multi-frame: local actions may be simple in an end-effector frame, while transport, upright-object handling, and whole-body coordination are better represented in a base-aligned frame. However, modern diffusion-based visuomotor policies typically commit to a single predefined action frame, forcing one denoiser to model action distributions that are often unnecessarily complex in that frame. We propose Mixture of Frames Policy (MoF), a diffusion policy that performs synchronized action denoising across multiple coordinate frames. MoF maintains a single canonical diffusion state, re-expresses it in several task-relevant frames, applies frame-specialized denoisers, and fuses their noise predictions back in the canonical frame. To make this possible for intermediate noisy diffusion states, we introduce a column-based 6D rotation representation within an SE(3) action parameterization that supports exact, differentiable frame transformations without requiring noisy rotations to lie on the SO(3) manifold. Across nine simulated bimanual manipulation tasks, we show that the best action frame is task-dependent and that MoF improves over oracle frame selection and standard Mixture-of-Experts (MoE) baselines. We further evaluate MoF on two real-world bimanual mobile manipulation tasks, demonstrating that it outperforms all constituent single-frame baselines. Project homepage: https://mofpo.github.io
Primary: Stanford University
All Institutions: Stanford University, Apple, Toyota Research Institute
The paper presents a highly innovative and technically sound approach to improving diffusion-based visuomotor policies by leveraging multi-frame reasoning. The introduction of a column-based 6D rotation representation for exact noisy state transformation is a significant methodological advance. Comprehensive experiments in simulation and on real-world robots demonstrate substantial improvements over strong baselines, including oracle frame selection. The work is well-motivated, rigorously evaluated, and addresses a fundamental limitation in current robotic policy architectures. It is a strong candidate for top-tier publication and will likely influence future research in robotic policy learning.
The paper proposes a novel architecture for diffusion-based visuomotor policies called Mixture of Frames (MoF). The core insight is that robotic manipulation actions are naturally represented in different coordinate frames at different stages of a task (e.g., end-effector for grasping, base for transport). Standard diffusion policies fix a single action frame, forcing the denoiser to model complex, frame-dependent distributions. MoF maintains a single canonical diffusion state but applies frame-specialized denoisers in their respective frames, fusing the noise predictions back into the canonical frame. A critical technical contribution is the introduction of a column-based 6D rotation representation for SE(3) actions. This allows for exact, differentiable frame transformations of *noisy* intermediate states without requiring projection onto the SO(3) manifold, which is non-linear and lossy. This mathematical insight is elegant and solves a significant implementation hurdle for multi-frame reasoning in diffusion policies. The method is generalizable to any diffusion policy and can be implemented as a drop-in replacement or extension.
The evaluation is comprehensive and rigorous. The authors first establish the motivation by showing that action frame choice has a substantial, task-dependent impact on performance in bimanual mobile manipulation, with a 15% gap between the best and worst single frames. They then demonstrate that MoF outperforms oracle frame selection (which knows the best frame per task) and standard Mixture-of-Experts (MoE) baselines that do not operate in multiple action frames. The ablation studies are particularly strong, showing that the proposed column-based representation is essential (orthogonalization leads to collapse in ensemble variants) and that the router effectively switches between frames based on task phase. Real-world experiments on two complex bimanual mobile manipulation tasks (Pouring, Serving) further validate the approach, showing significant improvements over single-frame baselines. The use of simulated benchmarks (BiGym, DexMimicGen) and real-world validation provides a robust evidence base.
The paper provides detailed descriptions of the method, including the specific rotation representation and the fusion mechanism. The project homepage link suggests code availability. The experimental setup is well-described, including dataset sizes, training epochs, and evaluation metrics. The ablation studies provide clear evidence of component contributions. The reliance on standard simulators (BiGym) and real-world hardware (HoMMI) enhances reproducibility.
The method relies on a designer-specified set of candidate frames. While the router learns to weight them, the set must be predefined. The paper acknowledges this and suggests future work on learning or discovering frames. Additionally, the router is supervised via denoising loss, which may not perfectly align with policy performance, though the results suggest it works well in practice. The current experiments focus on smaller diffusion policies; scaling to large VLA models is noted as future work.
This work has significant implications for robotic manipulation, particularly in complex, multi-stage tasks where different coordinate systems are advantageous. By improving the robustness and performance of visuomotor policies, it contributes to the broader goal of autonomous robots capable of operating in unstructured environments. The technical contribution regarding SE(3) transformations is also valuable for the broader machine learning community working with geometric deep learning and diffusion models. The paper presents a highly innovative and technically sound approach to improving diffusion-based visuomotor policies by leveraging multi-frame reasoning. The introduction of a column-based 6D rotation representation for exact noisy state transformation is a significant methodological advance. Comprehensive experiments in simulation and on real-world robots demonstrate substantial improvements over strong baselines, including oracle frame selection. The work is well-motivated, rigorously evaluated, and addresses a fundamental limitation in current robotic policy architectures. It is a strong candidate for top-tier publication and will likely influence future research in robotic policy learning.
Dexterous grasp generation across robot hands is challenging because hands differ in kinematic topology, actuation dimensions, and native command spaces. We introduce GraspGraphNet, a topology-aware grasp generation framework that represents each hand as a URDF-derived kinematic graph and directly generates executable palm poses and joint configurations. GraspGraphNet combines hierarchical object surface encoding, differentiable forward kinematics, and dynamic world-edge message passing to model evolving robot-object interactions. It applies conditional flow matching directly in executable palm-pose and joint-state space, avoiding post-processing optimization, inverse kinematics, and retargeting. Using a shared model trained on Barrett Hand, Allegro Hand, and Shadow Hand, GraspGraphNet achieves an average success rate of 83.48% with 40ms inference time per grasp on a 40-object benchmark. Without retraining, the same model achieves 72.70% success on controlled finger-removal variants, demonstrating robustness to hand-topology variations. These results suggest that graph-structured hand representations can effectively support dexterous grasp generation across robot hands with different kinematic structures. Project: https://lysees.github.io/graspgraphnet-page
Primary: Korea Advanced Institute of Science and Technology (KAIST)
All Institutions: Korea Advanced Institute of Science and Technology (KAIST)
GraspGraphNet presents a significant technical contribution to multi-embodiment robotic manipulation by introducing a topology-aware, graph-based flow matching framework that achieves state-of-the-art performance and efficiency in generating executable dexterous grasps across diverse robot hands.
The paper proposes GraspGraphNet, a novel framework for multi-embodiment dexterous grasp generation. The core innovation lies in representing robot hands as URDF-derived kinematic graphs and object surfaces as hierarchical point clouds. By integrating differentiable forward kinematics into a graph neural network (GNN) architecture, the model respects the topological constraints of each hand. Crucially, it employs dynamic world-edge message passing to update robot-object interaction features at every step of a conditional flow matching process. This allows the model to generate executable palm poses and joint configurations directly, bypassing the need for inverse kinematics or post-processing optimization typically required by contact-based methods. The use of flow matching for continuous state refinement is a sophisticated choice that addresses the multimodality of grasp distributions better than direct regression or standard diffusion models.
The experimental evaluation is comprehensive and rigorous. The authors evaluate on three distinct dexterous hands (Barrett, Allegro, Shadow) across 40 objects, providing a robust benchmark for multi-embodiment generalization. The results show state-of-the-art success rates (83.48% average) while significantly outperforming baselines in inference speed (40ms vs. 265ms+ for competitors). The finger-removal experiments are particularly compelling, demonstrating that the graph-based representation allows for zero-shot generalization to topologically modified hands (72.70% success), a feat previous methods struggle with. Real-world deployment on a Leap Hand further validates the practical utility of the approach. The ablation studies effectively isolate the contributions of dynamic world edges, cross-attention, and the flow-matching objective.
The paper provides sufficient implementation details, including network architecture dimensions, training hyperparameters (AdamW, learning rate, epochs), and dataset sources (CMapDataset). The use of standard simulation environments (Isaac Gym) and publicly available datasets enhances reproducibility. The inclusion of a project page suggests code availability, which is critical for replication in this domain.
The primary limitation is the reliance on URDF definitions, which requires accurate kinematic models for each hand. While this is standard in simulation, real-world deployment may require precise calibration or adaptation to handle kinematic drift or non-rigid deformations not captured in the URDF. Additionally, the model is evaluated on a specific set of hands; generalization to radically different morphologies (e.g., soft robotic hands or grippers with vastly different joint limits) is not demonstrated, though the authors acknowledge this as future work. The reliance on flow matching also introduces a small number of integration steps (K=3), which, while efficient, may occasionally produce less optimal grasps compared to methods with more extensive search or optimization phases, although the speed trade-off is favorable.
This work significantly advances the field of robotic manipulation by providing a unified, efficient, and robust method for dexterous grasp generation across diverse hardware. By eliminating the need for hand-specific retargeting and optimization, it lowers the barrier to deploying dexterous manipulation policies on new robotic platforms. The graph-based approach offers a principled way to handle structural variations, which could inspire similar topology-aware representations in other robotic control tasks. The efficiency gains (40ms inference) make real-time dexterous manipulation more feasible for practical applications. GraspGraphNet presents a significant technical contribution to multi-embodiment robotic manipulation by introducing a topology-aware, graph-based flow matching framework that achieves state-of-the-art performance and efficiency in generating executable dexterous grasps across diverse robot hands.
Bipedal robots are challenging to control because they operate close to instability, where small variations in foot-terrain contact can rapidly destabilize locomotion. On rigid terrain, bipedal robots mitigate this fragility by using well-established contact mechanics and control strategies. On flowable surfaces such as granular slopes, foot contact can induce large surface deformations and solid-fluid-like transitions, coupling terrain effects with robot dynamics, leading to underperformance or failure. This is partly due to the lack of reliable methods to represent the dynamics of flowable terrain, making it difficult to account for terrain effects in locomotion design. Here, we investigate how controlling terrain response can improve bipedal locomotion on granular slopes by studying the terradynamics of cleated feet, thin plates emanating from the foot soles. Systematic studies of a small-scale (1.4 kg) robophysical biped reveal that cleats with sparse and dense spacing lead to excessive terrain yielding and resistance, respectively, degrading performance and leading to failure. An intermediate cleat spacing distributes interaction forces to maintain substrate stresses near (or below) the yield threshold, enabling walking on granular slopes up to 30 degrees. Guided by these principles, we design a foot that actively adjusts cleat depth and accommodates both rigid and granular terrain. We also demonstrate that the principles of effective foot-terrain interaction translate to a larger (15 kg) autonomous biped. Our study presents an alternative to conventional body-centric robot control approaches, which regulate terrain-induced effects through body motion, by instead regulating terrain interactions through limb-centric approach.
Primary: Georgia Institute of Technology
All Institutions: Georgia Institute of Technology, Northeastern University
This paper presents a seminal contribution to robotic locomotion on deformable terrain, establishing a mechanistic framework for cleated-foot interaction that enables robust bipedal walking on steep granular slopes. By systematically characterizing terradynamics and validating findings across multiple scales, the authors provide a foundational understanding that bridges the gap between granular physics and legged robot design, offering a pathway to more capable autonomous systems in challenging natural environments.
The paper employs a rigorous robophysical approach, combining systematic experimental variation of foot morphology (cleat spacing and depth) with granular terradynamics analysis. The methodology is grounded in physical principles rather than purely data-driven learning, utilizing Particle Image Velocimetry (PIV) and dual-plate intrusion tests to elucidate the underlying mechanics of granular flow. The introduction of a limb-centric control paradigm, where terrain interaction is regulated through foot design rather than just body dynamics, represents a significant conceptual shift in legged locomotion research. The integration of these insights into an adaptive foot mechanism (retractable cleats) and validation on a large-scale autonomous biped (HECTOR) demonstrates a robust engineering methodology.
The experimental evaluation is comprehensive and multi-scaled. It begins with a small-scale constrained robot (BLUEY) to isolate variables, proceeds to granular intrusion tests for mechanistic understanding, and culminates in challenging field-relevant scenarios with a large-scale autonomous robot (HECTOR). The use of PIV to visualize particle flow provides unique, high-quality data linking foot geometry to substrate response. The results clearly demonstrate that intermediate cleat spacing optimizes performance by balancing traction and substrate disturbance, a finding that is both counter-intuitive and practically significant. The successful translation of these principles to a 15 kg robot is a strong validation of the findings.
The paper provides detailed descriptions of the robot kinematics, gait generation (ZMP-based), and experimental setups. The inclusion of pseudocode for the adaptive cleat control and specific parameters for the granular substrate (poppy seeds, volume fraction) enhances reproducibility. However, the reliance on specific robophysical platforms and the complexity of the granular terrain preparation (air-fluidization) may pose challenges for exact replication without access to similar infrastructure. The open-source nature of the HECTOR platform aids in this regard.
The study is limited to rectangular cleats and specific granular media (poppy seeds). The scaling laws derived may not directly apply to human-scale robots or different substrate types (e.g., cohesive soils) without further validation. The gait strategies tested are relatively simple (quasi-static for BLUEY, limited MPC exploration for HECTOR), leaving room for optimization. The authors acknowledge these limitations and suggest future work involving DEM simulations and more complex terrain.
This work has significant implications for the field of legged robotics, particularly for applications in unstructured environments such as search and rescue, planetary exploration, and agricultural robotics. By providing a mechanistic framework for locomotion on flowable terrain, it enables the design of more robust and versatile robotic systems. The shift from body-centric to limb-centric control strategies offers a new paradigm for handling terrain-robot coupling, potentially influencing future research in adaptive morphology and control. This paper presents a seminal contribution to robotic locomotion on deformable terrain, establishing a mechanistic framework for cleated-foot interaction that enables robust bipedal walking on steep granular slopes. By systematically characterizing terradynamics and validating findings across multiple scales, the authors provide a foundational understanding that bridges the gap between granular physics and legged robot design, offering a pathway to more capable autonomous systems in challenging natural environments.
Existing robotic perception is constrained by sensors that are either robot-mounted or permanently fixed in the environment, locking perception to a limited set of viewpoints. Yet as robots perform increasingly diverse tasks, the most informative viewpoint shifts from one task to the next-often somewhere onboard sensor and static infrastructure can not readily satisfy. To address this gap, we propose SensorPerch, a novel realization of active perception that decouples sensing from both the robot embodiment and the environment by treating sensors as independent physical entities that the robot can autonomously detach and re-attach within the environment. SensorPerch presents one realization of this paradigm: a lightweight, wireless, reconfigurable sensor platform that can perch on diverse surfaces, paired with a viewpoint-selection framework that determines task-optimal sensor placements. Together, these enable robots to construct task-relevant viewpoints on demand, independent of the robot's current position and available fixed infrastructure. We demonstrate the paradigm on two task classes: (i) object-coupled perception, where SensorPerch enables persistent object-state detection beyond the robot's current position, achieving successful event detection even when the robot is not nearby; and (ii) policy-coupled perception, where SensorPerch allows robots to construct diverse, policy-specific viewpoints for various policies, achieving success rates comparable to those obtained using oracle viewpoints.
Primary: Cornell University
All Institutions: Cornell University
SensorPerch represents a significant step towards more versatile and autonomous robotic systems. By decoupling perception from fixed infrastructure and robot embodiment, it enables robots to adapt their sensing capabilities dynamically to task requirements, leading to several broader impacts: 1. **Enhanced Robot Autonomy:** Robots can operate more independently in diverse, unstructured environments without relying on pre-installed cameras or human intervention for viewpoint adjustment. 2. **Improved Task Performance:** The ability to construct task-optimal viewpoints on demand can lead to higher success rates in complex manipulation, monitoring, and interaction tasks, especially in scenarios requiring persistent observation or specific perspectives. 3. **New Active Perception Paradigm:** This work establishes a new paradigm for active perception, shifting from merely adjusting onboard sensors to physically reconfiguring the sensing infrastructure itself. This could inspire future research into modular, reconfigurable robotic components beyond just sensors. 4. **Applications in Diverse Fields:** Beyond household robotics, this technology could be transformative for applications in industrial inspection, disaster response (deploying sensors in hazardous areas), construction, and even scientific exploration, where dynamic, adaptive sensing is crucial. 5. **Reduced Infrastructure Costs:** By allowing a single robot to deploy and manage its own sensing network, it could reduce the need for expensive, permanently installed camera systems or multi-robot coordination for comprehensive coverage. SensorPerch introduces a novel paradigm for active perception by treating sensors as independent, reconfigurable physical entities that robots can autonomously deploy and manage. This comprehensive system, integrating robust hardware with an intelligent software framework for task-conditioned viewpoint selection, significantly enhances robotic perception capabilities, enabling persistent object-state detection and policy-aligned viewpoints in diverse real-world tasks, thereby pushing the boundaries of robot autonomy and adaptability.
The methodology proposed in SensorPerch is a comprehensive and well-integrated system that addresses a critical limitation in robotic perception. The core idea of decoupling sensing from the robot's embodiment and fixed infrastructure by treating sensors as autonomous, reconfigurable physical entities is genuinely novel. The system comprises both robust hardware and an intelligent software framework. The hardware design is modular, lightweight (201.4g), and self-contained, featuring an onboard Raspberry Pi 4B for computation, a 2-DoF motorized gimbal for viewpoint adjustment, a compact battery for independent operation (2.25 hours), and a vacuum-based attachment mechanism for diverse surfaces. The integration of a charging dock on the robot's mobile base for autonomous recharging is a practical and essential detail for long-term operation. The cost-effectiveness ($94) is also a significant advantage. The software framework, "Where to Perch," is equally sophisticated. It leverages a hybrid scene representation combining semantic SLAM for geometric reasoning and mountable surface identification, and a radiance-field model (NeRF) for novel view synthesis. This allows for efficient evaluation of candidate viewpoints in simulation without physical deployment. Viewpoint sampling is intelligently biased towards task-relevant regions using a VLM. The task-conditioned scoring function is a flexible abstraction, instantiated for two critical use cases: object-coupled perception (using VLM queries for object state detection) and policy-coupled perception (using DINOv2 embeddings to match a policy's training distribution). Finally, the system incorporates model-based pose estimation and grasp-and-place primitives for autonomous detachment and reattachment of the sensor platforms. The overall methodology presents a complete, closed-loop system for active, reconfigurable perception.
The experimental evaluation is thorough and compelling, covering platform capabilities, task performance, and system efficiency. 1. **Platform Evaluation:** Quantitative metrics for attachment accuracy (0.42 cm, <1 degree), battery life (2.25 hours), and thermal performance demonstrate the hardware's robustness and suitability for real-world, long-horizon tasks. 2. **Task Evaluation:** This is the strongest part of the evaluation. * **Object-coupled perception:** Tested across five diverse scenarios (pot boiling, human fall, fire hazard, cup dropped, water leakage), SensorPerch consistently achieves high event detection success rates, closely matching oracle performance and significantly outperforming baselines like "wrist camera only" or "random feasible placement." This validates its ability to provide persistent, task-relevant perception beyond the robot's immediate vicinity. * **Policy-coupled perception:** Evaluated on three manipulation tasks (peeling, cutting, pouring), SensorPerch achieves success rates (80-90%) comparable to oracle viewpoints, demonstrating its capability to construct policy-specific views that align with training data distributions. This is a crucial validation for enabling robust policy execution in varied settings. * **Baselines:** The comparison against a comprehensive set of baselines (w/o 2-DoF, random, oracle, wrist-only, fixed third-person) rigorously isolates the contributions of different components and the overall system. * **Multi-platform demonstration:** The real-world demonstration with three platforms simultaneously monitoring different aspects of a breakfast preparation task (pot, cutting, human's cup) is highly impressive and showcases the system's scalability and practical utility in complex, multi-task scenarios. 3. **System Evaluation:** The latency breakdown provides valuable insights into the computational overhead. While initial scene reconstruction is time-consuming (369s), it's a one-time cost. Viewpoint sampling, synthesis, and scoring are performed in seconds (0.91s to 7.95s), and the full detach-reattach loop takes a reasonable 12.53 seconds. Streaming latency (180ms) is suitable for real-time perception, even with multiple platforms. These metrics confirm the system's practical usability.
The paper provides detailed descriptions of the hardware components, including specific models (Raspberry Pi 4B, MG90S servos, RealSense D435i, 18650 battery) and their integration. The software stack outlines the use of semantic SLAM, radiance fields (NeRF), VLM, DINOv2, and ROS. A lightweight ROS package is mentioned, which is a good step towards reproducibility. However, the absence of a public code repository or specific details on the semantic SLAM and NeRF implementation (e.g., which specific open-source projects were used, training data for NeRF, VLM model used) makes full replication challenging without further information. The experimental setup is well-described, but the exact datasets for policy training and VLM queries would be needed.
The authors acknowledge several pertinent limitations: 1. **Surface Dependence:** The vacuum-based mounting mechanism relies on the availability of flat, non-porous, rigid surfaces. This limits deployment in environments with soft, porous, or highly irregular textures (e.g., carpets, fabric, rough stone). 2. **Vacuum Reliability:** The long-term reliability of vacuum seals under various environmental conditions (dust, temperature changes, vibrations) is a potential concern, although the paper demonstrates robust attachment in its experiments. 3. **Robot Motion and Occlusions:** The current framework does not explicitly model how the robot's own motion might create or resolve occlusions, or how it affects viewpoint quality over time. This could lead to suboptimal placements if the robot itself becomes an occluder. Additional limitations could include: 4. **Scalability to Many Sensors:** While a three-sensor demo is shown, managing and coordinating a much larger fleet of reconfigurable sensors could introduce new challenges in terms of communication, power management, and optimal placement strategies. 5. **Initial Scene Reconstruction Time:** The 369-second scene reconstruction time, while a one-time cost, might be prohibitive for highly dynamic environments or those requiring frequent re-mapping.
SensorPerch represents a significant step towards more versatile and autonomous robotic systems. By decoupling perception from fixed infrastructure and robot embodiment, it enables robots to adapt their sensing capabilities dynamically to task requirements, leading to several broader impacts: 1. **Enhanced Robot Autonomy:** Robots can operate more independently in diverse, unstructured environments without relying on pre-installed cameras or human intervention for viewpoint adjustment. 2. **Improved Task Performance:** The ability to construct task-optimal viewpoints on demand can lead to higher success rates in complex manipulation, monitoring, and interaction tasks, especially in scenarios requiring persistent observation or specific perspectives. 3. **New Active Perception Paradigm:** This work establishes a new paradigm for active perception, shifting from merely adjusting onboard sensors to physically reconfiguring the sensing infrastructure itself. This could inspire future research into modular, reconfigurable robotic components beyond just sensors. 4. **Applications in Diverse Fields:** Beyond household robotics, this technology could be transformative for applications in industrial inspection, disaster response (deploying sensors in hazardous areas), construction, and even scientific exploration, where dynamic, adaptive sensing is crucial. 5. **Reduced Infrastructure Costs:** By allowing a single robot to deploy and manage its own sensing network, it could reduce the need for expensive, permanently installed camera systems or multi-robot coordination for comprehensive coverage. SensorPerch introduces a novel paradigm for active perception by treating sensors as independent, reconfigurable physical entities that robots can autonomously deploy and manage. This comprehensive system, integrating robust hardware with an intelligent software framework for task-conditioned viewpoint selection, significantly enhances robotic perception capabilities, enabling persistent object-state detection and policy-aligned viewpoints in diverse real-world tasks, thereby pushing the boundaries of robot autonomy and adaptability.
Robust motion planning in dense traffic requires autonomous vehicles to interact in rare and safety-critical scenarios that are underrepresented in naturalistic driving data. Although adversarial training offers a feasible solution, existing methods often rely on external scenario generators, heuristic perturbations, or simulator-heavy rollouts, which makes them difficult to integrate with modern autoregressive planners. Here, we cast adversarially robust planner learning as a constrained min-max game and propose Adversarial World Modeling (AWM), a theoretically grounded multi-agent self-play fine-tuning framework. Since solving the exact game is intractable, AWM introduces a principled decoupled solver. In the inner minimization, the planner's predictive world model is converted into a role-conditioned adversary that learns sparse, scene-adaptive attack coalitions via counterfactual credit assignment. In the outer maximization, the ego planner optimizes a regret-aware robust best response against the frozen AWM, utilizing tail-risk weighting and reference-anchored trust regions to improve hard-case recovery while preserving nominal driving behavior. Experiments on the nuPlan and InterPlan benchmarks demonstrate that our method generates transferable adversarial interactions and yields a robust planner that achieves competitive closed-loop performance in both nominal and highly interactive long-tail scenarios. Theoretical analysis justifies the decoupled solver and the main optimization components.
Primary: unknown (information not provided in the paper text)
All Institutions: unknown (information not provided in the paper text)
This work has significant broader impact for autonomous driving and multi-agent reinforcement learning: * **Enhanced AV Safety**: By systematically exposing and mitigating vulnerabilities in rare, safety-critical scenarios, AWM contributes directly to improving the robustness and safety of autonomous vehicles in dense and interactive traffic. * **Robust ML for Robotics**: The framework provides a principled approach to adversarial training for complex, high-dimensional robotic systems, particularly those relying on autoregressive generative models. * **Multi-Agent Self-Play**: The novel counterfactual credit assignment and sparse coalition learning mechanisms advance the state-of-the-art in multi-agent self-play, offering solutions to long-standing credit assignment problems. * **Efficient Adversarial Testing**: AWM's ability to generate transferable adversarial interactions makes it a valuable tool for stress-testing and evaluating the robustness of various existing and future motion planners, potentially accelerating development and validation cycles. * **Integration with Modern Architectures**: By aligning with the autoregressive, tokenized paradigm, AWM offers a practical and scalable solution for robustifying modern generative planners, bridging a gap where traditional adversarial methods struggle. This paper introduces Adversarial World Modeling (AWM), a multi-agent self-play fine-tuning framework that repurposes a planner's predictive world model as a role-conditioned adversary to robustify autoregressive motion planning. The framework's principled decoupled solver, novel counterfactual credit assignment, and regret-aware constrained optimization provide a theoretically grounded and empirically validated approach to generate transferable adversarial interactions, significantly improving planner robustness in long-tail scenarios while preserving nominal driving performance.
The paper proposes Adversarial World Modeling (AWM), a multi-agent self-play fine-tuning framework for robust motion planning, framed as a constrained min-max game. The core innovation lies in repurposing the planner's own predictive world model as a role-conditioned adversary, which is a highly elegant and integrated solution compared to external scenario generators or heuristic perturbations. The methodology is theoretically grounded and addresses three key challenges: 1. **Multi-agent credit assignment**: AWM introduces a novel counterfactual credit assignment mechanism via role switching. By comparing factual active coalitions with leave-one-out and full background counterfactuals, it efficiently attributes scene-level utility changes to individual and synergistic adversarial behaviors. This is a significant improvement over naive reward sharing. 2. **Min-max optimization instability**: The paper proposes a principled decoupled solver. Stage A learns a frozen adversarial world model (AWM) that generates sparse, scene-adaptive attack coalitions. Stage B then optimizes the ego planner as a robust best response against this *frozen* AWM. This separation effectively mitigates the instability of simultaneous non-convex min-max updates. 3. **Nominal performance degradation**: The planner optimization in Stage B incorporates a regret-aware objective with tail-risk weighting (CVaR) to prioritize rare but critical failures, and crucially, reference-anchored trust regions with dual variables to preserve nominal driving behavior and enforce safety constraints. This ensures robustness is gained without sacrificing general driving competence. The detailed mechanisms for adversarial world modeling, including role-conditioned generation, sparse coalition learning (via proposal probing, primary selection, conditional pairing, and scene-adaptive calibration), are well-articulated and designed to generate plausible yet challenging multi-agent interactions. The use of a shared autoregressive tokenized rollout structure for both planning and adversarial generation ensures seamless integration. The theoretical analysis, mentioned in the abstract and contributions, provides justification for the decoupled solver and optimization components, adding significant rigor.
The experimental evaluation is comprehensive and rigorous, addressing three key questions: 1. **Closed-loop planning performance**: AWM-Planner is evaluated on the nuPlan and InterPlan benchmarks, including a challenging InterPlan-LongTail subset. It demonstrates competitive performance, significantly improving on interaction-heavy long-tail scenarios (e.g., +5.81 on InterPlan-LongTail) while maintaining or slightly improving nominal driving behavior on nuPlan (e.g., +2.60 on Test14-hard NR, -0.01 on Test14-random NR). This validates the method's ability to improve robustness without sacrificing general performance. 2. **Transferability of AWM as an adversarial simulator**: A crucial experiment deploys the learned AWM as a non-ego sim-agent traffic model for *seven different planners* (including rule-based, optimization-based, diffusion-based, and transformer-based). AWM consistently reduces the closed-loop score for all planners, demonstrating that the learned adversarial behaviors generalize beyond the planner it was trained against. This highlights AWM's potential as a valuable tool for stress-testing and robustifying various planning architectures. 3. **Component analysis (ablations)**: Extensive ablations are performed on both the AWM (Stage A) and planner (Stage B) components. * **AWM ablations**: Show that role-conditioned counterfactual credit assignment is superior to global reward broadcast, yielding stronger lower-tail degradation without sacrificing realism. The adaptive calibrated coalition learning is shown to provide sufficient adversarial stress while maintaining realism, outperforming forced K=2 attacks or simpler single-candidate approaches. * **Planner adaptation ablations**: Demonstrate the superiority of the decoupled self-play pipeline over planner-only refinement, simultaneous alternating, or alternating warmup, in terms of adversarial Tail-CVaR, closed-loop scores, and stability (lower GPU-hours, faster plateau, more stable seeds). Ablations on the Stage B objective confirm the importance of regret-aware CVaR for targeting rare severe risks and normal retention for preserving nominal performance. The use of diverse metrics (R-Score, progress, collision, TTC, drivable-area, speed-limit, driving-direction, Tail-CVaR) provides a holistic view of performance. Case studies visually illustrate the planner's improved recovery in adversarial scenarios and the nature of AWM-generated attacks.
The paper provides a high level of detail in its methodology section, including mathematical formulations for the game, decoupled solver, credit assignment, and planner objectives. It explicitly mentions appendices for "implementation details," "theoretical analysis," "architectural details," "probe signals and coverage metrics," "details for $f_pri$ and $f_pair$," "full AWM training objective," "analytical derivations and loss formulations," and "how the learned AWM is instantiated as a closed-loop sim-agent evaluator." The algorithm is summarized in Algorithm 1 in the appendix. This comprehensive documentation suggests a strong commitment to reproducibility.
The authors acknowledge several limitations: 1. **Fidelity of the tokenized world model**: AWM's effectiveness is inherently tied to the fidelity of the underlying tokenized world model. If the world model cannot accurately represent complex multi-agent dynamics, the generated adversarial scenarios may be unrealistic or ineffective. 2. **Exhaustive scenario coverage**: While AWM generates scene-adaptive attacks, it does not guarantee exhaustive coverage of all possible safety-critical scenarios. This is a common challenge in adversarial training for complex environments. 3. **Formal closed-loop safety guarantees**: The framework does not provide formal, provable safety guarantees, which remains a grand challenge for learning-based autonomous driving systems. 4. **Computational cost**: Although the decoupled solver improves stability and efficiency compared to simultaneous training, training large autoregressive models for both the world model and planner can still be computationally intensive.
This work has significant broader impact for autonomous driving and multi-agent reinforcement learning: * **Enhanced AV Safety**: By systematically exposing and mitigating vulnerabilities in rare, safety-critical scenarios, AWM contributes directly to improving the robustness and safety of autonomous vehicles in dense and interactive traffic. * **Robust ML for Robotics**: The framework provides a principled approach to adversarial training for complex, high-dimensional robotic systems, particularly those relying on autoregressive generative models. * **Multi-Agent Self-Play**: The novel counterfactual credit assignment and sparse coalition learning mechanisms advance the state-of-the-art in multi-agent self-play, offering solutions to long-standing credit assignment problems. * **Efficient Adversarial Testing**: AWM's ability to generate transferable adversarial interactions makes it a valuable tool for stress-testing and evaluating the robustness of various existing and future motion planners, potentially accelerating development and validation cycles. * **Integration with Modern Architectures**: By aligning with the autoregressive, tokenized paradigm, AWM offers a practical and scalable solution for robustifying modern generative planners, bridging a gap where traditional adversarial methods struggle. This paper introduces Adversarial World Modeling (AWM), a multi-agent self-play fine-tuning framework that repurposes a planner's predictive world model as a role-conditioned adversary to robustify autoregressive motion planning. The framework's principled decoupled solver, novel counterfactual credit assignment, and regret-aware constrained optimization provide a theoretically grounded and empirically validated approach to generate transferable adversarial interactions, significantly improving planner robustness in long-tail scenarios while preserving nominal driving performance.
The action space poses a major challenge in robot learning, since it is often high-dimensional, can span long time horizons, and frequently admits multi-modal optimal solutions. A good choice of action representation and loss function can help to address these concerns, but there are often trade offs. We propose Action Map Policy (AMP), which casts 3D closed-loop manipulation policy learning as a classification problem in image space. While classification has been an effective formulation in generative language models, applying it to robot action learning is difficult because naively discretizing high-dimensional continuous actions explodes the token vocabulary. Our key idea is to project 3D actions onto the camera image planes and treat each pixel location as a discrete class, thus controlling dimensionality while retaining multi-modality. This method supports millimeter-level precision for high-dimensional actions without requiring a prohibitively large vocabulary, while preserving fine-grained pixel-wise visual signals. Furthermore, it can predict the entire action chunk in a single forward pass, avoiding complex noise scheduling and iterative denoising while achieving substantially faster inference than diffusion policies. Experiments on various manipulation tasks show that AMP outperforms strong baselines, achieving higher success rates, faster inference, and enhanced spatial reasoning.
Primary: Brown University
All Institutions: Brown University
The authors acknowledge several limitations: 1. **Jittering**: Minor jittering in generated actions, which could be improved with geometric consistency losses. 2. **Workspace Limitation**: The workspace is limited by calibrated camera coverage, preventing actions outside the observed scope. This could be alleviated with larger-FOV or dual in-hand cameras. 3. **Gripper Specificity**: While the method is flexible, the current keypoint design focuses on a parallel-jaw gripper. Adaptation would be needed for dexterous or bimanual setups. 4. **Future Directions**: The paper also points to future work, such as integrating Q-functions for reinforcement learning and exploring pre-trained vision-language models, indicating that the current work is a foundational step. BROADER IMPACT: AMP offers a significant contribution to robot learning by providing an effective and efficient method for closed-loop 3D manipulation. Its ability to handle high-dimensional continuous actions, multi-modality, and achieve high precision with fast inference has broad implications for deploying robots in complex real-world tasks. The classification-based approach, which aligns action representation with observation space, could inspire new architectures and training paradigms in visuomotor control. The enhanced spatial reasoning demonstrated, particularly the responsiveness to small visual cues, is critical for robust and precise robot operation. This method could accelerate the development of more capable and adaptable robotic systems, potentially impacting fields like manufacturing, service robotics, and assistive technologies. The explicit action distribution output also opens avenues for safer and more interpretable robot behavior, as well as easier integration with reinforcement learning and vision-language models. Action Map Policy (AMP) introduces a highly novel and effective framework for 3D closed-loop manipulation by casting action learning as a pixel classification problem using projected keypoint heatmaps. This approach elegantly resolves the challenges of high-dimensional continuous actions, multi-modality, and precision, outperforming strong baselines in both simulated and real-world tasks with significantly faster inference and superior spatial reasoning. The paper's robust methodology, comprehensive experimental validation, and clear articulation of its advantages make it a substantial contribution to the field of robotics and machine learning.
The paper proposes Action Map Policy (AMP), a novel framework that reformulates 3D closed-loop manipulation policy learning as a pixel classification problem. The core idea is to represent 3D end-effector poses as a set of 3D keypoints, project these keypoints onto multiple camera image planes, and then predict their 2D pixel locations as discrete classes (heatmaps). This approach elegantly addresses the challenges of high-dimensional continuous action spaces, combinatorial explosion in naive discretization, and multi-modality. The use of a cross-entropy loss on soft Gaussian heatmaps for supervision is well-justified, providing stable training and naturally modeling multi-modal distributions. The X-Net architecture, a multi-view U-Net encoder combined with a Multi-View Transformer and U-Net decoder, is specifically designed to process multi-view images and output dense action feature maps, preserving spatial structure. The heatmap-to-action conversion process, involving argmax pixel selection, triangulation, and geometric pose recovery, is clearly defined and robust. A significant advantage is the single-pass inference, which is substantially faster than iterative denoising methods like diffusion policies. The method also leverages equivariant data augmentation, which is a natural fit for its spatially aligned action representation. The theoretical precision analysis demonstrating millimeter-level accuracy at reasonable image resolutions is a strong methodological justification.
The experimental evaluation is comprehensive and rigorous, spanning both simulated and real-world settings. 1. **Precision Analysis**: A theoretical analysis quantifies the achievable precision of the heatmap representation at different image resolutions, showing that $224 \times 224$ images yield approximately 1mm translation and 1.3-degree rotation precision, which is crucial for manipulation. This is a strong empirical justification for the chosen representation. 2. **Simulated Experiments**: AMP is evaluated on six challenging MimicGen tasks, covering rigid and articulated objects with varying precision requirements. It is compared against four strong baselines: Diffusion Policy (DiffPo), ACT, OAT, and Motion Track. AMP significantly outperforms these baselines on five out of six tasks, with an average gain of 20.7% in success rate. 3. **Ablation Studies**: Ablations demonstrate the importance of the in-hand camera view, soft labels (Gaussian width $\sigma=2$), and equivariant data augmentation, all contributing positively to performance. 4. **Real-World Experiments**: Three long-horizon, complex real-world tasks (make-coffee, toast-bread, steam-egg) are used to evaluate closed-loop visuomotor policy learning. AMP shows a substantial performance margin (e.g., 50% higher success rate than DiffPo on 'make-coffee') and faster inference speed (13.80 ms vs. 93.53 ms for DiffPo). 5. **Spatial Reasoning**: Two tasks (grasp-target-cup, stack-target-block) are designed to test spatial reasoning, where a laser pointer indicates the target among identical objects. AMP achieves 100% success, demonstrating its ability to respond to fine-grained visual cues, unlike baselines that struggle with mode collapse. The experiments are well-designed, the baselines are strong, and the results consistently highlight AMP's superior performance, efficiency, and robustness.
The paper provides good details for reproducibility. Key implementation choices are described, including image resolution, network architecture (U-Net encoder, MVT, U-Net decoder, skip connections), number of channels, and the use of the first 8 out of 12 predicted timesteps. Hyperparameters like learning rate, training epochs, and GPU used are specified. Crucially, the paper includes pseudocode for both training and inference, which is highly valuable. Details on data preprocessing, augmentation (including equivariant augmentation), and the soft-label width ($\sigma=2$) are provided. The description of the keypoint definition and pose recovery is also clear. The authors state that they focus on policy performance without using pretrained vision encoders for any method, ensuring a fair comparison.
The authors acknowledge several limitations: 1. **Jittering**: Minor jittering in generated actions, which could be improved with geometric consistency losses. 2. **Workspace Limitation**: The workspace is limited by calibrated camera coverage, preventing actions outside the observed scope. This could be alleviated with larger-FOV or dual in-hand cameras. 3. **Gripper Specificity**: While the method is flexible, the current keypoint design focuses on a parallel-jaw gripper. Adaptation would be needed for dexterous or bimanual setups. 4. **Future Directions**: The paper also points to future work, such as integrating Q-functions for reinforcement learning and exploring pre-trained vision-language models, indicating that the current work is a foundational step. BROADER IMPACT: AMP offers a significant contribution to robot learning by providing an effective and efficient method for closed-loop 3D manipulation. Its ability to handle high-dimensional continuous actions, multi-modality, and achieve high precision with fast inference has broad implications for deploying robots in complex real-world tasks. The classification-based approach, which aligns action representation with observation space, could inspire new architectures and training paradigms in visuomotor control. The enhanced spatial reasoning demonstrated, particularly the responsiveness to small visual cues, is critical for robust and precise robot operation. This method could accelerate the development of more capable and adaptable robotic systems, potentially impacting fields like manufacturing, service robotics, and assistive technologies. The explicit action distribution output also opens avenues for safer and more interpretable robot behavior, as well as easier integration with reinforcement learning and vision-language models. Action Map Policy (AMP) introduces a highly novel and effective framework for 3D closed-loop manipulation by casting action learning as a pixel classification problem using projected keypoint heatmaps. This approach elegantly resolves the challenges of high-dimensional continuous actions, multi-modality, and precision, outperforming strong baselines in both simulated and real-world tasks with significantly faster inference and superior spatial reasoning. The paper's robust methodology, comprehensive experimental validation, and clear articulation of its advantages make it a substantial contribution to the field of robotics and machine learning.
Whole-arm manipulation involves direct contact with the environment while the robot completes a task by distributing contact across multiple links as contacts form, slide, and break. This setting breaks common implicit assumptions in many learning-based manipulation pipelines: arm configuration tightly couples motion and contact forces, contact state is partially observed under occlusion, and purely learned rollouts can become physically inconsistent under distribution shift because many multi-link contact configurations are sparsely represented in the data. To address this, we propose TACTIC (Tactile and Vision Conditioned Contact-Centric Control), a receding-horizon controller for whole-arm manipulation. TACTIC uses a contact-centric hybrid predictive model that combines RGB-D, distributed tactile sensing, and a compact 2D proximity representation. The model couples a learned, action-conditioned latent dynamics model with analytical kinematics through contact Jacobians, enabling rollouts of future contact configurations and interaction forces. TACTIC integrates these rollouts into a sampling-based MPC planner with contact-aware action sampling: contact Jacobian-based projections steer sampled action sequences toward force-modulating directions, and objectives defined over predicted proximity and interaction forces trade task progress against whole-arm force regulation. We evaluate TACTIC in simulation against state-of-the-art model-based and model-free methods, and perform ablations that isolate the contribution of each design choice. TACTIC consistently outperforms other methods. We further demonstrate real-world performance on a robot with distributed tactile sensing across three whole-arm manipulation tasks that require multi-contact trajectories: turning over and repositioning a manikin, and goal-reaching in a 3D dynamic maze. Website: https://emprise.cs.cornell.edu/tactic
Primary: Cornell University
All Institutions: Cornell University, Carnegie Mellon University, Toyota Research Institute
TACTIC represents a significant advancement in whole-arm manipulation by effectively integrating tactile and visual information with hybrid predictive modeling and contact-aware MPC, demonstrating superior performance in complex, contact-rich real-world tasks.
The paper proposes TACTIC, a receding-horizon controller for whole-arm manipulation that addresses the challenges of partial observability and physical inconsistency in contact-rich tasks. The core innovation lies in a hybrid predictive model that fuses learned latent dynamics with analytical kinematics via contact Jacobians. This allows for physically consistent rollouts while leveraging the flexibility of learned models for unmodeled dynamics. The integration of this model into a Model Predictive Path Integral (MPPI) planner with contact-aware action sampling is a sophisticated engineering and theoretical contribution. The use of distributed tactile sensing and RGB-D data to form a contact-centric representation is well-motivated and technically sound.
The evaluation is comprehensive, covering both simulation and real-world experiments. The real-world demonstrations on a robot with distributed tactile sensing performing complex tasks like turning over a manikin and navigating a dynamic maze are strong evidence of the method's efficacy. The ablation studies help isolate the contributions of different components (e.g., tactile vs. vision, learned vs. analytical dynamics). The comparison against state-of-the-art model-based and model-free methods provides a solid baseline for performance claims. The latency analysis demonstrates the practical feasibility of the approach at ~12Hz.
The paper provides a detailed system overview, including latency breakdowns and hyperparameter tables in the appendix. The website link suggests the availability of additional resources, though code is not explicitly linked in the provided text. The description of the sensor fusion and MPC pipeline is sufficiently detailed for reproduction by experts in the field. The use of standard simulators (implied by "simulation") and real hardware with known specifications aids reproducibility.
The method relies on accurate tactile sensing and RGB-D data, which may be expensive or difficult to scale to larger robots. The performance of the learned latent dynamics model is dependent on the quality and diversity of the training data. The contact Jacobian-based projections assume a certain level of contact stability, which might be challenging in highly dynamic or slipping scenarios. The complexity of the hybrid model and MPC planner may limit its application to robots with significant computational resources.
This work advances the field of robotics by enabling more dexterous and robust manipulation capabilities, which are crucial for applications in healthcare, logistics, and domestic assistance. The integration of tactile sensing with vision and control theory provides a blueprint for future research in contact-rich manipulation. The real-world demonstrations highlight the potential for these methods to be deployed in real-world scenarios, moving beyond simulation-only evaluations. TACTIC represents a significant advancement in whole-arm manipulation by effectively integrating tactile and visual information with hybrid predictive modeling and contact-aware MPC, demonstrating superior performance in complex, contact-rich real-world tasks.