Random Remasking Scales Masked Diffusion Inference
Abstract
Masked diffusion models (MDMs) generate sequences by progressively unmasking tokens, but errors introduced during decoding are typically irreversible. Recent work has shown that iterative correction can improve sample quality, yet existing theory fails to explain this given a learned, imperfect denoiser model. We re-formulate MDM inference as a Markov chain on the clean sequence space, defined by a remask-denoise kernel that randomly re-introduces masks and then denoises. This helps us analyze the actual denoising trajectory towards the data distribution. The analysis motivates Random Remask-Denoise (R$^2$D) inference, a purely test-time, finetuning-free, reward-free procedure. To further improve performance, we propose Trajectory Correction Training (TCT), a denoiser-training procedure that exposes the model to its own errors along the generation trajectory. On logical tasks, our approach exhibits a striking near-log-linear test-time scaling, achieving over 97% accuracy on hard Sudoku puzzles and over 99% accuracy on 3-SAT. Our approach also applies to pretrained models on actual language tasks, beating iterative correction baselines and the official Q-mode accuracy of LLaDA2.1-mini on HumanEval+.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a unified Markov-chain framework for masked diffusion inference, deriving a principled test-time scaling algorithm (R2D) and a corresponding training method (TCT) that significantly improve accuracy on logical and coding tasks.
Methodology
The paper proposes a unified Markov chain framework for Masked Diffusion Model (MDM) inference, re-formulating the process as a remask-denoise kernel on the clean sequence space. This theoretical shift allows for a rigorous analysis of the denoising trajectory using KL divergence contraction bounds. The authors derive a single-step update bound that identifies three key factors: timestep schedule, task structure, and model quality. Based on this, they propose Random Remask-Denoise (R2D) inference, an adaptive, reward-free test-time scaling method that iterates until prediction stabilizes. Additionally, they introduce Trajectory Correction Training (TCT), a training procedure that exposes the model to its own errors along the generation trajectory, effectively teaching it to correct mistakes. The methodology is sound, bridging the gap between diffusion theory and practical iterative correction by providing a principled justification for why and when remasking helps.
Results & evaluation
The experiments are rigorous and cover both synthetic logical tasks (Sudoku-Hard, 3-SAT) and a real-world language task (HumanEval+ with LLaDA2.1-mini). The results demonstrate a striking near-log-linear test-time scaling effect on logical tasks, achieving >97% accuracy on hard Sudoku and >99% on 3-SAT. The method outperforms baselines like ReMDM and ProSeCo in both accuracy and compute efficiency (NFE). The application to the 16B LLaDA2.1-mini model shows that R2D can beat the official Q-mode accuracy on HumanEval+ without any fine-tuning, which is a strong practical result. Ablation studies confirm the importance of uniform remasking and argmax decoding for the scaling effect.
Reproducibility
The paper provides detailed pseudo-code for both the inference algorithm (R2D) and the training procedure (TCT). Hyperparameters such as correction budgets, early-exit patience, and model architectures are specified. The use of standard datasets (SATLIB, Sudoku-Extreme) and a public model (LLaDA2.1-mini) enhances reproducibility. However, the specific implementation details of the TCT training loop (e.g., how batches are managed across chains) are described in the appendix but could benefit from more explicit code-level details.
Limitations
The primary limitation is the computational cost of iterative correction, although the paper argues it is more efficient than baselines. The method relies on argmax decoding for the scaling effect, which may limit diversity in generation tasks where sampling is preferred. The logical tasks, while impressive, are discrete and constraint-heavy; the generalizability to other types of continuous or open-ended generation tasks is less clear, though the HumanEval+ result is promising. The theory assumes a Markovian remask step, which is a simplification of some standard inference procedures.
Broader impact
This work provides a theoretical foundation for test-time scaling in discrete diffusion models, a rapidly growing area. By showing that iterative correction can be viewed as a Markov chain convergence process, it offers a principled way to allocate compute during inference. The ability to improve accuracy of pretrained models without fine-tuning (as shown with LLaDA2.1-mini) is highly valuable for practitioners who cannot retrain large models. It may influence future designs of diffusion-based LLMs to incorporate self-correction capabilities natively. The paper presents a unified Markov-chain framework for masked diffusion inference, deriving a principled test-time scaling algorithm (R2D) and a corresponding training method (TCT) that significantly improve accuracy on logical and coding tasks.
Author of this paper? Send a correction.