ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs
Abstract
Rapid advances in large language models have made software engineering agents capable of tackling complex coding tasks. Within a containerized environment, a coding agent receives tool responses or execution results, and determines its later actions by those earlier results. However, most existing coding agents never look ahead: their policies do not anticipate the consequences of actions, and the terminal reward barely attributes a failure to certain earlier steps, so an ill-judged edit can corrupt the repository state irreversibly before any signal arrives. Recent code world models aim to address these challenges, but they are either trained and then frozen before reinforcement learning (RL), or reduced to an auxiliary next-token loss on the environment's outputs. We present ANTE (Anticipate, Then Execute), an agentic RL framework that verifies the policy's pre-action predictions by real execution and turns them into per-step credit for its actions and trust-or-abstain decisions. With an internal belief stated before each action, Belief-Anchored Policy Optimization (BAPO) turns the settled predictions into per-step advantages by anchoring credit on the policy's own forecasts. To turn the trained forecasts into better actions at inference, thereby facilitating effective test-time scaling, ANTE-TTO (test-time optimization) selects among candidate actions without an external scorer. ANTE raises Qwen3-32B from 25.0% to 57.2% pass@1 on SWE-bench Verified, 6.2% above a matched GRPO baseline trained without the belief state; its world model's accuracy climbs from 44% to 69% during RL, and the learned foresight capability transfers across harnesses and out-of-distribution domains. In summary, we aim to build a code world model that knows what has happened, what holds now, and what the next action will achieve.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents ANTE, a framework that uses verified internal world models to improve credit assignment in coding agents via Belief-Anchored Policy Optimization, significantly boosting performance on SWE-bench Verified. The technical contribution is substantial, addressing a core challenge in agentic RL: the misalignment between sparse terminal rewards and the need for fine-grained action evaluation. By anchoring credit on the agent's own forecasts, ANTE creates a dense, self-supervised signal that guides policy optimization more effectively than standard methods. The methodology is innovative, moving beyond treating world models as static predictors or auxiliary losses, and instead integrating them dynamically into the RL loop. The experimental results are strong, showing a large absolute gain in pass@1 on a difficult benchmark, which is a key metric for coding agents. The significance to the field is high, as it provides a scalable and effective approach to training agents that can anticipate consequences, a crucial capability for real-world deployment. The work bridges the gap between world modeling and policy optimization, offering a new paradigm for agentic learning.
Methodology
The paper introduces ANTE (Anticipate, Then Execute), a framework that integrates a learned internal world model into the reinforcement learning loop for coding agents. The core innovation is "Belief-Anchored Policy Optimization" (BAPO), which uses the agent's own pre-action predictions (beliefs) about the environment's state to compute per-step credit assignment, rather than relying solely on sparse terminal rewards. This addresses the credit assignment problem in long-horizon agentic tasks. The method also includes ANTE-TTO (Test-Time Optimization), which leverages the trained world model to select among candidate actions at inference time without an external verifier. The methodology is sound, combining standard RL techniques with a novel use of self-predictive capabilities for credit assignment. The approach is distinct from prior work that freezes world models before RL or uses them only as auxiliary losses.
Results & evaluation
The experiments are conducted on SWE-bench Verified, a standard and challenging benchmark for software engineering agents. The paper reports a significant improvement in pass@1 from 25.0% to 57.2% using Qwen3-32B, outperforming a matched GRPO baseline by 6.2%. The world model's accuracy improves from 44% to 69% during RL, demonstrating that the foresight capability is learned effectively. The paper also claims transferability of the learned foresight across different harnesses and out-of-distribution domains, which is a strong claim supported by the reported results. The evaluation is rigorous and uses a large, capable base model, making the results highly relevant to current state-of-the-art practices.
Reproducibility
The paper provides detailed descriptions of the BAPO algorithm and the ANTE-TTO process. However, specific hyperparameters, training schedules, and implementation details for the world model architecture are likely in the appendix (referenced but not fully visible in the prompt). The use of standard benchmarks and open-source base models (Qwen3) aids reproducibility. The absence of a provided code repository in the extracted text is a minor drawback, but the methodological clarity is high.
Limitations
The primary limitation is the computational cost of maintaining and updating a world model during RL, which may be prohibitive for smaller models or resource-constrained settings. The reliance on a large base model (Qwen3-32B) means the results may not directly translate to smaller agents. Additionally, the "trust-or-abstain" decision mechanism's robustness to adversarial or highly noisy environment feedback is not deeply explored. The paper focuses on coding tasks, and generalization to other agentic domains (e.g., robotics, web navigation) is implied but not extensively tested.
Broader impact
This work has significant implications for the development of autonomous agents in complex, stateful environments. By improving credit assignment through internal world models, it offers a pathway to more reliable and efficient agent training. The technique of using self-predictions for credit assignment could be generalized to other domains where sparse rewards are a bottleneck, such as robotics or scientific discovery. The improvement in SWE-bench Verified performance suggests practical utility in software engineering automation. The paper presents ANTE, a framework that uses verified internal world models to improve credit assignment in coding agents via Belief-Anchored Policy Optimization, significantly boosting performance on SWE-bench Verified. The technical contribution is substantial, addressing a core challenge in agentic RL: the misalignment between sparse terminal rewards and the need for fine-grained action evaluation. By anchoring credit on the agent's own forecasts, ANTE creates a dense, self-supervised signal that guides policy optimization more effectively than standard methods. The methodology is innovative, moving beyond treating world models as static predictors or auxiliary losses, and instead integrating them dynamically into the RL loop. The experimental results are strong, showing a large absolute gain in pass@1 on a difficult benchmark, which is a key metric for coding agents. The significance to the field is high, as it provides a scalable and effective approach to training agents that can anticipate consequences, a crucial capability for real-world deployment. The work bridges the gap between world modeling and policy optimization, offering a new paradigm for agentic learning.
Author of this paper? Send a correction.