Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards
Abstract
Long-horizon agents have drawn growing attention for their ability to work autonomously -- from hours to days -- on economically valuable tasks, a capability essential to automated discovery and recursive self-improvement (RSI). Yet open-source models still struggle on long-horizon tasks, largely for lack of both training data and a working recipe, and scaling both is hard: long, complex tasks are difficult to curate and verify, and their rewards are sparse. We address both. First, we introduce Tev-Gym, the first long-horizon agent training dataset with verifiable, dense, progressive rewards -- 1,170 verified task environments across 7 domains. It is built on a latent-world-first approach that first constructs a latent world: a complete and correct hidden specification of the task domain, workflow, and expected state after each step. Each task is then progressively rendered from this specification into an agent-facing environment, enabling controllable complexity and a natural progress metric. Second, we turn this metric into a training recipe: via progress-potential reward shaping (PPRS), we convert progress into a dense per-step reward that improves training efficiency and makes otherwise-intractable tasks trainable. Training with this recipe yields Tev-27B, which achieves substantial gains on the Agents' Last Exam computing and math sciences domains (ALE-CM), matching GPT-6 Astra and Claude Opus 5 at Pass@5. Despite training only on ALE-related data, it generalizes to the Terminal-Bench (TB) 3.0 and 4.0 subsets, surpassing GLM 5.2 and Gemini 3.7 Flash, and reaches Claude Opus 4.8-level performance on middle-horizon tasks such as TB 2.1.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces a latent-world-first synthesis pipeline and Progress-Potential Reward Shaping (PPRS) to enable effective RL training for long-horizon agents, resulting in open-source models that match frontier closed-source systems on complex agentic benchmarks. This is a significant contribution that addresses the critical data and reward sparsity bottlenecks in agentic AI, offering a reproducible and scalable recipe for generating high-quality training environments and training signals.
Methodology
The paper introduces a "latent-world-first" synthesis pipeline for creating long-horizon agent environments. This is a significant methodological contribution because it addresses the core bottleneck in agentic RL: the lack of verifiable, dense rewards for complex, multi-step tasks. By constructing a complete, correct hidden specification (the latent world) first and then rendering the task and corruption from it, the authors ensure that ground truth is known by construction. This allows for the derivation of a "progress potential" function, which is used in a novel reward shaping scheme called Progress-Potential Reward Shaping (PPRS). PPRS is theoretically grounded in potential-based reward shaping but adapted for GRPO to prevent reward starvation in groups where no rollout succeeds. The methodology is rigorous, with clear stages for world construction, workflow graphing, artifact rendering, and controlled corruption. The inclusion of qualification gates (exploit resistance, progress validity, role independence) demonstrates a high level of care in preventing reward hacking and ensuring data quality.
Results & evaluation
The experiments are extensive and well-designed. The authors train two model sizes (9B and 27B) and evaluate them on the Agents' Last Exam (ALE-CM) and Terminal-Bench (TB) 3.0/4.0. The results show substantial gains, with the 27B model matching frontier closed-source models (GPT-6 Astra, Claude Opus 5) on Pass@5 for ALE-CM. Crucially, the paper demonstrates generalization to unseen benchmarks (TB), which is a strong indicator of learning robust agentic behaviors rather than overfitting to the training distribution. The ablation studies likely confirm the importance of both the dense reward and the specific data synthesis method. The comparison against prior open-source datasets (SWE-Smith, etc.) highlights the complexity gap, with Tev-Gym tasks being significantly longer and harder.
Reproducibility
The paper provides detailed descriptions of the synthesis pipeline, the reward shaping formula, and the training setup (DPPO, group size, token budgets). The "latent world" concept is well-defined, and the qualification gates are described formally. However, without access to the code or the specific generators for the 1,170 environments, full reproduction is difficult. The reliance on specific base models (Qwen3.5/3.8) and the proprietary nature of some benchmark subsets (if any) could limit reproducibility for external labs. The paper is a preprint, so code release status is unknown, but the methodological detail is high.
Limitations
The primary limitation is the scale of the dataset (1,170 environments) compared to massive web-scale datasets, though the quality is higher. The approach is currently focused on specific domains (software, data, math, etc.) and may not generalize to all types of long-horizon tasks (e.g., creative writing, open-ended research). The computational cost of generating and verifying these latent worlds is likely high. Additionally, the performance gains are relative to open-source baselines; while matching closed-source models is impressive, the absolute performance on the hardest tasks may still be limited by the base model's capabilities.
Broader impact
This work has high potential impact on the field of agentic AI. By providing a recipe for generating high-quality, verifiable long-horizon training data, it lowers the barrier for other labs to train competitive open-source agents. The PPRS technique is a generalizable insight for RL in sparse-reward settings. If the dataset and code are released, it could become a standard benchmark and training resource for the community, similar to how SWE-bench or Terminal-Bench have been used. It pushes the frontier of what open-source models can achieve in autonomous, long-duration tasks. The paper introduces a latent-world-first synthesis pipeline and Progress-Potential Reward Shaping (PPRS) to enable effective RL training for long-horizon agents, resulting in open-source models that match frontier closed-source systems on complex agentic benchmarks. This is a significant contribution that addresses the critical data and reward sparsity bottlenecks in agentic AI, offering a reproducible and scalable recipe for generating high-quality training environments and training signals.
Author of this paper? Send a correction.