Terrarium: Growing Personal Assistants via Scalable Synthesis of App Worlds
Abstract
Advances in large language models (LLMs) have brought the vision of a general-purpose personal assistant closer to reality. Such assistants could provide a unified natural-language interface to a broad ecosystem of applications. However, real-world applications do not readily support training at scale because access is often restricted and their state is difficult to reset. To address this challenge, we introduce Terrarium, a framework that synthesizes executable App Worlds from real-world application information, generates training task packages with evaluator-side rubrics from these worlds, and iteratively validates and refines both through agent rollouts. For App World synthesis, Terrarium organizes public Web information and in-application observations into structured app specifications that guide the construction of self-contained runtimes with callable operations and mutable, resettable state. For task synthesis, Terrarium constructs tasks grounded in App World state together with evaluator-side outcome rubrics that assess success without prescribing reference trajectories. We construct a training suite of 275 App Worlds and 13,344 tasks for outcome-based reinforcement learning across three Qwen3 model scales (4B, 8B, and 14B parameters). Across these scales, models trained with Terrarium outperform the corresponding Qwen3 models by an average of 18.34 percentage points on the $τ^2$-bench and 10.94 percentage points on the AppWorld bench. These results suggest that app-grounded world and task synthesis offers a scalable path toward general-purpose assistants capable of operating across broad application ecosystems.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a scalable framework for synthesizing executable app worlds and grounded training tasks to improve LLM agent performance, demonstrating significant gains on standard benchmarks through outcome-based reinforcement learning.
Methodology
The paper introduces "Terrarium," a framework for synthesizing executable "App Worlds" from real-world application information (web search + GUI traces) to train LLM agents. The core methodological contribution is the creation of a structured intermediate representation (app specification) that guides the construction of self-contained, resettable runtimes with mutable state. This addresses the key bottleneck in agent training: the lack of scalable, realistic, and controllable environments. The pipeline involves synthesizing the world, generating grounded tasks with evaluator-side rubrics (rather than reference trajectories), and using a closed-loop validation process where agent rollouts identify and repair defects in the world, task, or evaluator. The use of Group Relative Policy Optimization (GRPO) with binary outcome rewards is a solid choice for this setting, allowing the model to learn from success/failure signals without requiring dense trajectory supervision. The distinction between "Intent Execution" and "Intent Completion" (handling missing information) is a thoughtful addition that adds nuance to the training signal.
Results & evaluation
The experimental evaluation is rigorous and comprehensive. The authors train Qwen3 models at three scales (4B, 8B, 14B) and evaluate them on two independent benchmarks: $\tau^2$-bench and AppWorld. The results show consistent and significant improvements over the base models (18.34 points on $\tau^2$-bench, 10.94 on AppWorld). The ablation studies are particularly strong, demonstrating that GRPO outperforms SFT, that combining Execution and Completion supervision is beneficial, and that performance scales with the number of synthesized App Worlds. The comparison with other synthesis methods (AWM, EnvScaler) further validates the effectiveness of the app-grounded approach. The inclusion of BFCL Multi-Turn evaluation provides additional evidence of generalization.
Reproducibility
The paper provides extensive details on the training configuration, including hyperparameters, dataset statistics, and the judge prompt. The authors state they will open-source the framework and artifacts, which is a major plus for reproducibility. The detailed description of the closed-loop validation process and the failure taxonomy helps in understanding how the data quality is maintained. However, the actual code and data are not yet available, which is a minor limitation for immediate reproducibility.
Limitations
The primary limitation is the reliance on a large, proprietary model (Qwen3.5-397B-A17B) as the evaluator-side judge. This introduces a potential bias and makes the method less accessible to smaller labs. Additionally, the synthesized worlds are surrogates and may not perfectly capture the complexity and edge cases of real-world applications. The method is also computationally expensive, requiring significant GPU resources for both synthesis and training.
Broader impact
This work has significant potential to impact the field of LLM agents by providing a scalable path to training general-purpose assistants. The ability to synthesize realistic, controllable environments from public information could accelerate research in agent learning and enable the development of more capable and reliable AI systems. The open-sourcing of the framework and data will likely foster further innovation in this area. The paper presents a scalable framework for synthesizing executable app worlds and grounded training tasks to improve LLM agent performance, demonstrating significant gains on standard benchmarks through outcome-based reinforcement learning.
Author of this paper? Send a correction.