GPWM: Modeling the World as 3D Particle Motion
Abstract
We introduce GPWM, a Generative Particle World Model that predicts how 3D scenes evolve by generating the motion of particles and rigid bodies over time. Unlike video world models that generate scene appearance, GPWM operates directly in 3D physical space. A single representation and architecture cover rigid, deformable, cloth, and granular materials, in both passive physics and robotic interaction. GPWM is an object-aware multi-resolution transformer that combines flow matching with diffusion forcing to generate coherent physical trajectories far beyond its training horizon. We train and evaluate GPWM on ParticleWorlds, a 25.5-million-frame corpus spanning diverse geometries, materials, interactions, and robot embodiments. GPWM outperforms the best prior learned-dynamics baseline in each of six physical domains, reducing prediction error by 31% on average. Breaking from the domain-specific setting of prior methods, a single Unified GPWM models these heterogeneous regimes with one set of weights, largely outperforming per-domain baselines and even surpassing our own specialist models in two domains. GPWM further generalizes to unseen geometries, physical properties, object counts, and rollout horizons. Beyond prediction, GPWM enables contact-rich planning by learning a generative prior over joint actuator-object motion. Instead of searching over unconstrained actions, planning searches the model's latent noise for interactions that achieve a desired object-space goal. With the same world model, noise-space planning substantially outperforms action-space search on pushing and picking, where complex contact sequences are hard to discover. Executed open-loop in simulation, the resulting plans achieve goal-directed manipulation without reference action trajectories or goal-specific policies.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
GPWM introduces a unified particle-based world model that predicts 3D physical dynamics across diverse materials and interactions, achieving significant improvements in prediction accuracy and enabling effective contact-rich planning through noise-space search. The paper makes a strong contribution by demonstrating that a single generative model can handle heterogeneous physical regimes, outperforming domain-specific baselines, and providing a scalable framework for learning physical priors that can be directly applied to robotic manipulation tasks.
Methodology
The paper proposes GPWM, a Generative Particle World Model that shifts the paradigm of world modeling from pixel-space video generation to 3D physical space. By representing scenes as particles and rigid bodies, the model avoids the high-dimensional burden of appearance modeling, focusing instead on physical dynamics. The architecture is an object-aware multi-resolution transformer that integrates flow matching with diffusion forcing. This combination allows for coherent trajectory generation over long horizons, addressing the common issue of error accumulation in autoregressive world models. The unification of rigid, deformable, cloth, and granular materials under a single set of weights is a significant architectural and conceptual contribution, demonstrating the potential of particle-based representations to generalize across diverse physical regimes.
Results & evaluation
The authors introduce ParticleWorlds, a substantial 25.5-million-frame corpus, which serves as a strong benchmark for evaluating physical world models. The experimental results show a 31% average reduction in prediction error compared to the best prior learned-dynamics baselines across six physical domains. The unified model's ability to outperform specialist models in some domains is a compelling result. Furthermore, the application of the model to contact-rich planning via noise-space search demonstrates practical utility, showing that the learned generative prior can guide manipulation tasks without explicit action trajectories. The evaluation covers prediction, generalization to unseen geometries and horizons, and downstream planning tasks, providing a comprehensive assessment.
Reproducibility
The paper provides detailed descriptions of the architecture, training objectives, and dataset composition. However, as is common with large-scale generative models, the computational resources required for training on a 25.5M frame corpus are significant. The availability of code and pre-trained models is not explicitly confirmed in the provided text, which may limit immediate reproducibility for smaller labs. The detailed breakdown of the particle representation and the specific implementation of flow matching with diffusion forcing offers a clear path for replication, assuming access to the dataset and sufficient compute.
Limitations
The primary limitation is the reliance on simulation data for training. While the model generalizes to unseen geometries and properties within the simulation, its performance in the real world with sensor noise and unmodeled dynamics is not evaluated. Additionally, the particle-based representation, while efficient for physics, may struggle with capturing high-frequency visual details or complex material interactions that are not well-modeled by the particle system. The computational cost of inference for long-horizon rollouts could also be a barrier for real-time applications.
Broader impact
This work has the potential to significantly impact the field of robotics and embodied AI by providing a scalable and unified framework for learning physical dynamics. By decoupling physical prediction from appearance generation, it offers a more efficient path to building world models for control and planning. The introduction of a large-scale, diverse dataset for physical world modeling will likely become a standard benchmark for future research. The approach to planning via noise-space search could inspire new methods for sample-efficient reinforcement learning and manipulation. GPWM introduces a unified particle-based world model that predicts 3D physical dynamics across diverse materials and interactions, achieving significant improvements in prediction accuracy and enabling effective contact-rich planning through noise-space search. The paper makes a strong contribution by demonstrating that a single generative model can handle heterogeneous physical regimes, outperforming domain-specific baselines, and providing a scalable framework for learning physical priors that can be directly applied to robotic manipulation tasks.
Author of this paper? Send a correction.