WeatherDreamer: Learning a World Model from Multimodal Earth Observations
Abstract
Machine learning weather models now rival operational global forecasts but still start from analyses produced by separate, expensive data assimilation (DA) systems, which for global models are typically updated every six hours. Observation-to-observation models avoid this dependence, but most omit the recursive filtering cycle that lets DA systems carry a background state through sparse, irregular observations. We introduce WeatherDreamer, a world model that learns this cycle end-to-end from 18 global observing systems, without reanalysis. It infers a latent state from observations, carries it forward hour by hour with learned dynamics, corrects it each hour with new observations, and can predict what any of its observing systems would measure at any location. At held-out stations, its ensembles beat NOAA's Global Ensemble Forecast System (GEFS) for temperature, dew point and wind out to 12 h and are better calibrated than GEFS and the ECMWF ensemble. Precipitation, cloud-top height and solar irradiance, never inputs at inference, are inferred with skill matching or exceeding operational products at short leads. Against the first guess of NOAA's Gridpoint Statistical Interpolation (GSI) DA system, WeatherDreamer's short-range departures are within 10% of GSI's at surface stations and within 25% for moisture-sensitive microwave channels, but up to five times larger for temperature-sounding channels. Never given observation errors, its learned update assumes errors close to known instrument and retrieval errors of microwave sounders and scatterometer winds. WeatherDreamer also separates viewing geometry from atmospheric state, recovering 83-96% of observed limb darkening and producing seamless nadir-view imagery. Producing an hourly global analysis in about 1.5 minutes on one GPU, WeatherDreamer shows that assimilation and forecasting can be learned together directly from observations.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
WeatherDreamer introduces a unified world model that learns data assimilation and forecasting end-to-end from raw multimodal observations, achieving competitive or superior performance to operational systems like GEFS and ECMWF while implicitly learning observation error statistics. The paper demonstrates a significant technical leap in applying deep learning to physical systems by replacing hand-crafted variational assimilation with a learned recurrent latent state, offering a scalable and efficient alternative to traditional numerical weather prediction pipelines.
Methodology
The paper proposes WeatherDreamer, a world model that unifies data assimilation and forecasting into a single end-to-end learning pipeline. The core architectural innovation is the use of a recurrent state-space model (likely a variant of S4 or Mamba, given the "recurrent cell" description) that maintains a latent state. This state is updated via an "observation pathway" (assimilation) and an "action pathway" (dynamics/forcing). A significant technical contribution is the use of Hyperspherical Observation Tokenizers (SVAEs with von Mises-Fisher posteriors) to compress multimodal satellite data, which the authors claim yields 5-10x higher compression than Gaussian latents. The decoding mechanism uses Local Implicit Image Functions (LIIF) to allow continuous, arbitrary-resolution rendering of physical fields, avoiding grid artifacts. The model learns the recursive filtering cycle (predict, correct, predict) directly from 18 global observing systems without explicit reanalysis or hand-crafted observation error covariances.
Results & evaluation
The evaluation is rigorous and comprehensive. The model is trained on 2020-2023 data and tested on 2024-2025. It outperforms NOAA's GEFS and ECMWF's IFS ENS in ensemble calibration (CRPS) for temperature, dew point, and wind out to 12 hours. Crucially, it infers precipitation, cloud-top height, and solar irradiance—modalities not used as inputs at inference—with skill matching or exceeding operational products. The comparison against the NCEP GSI first guess is particularly strong, showing that the learned assimilation cycle produces departures within 10% of GSI's for surface stations. The paper also demonstrates that the model implicitly learns observation error statistics, with learned gains matching known instrument errors.
Reproducibility
The paper provides extensive supporting information detailing preprocessing, tokenizer architecture, training hyperparameters (AdamW, learning rates, batch sizes), and verification methods. The use of standard public datasets (IMERG, GSI diagnostics, ISD stations) and open-source baselines (GEFS, IFS) enhances reproducibility. However, the specific implementation of the "recurrent state-space model" and the proprietary nature of Zeus AI's infrastructure may limit full reproduction by external groups.
Limitations
The model is evaluated primarily on short-range forecasts (up to 12 hours) and short-range assimilation. Its performance on longer lead times (beyond 12h) is not fully detailed in the abstract, though the world model framework suggests potential for longer horizons. The reliance on a single GPU for inference is a strength, but the training cost (4 GH200 GPUs for 70k steps) is significant. The separation of viewing geometry from atmospheric state is a novel claim, but the generalization to new satellite orbits or sensors not seen in training is not explicitly tested.
Broader impact
This work has the potential to disrupt the traditional weather forecasting pipeline, which relies on separate, expensive data assimilation systems. By learning assimilation and forecasting jointly, it could reduce computational costs and improve the integration of new observing systems. The approach is applicable to other domains where sparse, irregular observations must be fused with physical dynamics, such as oceanography, climate science, and even certain industrial monitoring systems. WeatherDreamer introduces a unified world model that learns data assimilation and forecasting end-to-end from raw multimodal observations, achieving competitive or superior performance to operational systems like GEFS and ECMWF while implicitly learning observation error statistics. The paper demonstrates a significant technical leap in applying deep learning to physical systems by replacing hand-crafted variational assimilation with a learned recurrent latent state, offering a scalable and efficient alternative to traditional numerical weather prediction pipelines.
Author of this paper? Send a correction.