Zatom-2: Multitask Pretraining on Atomistic Data for Generative Modeling across Domains
Abstract
Unified atomistic modeling has the potential to accelerate discovery in chemistry, materials science, and biology by bridging data-rich chemical domains and data-scarce biological contexts. However, existing generative approaches to atomistic modeling remain highly specialized to scientific disciplines (chemistry vs. biology) or do not leverage both high-volume organic (molecule) and inorganic (material) data for general-purpose pretraining. To this end, we introduce Zatom-2, an atomistic generative model pretrained on approximately five million structures from the OMol25 and OMat24 electronic structure datasets. Zatom-2 features a multiscale Transformer architecture coupled with conditional flow matching that supports force conditioning and foundational pretraining tasks such as generation, structure prediction, and prediction of molecular and material energies and forces. Empirically, Zatom-2 achieves better molecular distribution fidelity than Zatom-1 and achieves strong performance on existing molecule and material generation benchmarks. Zatom-2 demonstrates the ability to control sample generation across low- and high-force regimes, and enhances protein generation in a low-data setting through joint generative-predictive pretraining and transfer learning, increasing protein backbone designability in a length extrapolation setting from 67.8% without pretraining to 74.8% after finetuning on 2,000 protein domains.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
Zatom-2 introduces a unified atomistic generative model that leverages multitask pretraining on large-scale molecular and material datasets to enable cross-domain transfer to protein generation. By combining a novel atom1 tokenization scheme with conditional flow matching and physics-grounded auxiliary objectives (energy/force prediction), the model demonstrates superior distribution fidelity and, crucially, the ability to transfer geometric representations from small molecules to larger biomolecules, achieving significant gains in protein designability in low-data and out-of-distribution settings.
Methodology
The paper introduces Zatom-2, a unified atomistic generative model that bridges the gap between small molecule/material generation and protein structure prediction. The core methodological contribution is the "atom1" tokenization scheme for molecules/materials (analogous to "atom14" for proteins), which allows a single Transformer backbone to handle variable atom counts without separate diffusion tracks for atom identity. The model employs conditional flow matching for coordinate generation and incorporates a novel "force conditioning" mechanism that uses the mean force norm as a global descriptor to control the relaxation state of generated structures. A key innovation is the multitask pretraining curriculum that jointly optimizes for generation, structure prediction, and energy/force prediction (MLIP objectives). The authors hypothesize and demonstrate that the physics-grounded supervision from energy/force prediction shapes the latent representations in a way that improves generative fidelity and transferability, particularly in low-data regimes.
Results & evaluation
The experimental evaluation is extensive, covering pretraining on ~5 million structures from OMol25 and OMat24. The paper demonstrates that Zatom-2 outperforms its predecessor (Zatom-1) in distribution fidelity and achieves competitive results on standard benchmarks like QM9, MP20, and GEOM-Drugs. A significant portion of the evaluation focuses on the "force consistency" metric, showing the model can reliably distinguish between high-force (rattled) and low-force (relaxed) regimes. The most compelling results are in the transfer learning section, where Zatom-2 pretrained on small molecules/materials is fine-tuned on a small protein dataset (SCOPe-2k). The model shows significant improvements in designability and novelty, especially in out-of-distribution (longer sequence) settings, compared to training from scratch or using protein-specific models like RFdiffusion3. The scaling study confirms that both data coverage and model capacity yield complementary gains.
Reproducibility
The paper provides a strong reproducibility statement, including a link to the open-source code repository (https://github.com/Zatom-AI/nucleus) which contains documentation, data loading, and training/inference materials. The appendix details the model architecture, hyperparameters, and compute resources used (NVIDIA A100 GPUs). The authors also provide an AI Use Statement, disclosing the use of LLMs for code and writing assistance, which adds to the transparency of the research process.
Limitations
The model does not enforce exact rotational equivariance, relying instead on data augmentation and pair-biased attention, which may limit its performance in tasks requiring strict symmetry preservation compared to equivariant diffusion models. The MLIP (energy/force) prediction accuracy is lower than specialized potentials, though it serves as an auxiliary task. The protein generation experiments are limited to a small dataset (2k domains) and do not yet match the performance of state-of-the-art protein-specific models in in-distribution settings, though it shows promise in extrapolation. The paper acknowledges that precise force matching within regimes remains challenging.
Broader impact
This work has significant potential to accelerate discovery in chemistry, materials science, and biology by providing a unified framework for atomistic modeling. The ability to transfer knowledge from data-rich chemical domains to data-scarce biological contexts could reduce the computational cost of training specialized models. The force conditioning mechanism offers a new tool for controlling the physical state of generated structures, which is valuable for simulating non-equilibrium processes. The open-sourcing of the code and the use of large public datasets (OMol25, OMat24) will facilitate further research in this area. Zatom-2 introduces a unified atomistic generative model that leverages multitask pretraining on large-scale molecular and material datasets to enable cross-domain transfer to protein generation. By combining a novel atom1 tokenization scheme with conditional flow matching and physics-grounded auxiliary objectives (energy/force prediction), the model demonstrates superior distribution fidelity and, crucially, the ability to transfer geometric representations from small molecules to larger biomolecules, achieving significant gains in protein designability in low-data and out-of-distribution settings.
Author of this paper? Send a correction.