Machine Learning PapersLandmark papers
← All papers
RESEARCH PAPER / 2026-11-14

Nahual: A Sequence Model for Language and Atoms

Austin H. Cheng, Marcel Müller, Andreas Burger, Yeonghun Kang, Tsz Wai Ko, Marta Skreta, Luka Mucko, Ella Miray Rajaonson, Cher Tian Ser, Jérôme F. Gonthier, Alex Zook, Varinia Bernales, Alán Aspuru-Guzik

Matterlab · arXiv

Affiliations: Matterlab

Abstract

Generative models play a rapidly growing role in the design and simulation of molecules. The most widely used applications of generative models are in large language models (LLMs), owing to the natural flexibility of language for specifying and answering queries. However, generalist LLMs do not effectively handle large sets of 3D coordinates, whereas specialist generative models of 3D molecules cannot reach the controllability afforded by language. To bridge these capabilities, we propose Nahual, a decoder-only autoregressive diffusion model for any sequence of text and 3D molecules. To enable expressive autoregression on continuous coordinates, we propose next-token diffusion, whereby a causal transformer denoises each token while attending to the clean tokens before it, without increasing context size. We train Nahual on a curated set of twelve 3D chemistry tasks ranging from microsolvation to molecular conformer search to adsorption on metal surfaces, with associated evaluation metrics and baselines. The result is a single set of end-to-end-trained weights that simultaneously performs all tasks and surpasses state-of-the-art models specialized for crystal structure prediction and property-conditional molecule generation. Our results demonstrate the viability of scaling generalist decoder-only models for native multimodal understanding and generation across language and chemistry. Code is available at https://github.com/the-matter-lab/nahual.

The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.

Research summary

The paper introduces Nahual, a unified autoregressive diffusion model for joint language and 3D molecular generation. By proposing next-token diffusion, it effectively bridges the modality gap between discrete text and continuous 3D coordinates, demonstrating state-of-the-art performance across a diverse suite of chemical tasks with a single set of weights.

Methodology

The paper proposes "Nahual," a decoder-only autoregressive diffusion model designed to handle mixed sequences of text and 3D molecular coordinates. The core technical contribution is "next-token diffusion," a mechanism where a causal transformer denoises each continuous token (3D coordinates) while attending to clean preceding tokens. This approach aims to bridge the gap between the flexibility of LLMs and the precision required for 3D structural data, avoiding the context size explosion that would occur if 3D structures were tokenized into discrete high-dimensional vectors. The method is trained end-to-end on a curated set of twelve 3D chemistry tasks, ranging from microsolvation to adsorption on metal surfaces.

Results & evaluation

The evaluation is extensive, covering twelve distinct 3D chemistry tasks. The paper claims that the single set of weights surpasses state-of-the-art specialized models for crystal structure prediction and property-conditional molecule generation. The breadth of tasks (microsolvation, conformer search, adsorption) suggests a robust generalist capability. However, without access to the full experimental tables and baselines in the provided text fragment, the specific magnitude of improvement and the rigor of the comparison against specialized baselines must be inferred from the abstract's strong claims.

Reproducibility

The authors provide a GitHub repository (https://github.com/the-matter-lab/nahual), which is a strong indicator of reproducibility. The use of a "curated set" of tasks implies that data processing pipelines are likely shared. The architectural details of "next-token diffusion" are described sufficiently in the abstract to understand the high-level approach, though specific hyperparameters and training details would require the full paper or code inspection.

Limitations

The primary limitation is the computational cost associated with diffusion models, even with the autoregressive optimization. The model is specialized for chemistry; its generalizability to other 3D domains (e.g., protein folding, materials science beyond chemistry) is not explicitly demonstrated. The "curated" nature of the training data may limit the model's exposure to the full diversity of chemical space.

Broader impact

This work has significant potential impact on computational chemistry and drug discovery by providing a unified framework for language-guided molecular design. It moves the field towards generalist AI models that can understand and generate complex 3D structures, potentially reducing the need for task-specific models. The approach could be extended to other scientific domains requiring 3D coordinate generation. The paper introduces Nahual, a unified autoregressive diffusion model for joint language and 3D molecular generation. By proposing next-token diffusion, it effectively bridges the modality gap between discrete text and continuous 3D coordinates, demonstrating state-of-the-art performance across a diverse suite of chemical tasks with a single set of weights.

Author of this paper? Send a correction.