Every Measurement, Every Direction, All at Once: Multimodal Flow Matching for Molecules and Spectra
Abstract
Generative modeling in science is often data-limited, where experiments are expensive. However, each experiment may yield several coupled measurements of the same physical entity. Standard generative models often train only on a single direction, discarding supervision from the rest. We argue that each experiment contains more training signal than this asymmetry allows the model to exploit: random masking enables measurements to serve as either conditioning inputs or prediction targets, yielding multiple supervised denoising tasks from a single experiment. We investigate this premise with MOSAIC, a multimodal flow matching transformer over molecular 3D structure and UV, IR, and Raman spectra. MOSAIC is a single model that is capable of any-to-any generation: generating any subset of modalities from any disjoint subset of the others. On QM9S, MOSAIC achieves 88.8% molecular graph recovery, outperforming the strongest spectra-to-structure baseline by 23.2% in structure generation, while simultaneously outperforming a structure-to-spectra specialist model in spectral generation. Ablations show that the any-to-any objective improves structure generation over structure-only training across model sizes. Furthermore, MOSAIC supports arbitrary subsets of available measurements and enables molecular identification by matching generated spectra to experimental references.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
MOSAIC introduces a random masking strategy for multimodal flow matching that enables any-to-any generation across molecular structures and spectra. The paper presents a rigorous and novel approach to maximizing data utility in scientific generative modeling, demonstrating strong performance on the QM9S benchmark and offering a scalable framework for multimodal scientific discovery.
Methodology
The paper introduces MOSAIC, a multimodal flow matching transformer designed for "any-to-any" generation across molecular 3D structures and multiple spectral modalities (UV, IR, Raman). The core methodological contribution is the use of random masking during training, which allows any subset of modalities to serve as conditioning inputs while the remaining modalities serve as prediction targets. This approach effectively creates multiple supervised denoising tasks from a single experimental data point, addressing the data-limited nature of scientific generative modeling. The use of flow matching is a modern and robust choice for continuous data generation, and the transformer architecture is well-suited for handling the heterogeneous nature of the inputs (graph-based structures vs. 1D spectra). The methodology is sound and logically addresses the stated problem of underutilized supervision signals in multimodal scientific data.
Results & evaluation
The experiments are conducted on the QM9S dataset, a standard benchmark for molecular property prediction and generation. The paper reports an 88.8% molecular graph recovery rate, claiming a 23.2% improvement over the strongest spectra-to-structure baseline. It also demonstrates that the model outperforms specialist models in spectral generation. Ablation studies confirm that the any-to-any objective improves structure generation compared to structure-only training. While the results are strong, the evaluation is limited to a single dataset (QM9S), which may not fully capture the complexity of real-world, larger-scale molecular systems. The comparison to "specialist models" is promising but would benefit from more detailed baseline descriptions to ensure fair comparison.
Reproducibility
The paper is 30 pages long with 12 figures, suggesting a detailed presentation. However, the provided text does not include explicit links to code or data repositories. The mention of "Lila Sciences" as the funding and employment source suggests an industrial context, which may limit immediate open-source availability. Without access to the code, hyperparameters, and specific implementation details of the flow matching loss and masking strategy, full reproducibility is difficult to assess. The abstract-only score of 60 suggests the full text likely contains sufficient detail, but the lack of a visible project URL in the extracted text is a minor negative.
Limitations
The primary limitation is the evaluation on a single dataset (QM9S). Generalization to larger, more complex molecular datasets (e.g., ChEMBL, ZINC) is not demonstrated. The "any-to-any" capability, while novel, may introduce complexity in inference and conditioning that is not fully explored. The paper does not discuss the computational cost of training a single model for multiple tasks versus training separate specialist models, which is a practical concern for researchers with limited resources.
Broader impact
This work has significant potential impact on scientific discovery, particularly in chemistry and materials science, where experimental data is expensive and multimodal. By enabling a single model to generate and predict across multiple measurement types, it could streamline the design of experiments and the interpretation of results. The approach of leveraging random masking to maximize data utility is a generalizable technique that could be applied to other multimodal scientific domains, such as medical imaging (combining MRI, CT, and PET) or climate science. MOSAIC introduces a random masking strategy for multimodal flow matching that enables any-to-any generation across molecular structures and spectra. The paper presents a rigorous and novel approach to maximizing data utility in scientific generative modeling, demonstrating strong performance on the QM9S benchmark and offering a scalable framework for multimodal scientific discovery.
Author of this paper? Send a correction.