ISLA: A Coordinate-Free, Measure-Weighted Transformer for Boundary-Driven PDEs
Abstract
A foundational principle of physics is that all coordinate systems are equally valid: the behavior of a PDE-governed system does not depend on the origin or orientation of the frame, nor on the units or mesh used to write it down, and neither should a surrogate model. Yet the neural surrogates with the lowest errors on today's aerodynamics benchmarks lack these invariances and fail under simple transformations of the same problem: for models trained on PDEs of cars in one pose (i.e., translation and rotation) and from one mesher, rotating the full PDE raises the surface-pressure error from 4% to more than 100%, and re-sampling its boundary raises it to about 30%. We introduce ISLA (Invariant Slice Attention), a slice-attention transformer for steady, boundary-driven PDEs in which no learned operation receives a coordinate and every sum over the boundary is a quadrature. Positions enter only as distances and relative angles to learned slice anchors, lengths are measured in a unit computed from the boundary, and interior points query the boundary through a tree-accelerated learned kernel without influencing one another. With every boundary cell a token and an exact boundary read, ISLA is equivariant to rigid motions and reflections, invariant to the units of its declared data, and consistent under mesh refinement, while at every setting each query is independent of all other queries. On AeroZoo, 2,816 flows that vary in shape, size, speed, and boundary conditions, its mean reference-scaled velocity and pressure errors are at least 3.1 and 3.3 times lower than any baseline's. On AirfRANS its errors are second only to those of a boundary-integral model (GLOBE) and at least 1.6 times lower than those of Transolver and GeoTransolver.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents ISLA, a coordinate-free transformer for boundary-driven PDEs that achieves significant error reductions and robust invariance properties. By eliminating explicit coordinate inputs in favor of geometric invariants and quadrature-based sums, the method addresses a fundamental limitation in current neural surrogates, offering a principled approach to equivariant learning that is validated on large-scale aerodynamic benchmarks.
Methodology
The paper introduces ISLA (Invariant Slice Attention), a transformer architecture designed specifically for boundary-driven PDEs. The core innovation is the removal of explicit coordinate inputs, replacing them with geometric invariants (distances to learned slice anchors, relative angles) and measure-weighted sums (quadrature) over boundary cells. This design ensures equivariance to rigid motions and reflections, invariance to units, and consistency under mesh refinement. The use of a tree-accelerated learned kernel for interior-boundary interactions is a significant architectural choice that balances computational efficiency with the need for global context without direct interior-interior interactions. The theoretical grounding in coordinate-free principles is strong and addresses a known weakness in current neural surrogate models.
Results & evaluation
The evaluation is conducted on two major aerodynamics benchmarks: AeroZoo (2,816 flows) and AirfRANS. The results demonstrate substantial improvements over baselines, with mean reference-scaled velocity and pressure errors reduced by at least 3.1x and 3.3x on AeroZoo compared to any baseline. On AirfRANS, ISLA performs second only to a specialized boundary-integral model (GLOBE) and outperforms general-purpose transformers like Transolver. The inclusion of tests for invariance (rotation, translation, re-sampling) directly validates the paper's central claims, showing that while baselines fail catastrophically under these transformations, ISLA maintains performance.
Reproducibility
The paper is submitted to ICML 2026, implying a high standard for reproducibility. However, the provided text is a template/summary and does not include the specific hyperparameters, code links, or detailed implementation notes typically found in the full appendix. The mention of "tree-accelerated" kernels suggests reliance on specific geometric data structures, which may require careful implementation. Without access to the code repository (not provided in the text), full reproducibility is difficult to assess, but the methodological clarity is high.
Limitations
The method is tailored for steady, boundary-driven PDEs. Its applicability to time-dependent problems or PDEs with significant interior sources/sinks is not discussed. The reliance on boundary cells as tokens may limit scalability for problems with extremely complex or high-resolution boundaries. The comparison to GLOBE, which is a boundary-integral model, suggests that for pure boundary-value problems, specialized integral methods may still hold an edge, positioning ISLA as a strong general-purpose alternative rather than the absolute state-of-the-art for all cases.
Broader impact
This work has significant implications for the field of scientific machine learning, particularly in engineering applications like aerodynamics where coordinate invariance is physically meaningful but often ignored by data-driven models. By providing a framework that respects the symmetries of the underlying physics, ISLA could become a standard component in surrogate modeling pipelines, reducing the need for data augmentation and improving generalization across different problem setups. The paper presents ISLA, a coordinate-free transformer for boundary-driven PDEs that achieves significant error reductions and robust invariance properties. By eliminating explicit coordinate inputs in favor of geometric invariants and quadrature-based sums, the method addresses a fundamental limitation in current neural surrogates, offering a principled approach to equivariant learning that is validated on large-scale aerodynamic benchmarks.
Author of this paper? Send a correction.