Machine Learning PapersLandmark papers
← All papers
RESEARCH PAPER / 2026-11-13

Elasticy: A Runtime-Reconfigurable Architecture for Dynamic Streams

Konstantin J Hossfeld, Nathan Sobotka, Gina Sohn, Tun-Yu Chang, Kunle Olukotun

Stanford University · arXiv

Affiliations: Stanford University

Abstract

Reconfigurable Dataflow Accelerators (RDAs) have demonstrated high performance and energy efficiency for dense, regular tensor applications. However, conventional RDAs with static configurations show dramatically reduced performance and efficiency on irregular tensor workloads with data-dependent, non-uniform, dynamic tensor shapes. We propose Elasticy, a dynamically reconfigurable dataflow accelerator that employs streaming and tile abstractions to minimize the overhead of dynamic support. By encoding dynamic shapes into streams using rank-variable expressions, Elasticy exploits data-dependent tensor shape dynamism to minimize off-chip data traffic and wasted compute cycles. Elasticy employs stream-embedded reconfiguration information to trigger fine-grained runtime scheduling decisions. Elasticy also introduces a novel distributed linked streaming memory (DLSM) that more effectively uses on-chip memory to minimize off-chip data traffic and support dynamically sized tensors. Both reconfiguration and shape dynamism incur minimal overhead in Elasticy: reconfiguration occurs infrequently at the granularity of a stream of tiles, while rank-variable expressions enable lightweight handling of dynamic dimensions. On a Llama 3-8B decode layer, Elasticy achieves 92.6% of its roofline performance, compared to 69.7% on an H100. On evaluated GEMMs, the DLSM reduces total off-chip traffic by up to 61.4% relative to the best evaluated cache configuration, with no additional burden of programmer-managed memory. Elasticy's distributed runtime scheduler overlaps dependent kernels, speeding up kernel pairs by up to 1.42x. Programs can also run in both space and time: each phase of a layer's graph is pipelined spatially, while phases fold onto the same fabric over time. Compared to a traditional spatial accelerator, Elasticy achieves a 6.34x-31.67x improved area delay product on full layers.

The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.

Research summary

Elasticy introduces a runtime-reconfigurable dataflow accelerator with a novel distributed linked streaming memory to efficiently handle dynamic tensor shapes in LLMs. The paper demonstrates that by encoding dynamic shapes into streams and using fine-grained runtime scheduling, it is possible to achieve high roofline performance and reduced off-chip traffic compared to both static RDAs and general-purpose GPUs, offering a promising path for efficient, dynamic AI inference hardware.

Methodology

The paper proposes Elasticy, a runtime-reconfigurable dataflow accelerator (RDA) designed to handle irregular, dynamic tensor shapes common in modern LLMs (e.g., MoE routing, variable sequence lengths). The core innovation lies in the "Distributed Linked Streaming Memory" (DLSM), which replaces traditional scratchpads or caches with a linked-list structure that allows dynamic allocation and spilling to neighbors, significantly reducing off-chip traffic. The architecture employs a stream-of-tiles execution model where dynamic shapes are encoded via rank-variable expressions, allowing the hardware to process ragged tensors without padding. A distributed scheduler manages reconfiguration at the granularity of streams, enabling fine-grained temporal overlap of kernels. The methodology is rigorous, detailing the compiler flow from a Streaming Tensor Program (STeP) IR to the hardware ISA, including functional remapping, packing, and placement strategies.

Results & evaluation

The evaluation is conducted using a cycle-accurate Bluespec SystemVerilog simulator, which provides high fidelity for architectural studies. The authors compare Elasticy against a static RDA baseline and an NVIDIA H100 GPU. Key results include achieving 92.6% of roofline performance on a Llama 3-8B decode layer, compared to 69.7% on the H100. The DLSM reduces off-chip traffic by up to 61.4% relative to the best cache configuration. The paper also demonstrates speedups in kernel overlap (up to 1.42x) and improved area-delay product (ADP) compared to static spatial accelerators. The inclusion of real-world LLM workloads (Llama, Qwen, Mamba) strengthens the relevance of the results.

Reproducibility

The paper provides detailed descriptions of the hardware architecture, compiler pipeline, and simulation setup. However, as is typical for high-level architectural papers, the full Bluespec SystemVerilog code and compiler source are not provided in the text. The evaluation relies on a simulator, which is standard for this field, but the lack of open-source code limits immediate reproducibility for external researchers. The specific parameters for the H100 baseline (vLLM version, kernel selection) are well-documented, aiding in the validation of the software baseline.

Limitations

The primary limitation is that the evaluation is simulation-based, not on physical silicon. While cycle-accurate simulation is rigorous, it may not capture all power and thermal effects of a real implementation. Additionally, the paper focuses on dense-matrix applications with coarse-grained dynamism; it may not be optimal for highly irregular, fine-grained sparse workloads. The area overhead, while described as minimal, is still a consideration for practical deployment compared to static architectures.

Broader impact

This work contributes significantly to the field of ML hardware by addressing the growing mismatch between static accelerator designs and dynamic AI workloads. The proposed DLSM and runtime reconfiguration techniques could influence future accelerator designs, particularly for edge devices or specialized inference chips where energy efficiency and dynamic workload handling are critical. It provides a viable alternative to GPU-like architectures for specific LLM inference patterns. Elasticy introduces a runtime-reconfigurable dataflow accelerator with a novel distributed linked streaming memory to efficiently handle dynamic tensor shapes in LLMs. The paper demonstrates that by encoding dynamic shapes into streams and using fine-grained runtime scheduling, it is possible to achieve high roofline performance and reduced off-chip traffic compared to both static RDAs and general-purpose GPUs, offering a promising path for efficient, dynamic AI inference hardware.

Author of this paper? Send a correction.