Language Models Share a Readout Bottleneck in Working Memory
Abstract
Large language models (LLMs) reproduce a human working-memory signature on the N-back task: performance declines as memory load N increases, and errors concentrate on lures, items that repeat the current stimulus at a neighboring but incorrect serial position. We ask whether this shared behavioral signature reflects a shared mechanism. The literature offers two candidate accounts. On a precision account, softmax attention cannot concentrate its weight on the item N positions back, so adjacent items are retrieved instead. On an interference account, attention selects the correct item, but the content of competing items corrupts the decision. We dissociate these accounts in 15 pretrained LLMs from three families (70 million to 14 billion parameters) and in minimal transformers trained from scratch. In all three families, the attention heads that carry the task attend to the target on most missed matches. Linear probes decode near-perfectly whether a matching letter occurred at the target or at a lure position, even on trials where the model false-alarms to the lure, although its own decisions barely distinguish the two. Sharpening attention yields negligible gains, whereas suppressing the letter identity of non-target items improves sensitivity (d') in nearly all model-by-load conditions. In larger models the task is spread over more heads, and removing interference helps more. In minimal transformers, precision limits a single attention layer, but a second layer removes this constraint. The working-memory limit of LLMs therefore lies in reading items out under interference rather than in locating them, consistent with interference accounts of human working memory.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper provides a rigorous mechanistic explanation for LLM working memory failures, demonstrating that the bottleneck lies in reading out items under interference rather than in locating them via attention. By combining theoretical analysis of minimal transformers with extensive empirical interventions on 15 pretrained LLMs, the authors successfully dissociate precision and interference accounts, offering a new perspective on the internal computations of large language models.
Methodology
The paper employs a rigorous mechanistic interpretability framework to dissect the working memory limitations of Large Language Models (LLMs) on the N-back task. The methodology is sophisticated, combining behavioral analysis (signal detection theory, $d'$), representational probing (linear probes on residual streams), and causal interventions (activation patching, attention sharpening, and subspace suppression of distractor content). A key strength is the theoretical grounding provided for minimal transformers, where the authors derive propositions showing that single-layer attention is fundamentally limited by softmax dispersion (precision account), while two-layer architectures can overcome this via direct comparison. This theoretical foundation is then contrasted with empirical findings in large pretrained models, where the bottleneck shifts to interference during readout. The use of activation patching to identify task-critical heads and the specific intervention of suppressing letter-identity directions in non-target items are methodologically sound and directly address the core hypothesis.
Results & evaluation
The experimental scope is impressive, covering 15 pretrained LLMs from three distinct families (Pythia, GPT-2, Qwen2.5) spanning a 200-fold range in parameter size (70M to 14B). Additionally, the authors train 720 minimal transformers from scratch to isolate architectural constraints. The results are consistent across models: attention heads correctly locate the target even on error trials, and linear probes can decode the correct answer from activations even when the model's decision is wrong. The causal intervention results are particularly compelling; suppressing distractor content significantly improves sensitivity ($d'$), whereas sharpening attention yields negligible gains. The inclusion of recurrent baselines (LSTM, LRU, Mamba) further validates that the observed decline is not an artifact of softmax attention alone but relates to content-dependent retrieval mechanisms.
Reproducibility
The paper provides detailed descriptions of the N-back task setup, prompt formatting, and intervention procedures. The use of standard, publicly available models (Pythia, GPT-2, Qwen2.5) enhances reproducibility. The minimal transformer training details (hyperparameters, seeds, architecture variants) are well-documented. However, the specific code for the subspace suppression intervention and the exact implementation of the linear probes are not provided in the text, which may pose a minor barrier for exact replication without access to the authors' codebase.
Limitations
The study is limited to the letter N-back task with synthetic stimuli, which may not fully capture the complexity of real-world working memory tasks. The mechanistic analysis relies on a specific line-by-line prompt format with correct earlier answers shown, which might not generalize to all prompting strategies. The head-level account may not transfer to models larger than 14B, as the identified heads become less critical in larger models. Furthermore, the linear probes only demonstrate that information is decodable, not necessarily that it is the sole driver of behavior, although the causal interventions help mitigate this concern.
Broader impact
This work has significant implications for the field of mechanistic interpretability and the understanding of LLM limitations. It challenges the prevailing "precision" narrative regarding attention failures and highlights "interference" as a key bottleneck. This insight could guide future model architectures and training objectives aimed at improving working memory and reasoning capabilities. The findings also contribute to the broader debate on the convergence of neural network mechanisms and their analogy to human cognitive processes. The paper provides a rigorous mechanistic explanation for LLM working memory failures, demonstrating that the bottleneck lies in reading out items under interference rather than in locating them via attention. By combining theoretical analysis of minimal transformers with extensive empirical interventions on 15 pretrained LLMs, the authors successfully dissociate precision and interference accounts, offering a new perspective on the internal computations of large language models.
Author of this paper? Send a correction.