Massive Activations Confound Sparse Autoencoders
Abstract
Sparse autoencoders (SAEs) are commonly evaluated by variance explained and ablation damage. We show how massive activations undermine both evaluations, and propose a correction. Massive activations cause up to 99.9% of activation variance to lie along the first principal component (PC1). Because the PC1 component has no unique sparse decomposition, each SAE run rebuilds it from a different set of features. These features dominate importance rankings: an average of 47 of the 50 most important features are unmatched in a separate run, compared with about 27 of a random 50. A dictionary can also report 99.96% of variance explained while leaving 33% of the variance outside PC1 unexplained. We bypass PC1 by training the SAE on activations with PC1 removed, then adding PC1 to the reconstruction. In affected models, the top-50 feature sharing rate rises from 4-39.5% to 73-99.5%. At the same TopK sparsity, the resulting SAE degrades the model up to 44% less and reduces content reconstruction error by up to 33.9%.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper identifies a critical confound in Sparse Autoencoder evaluation caused by massive activations and proposes a simple, effective correction (PC1 bypass) that significantly improves feature reproducibility and interpretability. This is a high-impact contribution that addresses a fundamental issue in the current state of the art, with rigorous experimental validation across diverse models and practical implications for future research in mechanistic interpretability.
Methodology
The paper identifies a critical flaw in the evaluation of Sparse Autoencoders (SAEs): the dominance of "massive activations" (specifically the first principal component, PC1) confounds both variance explained metrics and feature importance rankings. The authors demonstrate that PC1 accounts for up to 99.9% of activation variance in many models, leading to non-reproducible "important" features that are merely different decompositions of PC1. The proposed solution, "PC1 bypass," involves removing the PC1 projection from activations before SAE training and adding it back during reconstruction. This is a methodologically sound and rigorous approach that directly addresses the root cause of the instability. The inclusion of alternative fixes (magnitude penalties, post-hoc selection) provides a comprehensive toolkit for practitioners.
Results & evaluation
The experimental evaluation is extensive, covering a wide range of models (GPT-2, Pythia, Qwen, Llama, Gemma) and architectures (including Vision Transformers). The paper provides strong quantitative evidence for the confound (e.g., 47/50 top features unmatched across seeds) and the efficacy of the fix (sharing rates rising to 73-99.5%). The ablation studies comparing standard SAEs, PC1-bypass SAEs, and magnitude-penalized SAEs are thorough. The evaluation of interpretability (AUROC) and behavioral interventions (subject-verb agreement) adds practical value, showing that the corrected features are not just more stable but also more useful.
Reproducibility
The paper is highly reproducible. Code and trained SAE dictionaries are provided on GitHub and HuggingFace. The training hyperparameters, metrics definitions, and experimental setups are detailed in the main text and appendices. The use of standard datasets (WikiText-103) and public model weights ensures that other researchers can easily replicate the findings.
Limitations
The primary limitation is the computational cost of retraining SAEs with PC1 bypass, although the paper offers a post-hoc selection method as a cheaper alternative for existing dictionaries. The method assumes that PC1 is the primary source of massive activations; while this holds for the tested models, other forms of anisotropy might require more complex corrections. The paper focuses on residual stream activations; it is unclear if the same confound exists in other internal representations (e.g., attention heads) without further investigation.
Broader impact
This paper has significant implications for the field of mechanistic interpretability. By showing that standard SAE evaluations are fundamentally flawed in the presence of massive activations, it invalidates many prior conclusions about feature importance and reproducibility. The proposed correction is likely to become a standard preprocessing step for SAE training and evaluation, similar to how centering/normalization is standard. This work will force the community to re-evaluate existing SAE-based findings and adopt more robust evaluation metrics, thereby improving the rigor of the entire subfield. The paper identifies a critical confound in Sparse Autoencoder evaluation caused by massive activations and proposes a simple, effective correction (PC1 bypass) that significantly improves feature reproducibility and interpretability. This is a high-impact contribution that addresses a fundamental issue in the current state of the art, with rigorous experimental validation across diverse models and practical implications for future research in mechanistic interpretability.
Author of this paper? Send a correction.