Emergent Vision-Shaped Thinking in Text-Only Reasoning through Multimodal Training
Abstract
Does multimodal training shape how a language model thinks, even when the input is text only? We test this with targeted causal ablations across 10 multimodal configurations from 6 model families, with matched text-only backbones and cross-family transfer controls. We contrast spatial-imagery tasks that invite mental scene construction with symbol-manipulation tasks solvable by symbolic rules alone. We identify a vision-language axis in hidden-state space and ablate its projection at inference. In Chameleon-7B, this ablation drops accuracy on BIG-Bench Hard (BBH) navigate by 27.6 pp while leaving boolean expressions intact (+5.6 pp), a 33.2 pp double dissociation. The effect is direction-specific and layer-localized, extends to mental-simulation tasks such as mental rotation, paper folding, and perspective taking, and is not explained by task difficulty, reasoning depth, or sequential update load. Together, these results reveal emergent vision-shaped thinking (EVT): text-only reasoning that causally recruits representations introduced or strengthened by vision-language training, without assuming an image-like internal format. The measured effect is spatially concentrated at 7-8B and broadens to non-spatial visual attribute binding in the tested 32B configuration. In dual-path unified models (BAGEL, Janus-Pro), the causal handle localizes to the generation pathway as a compact one-dimensional carrier. The understanding pathway also encodes spatial information in a higher-rank, probe-decodable form, but is not the causal handle. To separate decodability from causal use, we analyze concentration, alignment, and semantic location. These diagnostics explain why behavior, probing, and intervention can diverge. Multimodal training thus reshapes the geometry of text-only reasoning, not just how visual input is processed.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper presents a compelling and rigorous causal analysis demonstrating that multimodal training reshapes the internal geometry of text-only reasoning by introducing or amplifying vision-shaped spatial structures that are causally recruited during inference. Through extensive ablation studies across diverse architectures, it identifies specific "vision-language axes" whose removal selectively impairs spatial reasoning tasks while leaving symbolic logic intact, revealing a mechanistic division of labor in unified models where the generation pathway carries a compact causal carrier for spatial structure. This work significantly advances the field of mechanistic interpretability by providing concrete evidence for "emergent vision-shaped thinking" and offering new diagnostics to distinguish between decodable features and causally used representations.
Methodology
The paper introduces a rigorous causal intervention framework to test "Emergent Vision-Shaped Thinking" (EVT). The core method involves identifying a "vision-language axis" in the hidden state space using a difference-in-means estimator between text and visual token representations. The authors then perform projection ablations (removing the component of the hidden state along this axis) during text-only inference. The methodology is strengthened by several controls: matched text-only backbones to distinguish introduced vs. amplified effects, energy-matched random directions to rule out generic damage, and layer-wise sweeps to localize the causal hub. The extension to dual-path models (BAGEL, Janus-Pro) to separate generation and understanding pathways is a particularly strong methodological contribution, revealing that the causal carrier for spatial reasoning resides in the generation pathway as a compact 1D signal, while the understanding pathway holds distributed, non-selective information.
Results & evaluation
The experimental evaluation is extensive, covering 10 multimodal configurations across 6 model families (Chameleon, Qwen3-VL, Liquid, Emu3, BAGEL, Janus-Pro). The use of a double dissociation paradigm (spatial-imagery tasks vs. symbol-manipulation tasks) is well-suited to the hypothesis. The headline result of a 33.2 pp double dissociation in Chameleon-7B is striking. The inclusion of mental simulation tasks (mental rotation, paper folding) and external benchmarks (bAbI, SpartQA) adds robustness. The analysis of "concentration, alignment, and semantic location" provides a deeper mechanistic understanding of why probing and intervention results can diverge. The experiments are not just behavioral but mechanistic, which is a high bar.
Reproducibility
The paper provides detailed descriptions of axis construction, intervention hooks, and evaluation protocols. It mentions releasing analysis code and benchmark scripts. The specific model configurations and task splits are described, though the exact code for the "mental-simulation battery" and specific axis construction for each heterogeneous architecture might require careful implementation to replicate exactly. The reliance on specific model versions (e.g., Chameleon-7B, Qwen3-VL) is a standard limitation.
Limitations
The study is limited to linear/low-rank interventions, potentially missing nonlinear mechanisms. The tasks are diagnostic rather than a complete taxonomy of spatial reasoning. The "introduction vs. amplification" attribution for Qwen models is constrained by uncertainty about pretraining data. The effect is observed in specific model families; generalizability to all future multimodal architectures is not guaranteed. The paper acknowledges that it tests necessity, not the full computational circuit.
Broader impact
This paper has significant implications for the understanding of multimodal LLMs. It challenges the assumption that text-only reasoning in VLMs is purely linguistic, showing that visual-spatial geometry is causally recruited even without visual input. This has implications for model design (e.g., how to structure dual-path models), safety (potential for manipulating spatial reasoning), and interpretability (new diagnostics for causal use vs. decodability). It provides a new lens for evaluating the "thinking" capabilities of multimodal models. The paper presents a compelling and rigorous causal analysis demonstrating that multimodal training reshapes the internal geometry of text-only reasoning by introducing or amplifying vision-shaped spatial structures that are causally recruited during inference. Through extensive ablation studies across diverse architectures, it identifies specific "vision-language axes" whose removal selectively impairs spatial reasoning tasks while leaving symbolic logic intact, revealing a mechanistic division of labor in unified models where the generation pathway carries a compact causal carrier for spatial structure. This work significantly advances the field of mechanistic interpretability by providing concrete evidence for "emergent vision-shaped thinking" and offering new diagnostics to distinguish between decodable features and causally used representations.
Author of this paper? Send a correction.