The Information Geometry of Representation Formation
Abstract
Most analyses of neural representations focus on trained endpoints, leaving the process by which internal structure forms comparatively unmeasured. We instead treat representation formation itself as an object of study, asking whether the path a representation takes through training reveals information that its endpoint does not. We formalize this idea by viewing representation learning as a geometric trajectory, using Bures geometry to measure movement through representation space and quantum Fisher information to characterize local sensitivity to input factors. We find that learning concentrates local sensitivity on task-relevant factors, and that changing only the prediction target reorganizes this geometry around the corresponding visual factor. More strikingly, representation trajectories become systematically less direct as the learned computation requires higher-order interactions among input factors. This effect persists under matched-performance and training-duration controls and transfers from synthetic vector inputs to residual CNNs on rendered and natural images. Importantly, formation history contains predictive information about the stability of the learned representation: controlling for interaction order, path tortuosity predicts its subsequent drift under continued training and sensitivity to parameter perturbations, whereas endpoint displacement does not. Finally, representation-aware optimization lets us actively shape these trajectories, showing that formation geometry is not only measurable but actionable. Together, these results establish representation formation as a new source of information about learning: its geometry reveals properties of the computation being constructed, predicts subsequent representational stability, and provides a target for intervention during optimization.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper establishes representation formation as a measurable geometric object that reveals computational structure and predicts stability. By linking Bures geometry and Quantum Fisher Information to training trajectories, it provides a novel framework that not only diagnoses the complexity of learned computations but also enables targeted interventions to shape the learning path.
Methodology
The paper proposes a rigorous information-geometric framework for analyzing representation formation during training. It defines the representation state as the trace-normalized covariance of hidden activations and utilizes Bures distance to measure the trajectory of this state through training. The local geometry is characterized by Quantum Fisher Information (QFI), which identifies input factors to which the representation is locally sensitive. The authors derive a "Representation Natural Gradient" (RepNG) by pulling back the Bures/QFI metric to parameter space, allowing for optimization that explicitly constrains representational movement. The theoretical connection between the local QFI metric and the global Bures trajectory is well-established, providing a unified geometric language for both local sensitivity and global path tortuosity.
Results & evaluation
The experiments are extensive and well-controlled. The authors validate the local geometry (QFI) on synthetic tasks and dSprites, showing it correctly identifies task-relevant factors. The core empirical finding—that higher-order computational interactions lead to higher path tortuosity—is tested across MLPs and Residual CNNs, using both synthetic and natural images (CIFAR-10 composites). Crucially, the paper includes matched-performance and matched-training-duration controls to rule out trivial explanations. The predictive power of tortuosity for subsequent representational stability (drift and parameter sensitivity) is demonstrated in a fresh 30-seed sweep. The intervention study shows RepNG reduces tortuosity compared to standard optimizers at matched performance.
Reproducibility
The paper provides a detailed reproducibility statement, including specific hyperparameters, seed counts, and code availability. The GitHub repository link is provided. The experimental protocols are described in sufficient detail in the appendices to allow for replication, including the specific construction of the parameter subspaces for the optimizer comparison.
Limitations
The primary limitation is scale. The experiments are conducted on small networks (MLPs and small ResNets) and relatively simple tasks (parity, sign patterns, dSprites factors). It remains an open question whether these geometric properties hold for large-scale foundation models or complex natural language tasks. Additionally, the RepNG method requires estimating the pullback metric, which may be computationally expensive for very large networks, though the paper demonstrates it in a 24-dimensional subspace.
Broader impact
This work shifts the focus of representation analysis from static endpoints to dynamic formation processes. It provides a new metric (tortuosity) that correlates with computational complexity and stability, offering a new diagnostic tool for understanding learning dynamics. The RepNG optimizer offers a practical method for stabilizing representations during training, which could be valuable in continual learning and transfer learning scenarios. The paper establishes representation formation as a measurable geometric object that reveals computational structure and predicts stability. By linking Bures geometry and Quantum Fisher Information to training trajectories, it provides a novel framework that not only diagnoses the complexity of learned computations but also enables targeted interventions to shape the learning path.
Author of this paper? Send a correction.