Language Models Represent and Transform Concepts with Shared Geometry
Zhimin Hu, Lanhao Niu, and Sashank Varma
Mechanistic Interpretability Workshop at ICML 2026, 2026
TL;DR: This work characterizes the geometry of how context structures concept spaces across language models, showing it is semantically organized and shared across models.
@article{hu2026representational,title={Language Models Represent and Transform Concepts with Shared Geometry},author={Hu, Zhimin and Niu, Lanhao and Varma, Sashank},journal={Mechanistic Interpretability Workshop at ICML 2026},year={2026},}
CogSci 2026
The Representational Geometry of Number
Zhimin Hu, Lanhao Niu, and Sashank Varma
CogSci, 2026
TL;DR: This work finds that language models preserve relational structure between concepts to support generalization, while projecting them into distinct subspaces to support multitasking, using number as a testbed.
@article{hu2026representationam,title={The Representational Geometry of Number},author={Hu, Zhimin and Niu, Lanhao and Varma, Sashank},journal={CogSci},year={2026},}
Language Models Represent and Transform Concepts with Shared GeometryMechanistic Interpretability Workshop at ICML 2026, 2026TL;DR: This work characterizes the geometry of how context structures concept spaces across language models, showing it is semantically organized and shared across models.