📑 arXiv 2d ago
Disentangling Mathematical Reasoning in LLMs: A Methodological Investigation of Internal Mechanisms
Investigation of LLM arithmetic reveals models recognize tasks early but generate correct results only in final layers, with proficient models exhibiting clear division of labor: attention modules propagate input information while MLP modules aggregate it. This attention-MLP specialization is absent in less capable models, traced via early decoding across layers.