The Distinction Between Decodability and Detachability

This research addresses a fundamental limitation in current LLM architecture: the difference between a model's ability to recall information (decodability) and its ability to isolate specific knowledge components for modification or removal (detachability). While standard training methods successfully encode vast amounts of data, they often result in highly entangled parameter states. The authors demonstrate that even when a model can accurately retrieve or 'decode' a specific fact, that fact is frequently distributed across non-contiguous parameter clusters, making it impossible to surgically remove or update without degrading overall model performance.

Data Granularity as the Driver of Modularity

The core finding is that parametric modularity is not an emergent property of scale, but a direct consequence of training data granularity. When models are trained on coarse-grained, heterogeneous datasets, knowledge becomes deeply entangled. Conversely, when training data is structured with higher granularity—where specific concepts, domains, or tasks are presented in distinct, self-contained sequences—the model develops more modular internal representations.

This structural alignment between input data and internal parameter organization suggests that developers can influence the 'editability' of a model by curating training pipelines that prioritize conceptual separation. The research implies that future efforts to improve model reliability and safety should focus on data-centric engineering rather than relying solely on post-hoc fine-tuning or model editing techniques, which often fail because the underlying knowledge is not truly detachable.