Anthropic researchers have identified a 'global workspace' in large language models (LLMs), where information is integrated across layers, analogous to the Global Neuronal Workspace theory of consciousness. This finding provides a new framework for understanding how LLMs combine information across layers, potentially leading to improved model interpretability and alignment. The work defines a 'J-space' based on the expected change to final logits from perturbations at each layer, revealing a shared subspace across contexts.
Background
Global Workspace Theory (GWT) is a leading scientific account of consciousness, proposing that information becomes globally available for reasoning and action when broadcast across a neural workspace. In transformer-based LLMs, information flows through multiple layers of self-attention and feedforward networks, allowing integration across tokens. This research adapts GWT to artificial neural networks, suggesting a similar integrative mechanism exists in LLMs.
References
Discussion
Community comments show mixed reactions: some appreciate the conceptual connection to consciousness, while others (like snaking0776) caution against overstating the analogy, noting that the J-space is more about a shared reasoning subspace. Several commenters recall related experiments, such as duplicating layers to improve math ability, and Neel Nanda's independent replication.