Scaling Interpretable Transformers with Parity Bottleneck Layers
This research, published on arXiv by authors including Andrew Mack and colleagues from unspecified institutions, introduces the ParityTransformer, a GPT-2 scale architecture designed to improve the interpretability of large language models by construction. The core contribution is the development of a "Deep Parity Bottleneck" (DPB) mechanism that replaces the computationally expensive, over-complete learned bottlenecks typically used with sparse autoencoders (SAEs) for post-hoc interpretability. The problem this work addresses is the prohibitive cost of per-layer interpretability in large Transformer models, which stems from the need for wide, sparse representations that are difficult to train efficiently. The ParityTransformer aims to bridge this gap by integrating interpretability directly into the model's forward pass.
Two pivotal technical ideas underpin the ParityTransformer. First, the Deep Parity Bottleneck (DPB) employs a parameter-free algebraic dictionary and a hierarchical mixture-of-experts approach to enforce sparsity efficiently. This design provides a deterministic incoherence guarantee, meaning features are guaranteed to be distinct, and crucially, eliminates the memory overhead associated with learned over-complete bases. Second, by ensuring that subsequent computations only act on features that have passed through this sparse bottleneck, the ParityTransformer ensures that its learned features are intrinsically utilized by the model during its forward pass. This directly tackles the question of whether post-hoc interpretability methods like SAEs recover features that the model actually uses. Empirically, the authors demonstrate that ParityTransformers match or exceed the performance of post-hoc SAEs on sparse probing tasks and offer superior performance in feature absorption, steering effectiveness, and causal interventions. This work is intended for researchers and software engineers working on Transformer architectures, especially those focused on model interpretability and understanding the internal mechanisms of large language models.
The ParityTransformer's success in integrating interpretability by design, rather than as an afterthought, holds significant implications. It enables the training of more interpretable large-scale models with a manageable "interpretability tax," potentially making internal model workings more accessible for debugging, verification, and fine-grained control. This approach could influence the development of future LLMs, shifting the paradigm towards inherently interpretable architectures and fostering greater trust and understanding in these powerful systems. The abstract does not contain the full paper content.