Parts-of-Speech as Emergent Categories in SAE Latent Space
Abstract
Sparse AutoEncoders (SAEs) offer a promising way to inspect language model representations, but it is still unclear what kind of linguistic structure their latents expose. We use part-of-speech (PoS) categories as a controlled test case to study whether morpho-syntactic information is encoded by individual latents or by structured groups of features. We find that PoS distinctions are highly recoverable from SAE activations, but do not align with one-to-one latent / category mappings. This recoverability is not reducible to lexical memorisation, and Open and Closed PoS classes differ substantially. Categories are supported by compact groups of sparse latents, with substantial variation across tags. These groups remain stable on held-out data, while also showing overlap between related categories. Our results show that SAEs localise morpho-syntactic information in a distributed and category-dependent form rather than through atomic grammatical features.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper demonstrates that Part-of-Speech categories in LLMs are encoded as distributed but compact groups of SAE latents, with significant structural differences between open and closed classes. By combining probing, feature salience, and coverage analysis, the authors provide a detailed map of how morpho-syntactic information is organized in sparse latent spaces, offering valuable insights into the interpretability of modern language models.
Methodology
The paper employs a rigorous three-step interpretability pipeline to analyze Part-of-Speech (PoS) encoding in Sparse Autoencoder (SAE) latents. First, it establishes recoverability using L1-regularized logistic regression probes on SAE activations from LLaMA-3-8B. Second, it moves beyond binary accuracy to localize information by analyzing feature salience (classifier coefficients) and coverage (the minimal number of latents required to cover 95% of instances for a given PoS tag). Third, it validates these localized groups on held-out data and a controlled synthetic dataset to test stability and additivity. The methodology is sound, moving logically from "is the information there?" to "where is it?" and "is it stable?". The use of a controlled dataset with minimal templates is a strong methodological choice to isolate lexical effects from syntactic ones.
Results & evaluation
The experiments are comprehensive, utilizing the GUM Treebank for natural data and a custom controlled dataset for validation. The results clearly demonstrate that PoS information is distributed across compact groups of latents rather than single monosemantic units. The distinction between Open-class (nouns, verbs) and Closed-class (determiners, conjunctions) PoS is well-supported, with closed classes showing more compact and stable latent groups. The ablation studies, including random label controls and layer-wise analysis, effectively rule out trivial explanations like lexical memorization or layer-specific artifacts. The finding that a small union of these latents (498 out of 131,072) supports strong multi-class classification is a significant quantitative result.
Reproducibility
The authors provide high reproducibility by releasing code, data, and the specific SAE checkpoint used. The GitHub repository contains the pipeline for extracting activations and running the probes. The use of standard libraries (Scikit-learn, Sparsify) and public models (LLaMA-3-8B, EleutherAI SAE) ensures that other researchers can easily replicate the findings. The detailed description of the subword-to-token alignment strategy (leftmost subword anchoring) is crucial for reproducibility in this domain.
Limitations
The study is limited to a single model (LLaMA-3-8B) and a single SAE variant, which may limit the generalizability of the findings to other architectures or sparsity regimes. The analysis is restricted to English, and it is unclear how these patterns would manifest in morphologically rich languages. The controlled dataset, while useful, is small and covers only a subset of syntactic structures. Additionally, the reliance on linear probes means that non-linear interactions between latents are not captured, potentially underestimating the complexity of the representation.
Broader impact
This work contributes to the broader field of mechanistic interpretability by providing a structured framework for analyzing how linguistic categories are represented in sparse latent spaces. It challenges the assumption of strict monosemanticity for linguistic features, suggesting instead a distributed but localized organization. This has implications for how interpretability tools are designed and how we understand the internal representations of LLMs. The findings could inform the development of more targeted interpretability methods that focus on groups of features rather than individual units. The paper demonstrates that Part-of-Speech categories in LLMs are encoded as distributed but compact groups of SAE latents, with significant structural differences between open and closed classes. By combining probing, feature salience, and coverage analysis, the authors provide a detailed map of how morpho-syntactic information is organized in sparse latent spaces, offering valuable insights into the interpretability of modern language models.
Author of this paper? Send a correction.