The Vocabulary Head Is the Hidden Tax on All Reasoning
Why it earned a slot
'A Model with No Head and Many Thoughts' lands a quiet bombshell: projecting hidden states through a large vocabulary head at every single token is computationally dominant, and it forces all reasoning to be expressed in discrete tokens. This connects directly to the CoT compression work (Every Token Leaves a Ripple) and the embedding space analysis (When Can We Work in Embedding Space). Together they reveal a representational bottleneck we've been ignoring while chasing parameter counts. The head isn't just an efficiency problem—it shapes what kinds of thought can even be expressed. Continuous reasoning in hidden space, compressed CoT, and embedding-based computation are all attempts to bypass a bottleneck that's been there since the transformer architecture was chosen. The question none of these papers fully answers is whether we can reason in a representation that isn't forced through a 50k-token vocabulary at every step, or whether that discrete constraint is actually a feature, not a bug.
Brendon Score: 9.2/10
- Quality: 9.0/10 — base
- Authority: 5.0/10 — +0.00
- Freshness: 8.4/10 — +0.17
- Relevance: 8.0/10 — +0.00
- Sum: 9.17
- Total (rounded): 9.2/10
Why this is here
Checks cleared: topic-dedup, title-form, publishable-prose.
First seen: .