Research analysis · NeuroAI encoding models

Brain-aligned LLMs need 1 to 3 percent of dimensions

How much of a language model do you need to predict a listening brain? Zach Paris and Emily Finn show that a low-rank slice - 1 to 3 percent of the embedding dimensions, saturating at a mean rank of about 20 in one dataset - recovers the full linear encoding performance that thousands of dimensions achieve, across intracranial and fMRI recordings, and that this subspace transfers across people. Alignment between artificial and biological representations is far more compressible than the field assumed, and that is good and bad news at once.

Source: A Low-Rank Space Captures Linear Alignment Between LLM and Human Neural Activity During Naturalistic Language Listening, arXiv preprint 2610.40511, 2026. Primary source. Read the full preprint HTML including methods and appendices.

What the work claims

This is a primary analysis paper combining encoding-model methodology with a generalisation study, and it arrives with unusually complete transparency: all datasets are public, and the authors release code with a reproduction guide. The central claim: linear prediction of human neural responses during naturalistic language listening from LLM embeddings is supported by a very small, structured subset of those embeddings - 1 to 3 percent of the embedding dimensionality, saturating at tens of dimensions (mean rank around 20 in the Podcast dataset) - and that these brain-predictive directions are not the principal axes of the embedding space.1

The claim has three load-bearing parts. First, efficiency: performance peaks at ranks far below full dimensionality, and low-rank models often beat the full-space ridge baseline - a denoising effect, not merely compression. Second, generality: the finding holds across three datasets - the Podcast ECoG dataset (nine participants, 1,330 intracranial electrodes, a 30-minute narrative), Brain Treebank Cars 2 sEEG (three participants, about 8,000 words), and the Deep fMRI stories dataset (eight participants) - across five contextual models (GPT-2, GPT-2 XL, GPT-J 6B, Llama 3 8B, RoBERTa Large), with a randomly initialized GPT-2 and static GloVe embeddings as controls, and across five depths per model.1 Third, transfer: subspaces learned without a participant's own data still predict that participant's responses near-optimally, and transfer partially across datasets and modalities, with the two iEEG datasets' linguistic-feature profiles correlating at r = 0.70 while each correlates near zero with fMRI (r = -0.11 and -0.12).1

How it works

The tool is ridge reduced-rank regression. A full ridge encoding model maps word embeddings to neural responses; the authors then decompose the fitted predictions and keep the embedding directions associated with the leading components of predicted neural activity. Embeddings are projected into this bottleneck subspace and a fresh ridge model is refit at each rank, with performance measured as correlation against held-out neural responses.1 The comparison baseline is PCA of the embeddings at matched rank. The separation between the two is the paper's sharpest result: PCA needs hundreds of dimensions to match what a few dozen RRR dimensions achieve, because the directions that predict the brain are nearly orthogonal to the directions of maximum embedding variance.1

Two design choices matter for trusting the number. Neural targets were pre-selected by significance against circularly shifted nulls with Benjamini-Hochberg correction, so the rank estimates are computed on electrodes and voxels that carry reliable signal rather than diluted by noise. Saturation rank was defined conservatively as the minimum rank at which all informative target-performance quantiles reach 99 percent of their maximum, with a peak-correlation floor of 0.005 to exclude unreliable targets.1 For fMRI, delay-specific subspace blocks were combined before projection so the rank bottleneck applies to the reconstructed design, not per delay.

The transfer experiments are the conceptual core. In leave-one-subject-out tests, only the embedding subspace travels; the mapping from subspace to brain is refit on the target participant. In the Podcast and fMRI datasets, transferred subspaces required a comparable number of dimensions to saturate - sometimes fewer - than subspaces learned with the held-out participant included.1 Cross-dataset transfer, the harsher test, remained better than transferred PCA but worse than within-dataset RRR, and the gap tracked stimulus differences as well as modality differences.

A feature-decoding analysis gives the result biological texture: what the low-rank subspace over-represents is interpretable. Sentence position and next-word prediction entropy are more decodable from RRR than from PCA at low ranks in all three datasets, while word concreteness is selectively over-represented in the fMRI subspace and absent from both iEEG ones - modality-specific biases, not generic compression artefacts.1

Where a skeptic should push

The most load-bearing assumption is that encoding-model alignment measures something about the brain-model relationship rather than about the statistical structure of language itself. The authors include a randomly initialized GPT-2 control and static GloVe embeddings partly to guard this, and their own citations acknowledge that randomly initialized models show nontrivial predictivity in prior work; the abstract's framing of the finding as holding across model conditions including randomly initialized ones cuts both ways - it argues robustness of the low-rank phenomenon, but also that alignment this cheap may not certify anything deep about trained models as models of the brain.1

Second, the direction of inference. These are encoding models: they predict neural responses from embeddings. The low-rank result says a small subspace of the model is sufficient for this prediction; it does not say the brain uses that subspace, computes the same function, or that the residual 97 to 99 percent of embedding dimensions are brain-irrelevant in any causal sense. Correlation-based encoding with linear maps is the weakest notion of alignment the field has, and the paper never claims more.

Third, scale and selection. Three datasets, seventeen participants across them, one language, one task (listening or viewing narratives). The 99-percent saturation threshold and the 0.005 correlation floor are defensible but consequential choices; the appendix shows sensitivity analyses, which is good practice, but the headline 1 to 3 percent figure inherits them. And the pre-selection of significant electrodes flatters cross-dataset numbers: a subspace transferred to a new participant is evaluated on the most predictable components of that participant's own signal. None of this overturns the result; all of it bounds how far the number should travel.

Cheaper readouts, weaker evidence

The non-obvious implication for organoid intelligence and biological computing is about the bandwidth budget of coupling living tissue to a model. The field's instinct has been that richer interfaces - more electrodes, higher sampling, denser feature spaces - buy richer readouts. This paper puts a hard counter on that instinct for one well-characterised system: the subspace of model state that linearly predicts neural activity has rank in the tens, not the thousands, and its directions are findable from other brains.1 If even partial analogies hold for stem-cell-derived neural tissue, the practical consequence is that an organoid readout or stimulation front-end does not need to chase the full dimensionality of its silicon partner - a compact linear bridge, identified once and transferred, may capture nearly everything a linear interface could extract anyway. That reframes hardware roadmaps: the scarce resource is not channel count but which low-dimensional directions matter and how stable they are over the tissue's life.

There is also a methodological gift. The RRR-plus-transfer protocol is exactly the right template for a field drowning in batch-to-batch biological variability: learn the predictive subspace on a library of recordings, then fit only the small final map on each new preparation. The paper's own result - that transferred subspaces sometimes saturate in fewer dimensions than participant-inclusive ones - suggests that pooling across preparations denoises the subspace estimate, which for organoids, where every dish is its own individual, could matter more than it does for human intracranial data.1

The threat is symmetrical and should be taken seriously. If 1 to 3 percent of a model's dimensions suffice for near-full linear neural prediction, and the same is true of a randomly initialized model, then decoding performance on a linear ridge metric is weak evidence of computational equivalence - cheap alignment can be manufactured by statistics of the stimulus and the measurement, not earned by shared computation. For a field that sometimes markets impressive encoding correlations as proof that tissue and model are doing the same thing, this paper is a calibration device: alignment claims need causal or representational tests beyond linear rank efficiency, or they risk measuring the bridge rather than the territory.1 The honest synthesis: the result makes building interfaces cheaper and evaluating substrate claims harder, and both halves of that sentence are the finding's real contribution.

The bottom line

Established: across three public datasets spanning iEEG and fMRI, five contextual language models at multiple depths, and controls, a low-rank embedding subspace of 1 to 3 percent of dimensions (saturating at tens of dimensions) recovers full linear encoding performance, beats PCA at matched rank by an order of magnitude in dimension count, and transfers across held-out participants with the final map refit per person.1 Not established: that the brain computes the model's function, that the low-rank structure is causal or behaviourally relevant, or that the numbers survive outside narrative language listening in healthy human cortex. What would confirm the broader significance: the same rank efficiency and cross-preparation transfer in a different modality or species, ideally with a causal perturbation test of the identified subspace. What would break it: failure of transferred subspaces to beat matched-rank PCA beyond language, which would demote the result from principle to parochial fact about linguistic embeddings.

Frequently asked questions

What does 1 to 3 percent actually mean?

The fraction of the LLM embedding dimensionality needed to recover full linear encoding performance. In absolute terms the subspaces saturate at tens of dimensions - a mean rank of about 20 in the Podcast dataset - while PCA of the embeddings needs hundreds of dimensions for comparable prediction.

Is this just dimensionality reduction?

No, and the PCA comparison proves it: at matched rank, unsupervised PCA performs far worse, because the brain-predictive directions are nearly orthogonal to the directions of maximum variance in the embeddings. Supervision against neural data is doing the work.

Which data and models were tested?

Three public datasets: Podcast ECoG (nine participants, 1,330 electrodes, a 30-minute narrative), Brain Treebank Cars 2 sEEG (three participants), and the Deep fMRI stories dataset (eight participants). Models: GPT-2, GPT-2 XL, GPT-J 6B, Llama 3 8B, RoBERTa Large, plus a randomly initialized GPT-2 and static GloVe embeddings as controls.

Does the subspace transfer across people?

Yes. Subspaces learned without a participant's own data predicted that participant's responses near-optimally, sometimes saturating in fewer dimensions than participant-inclusive subspaces, with only the final mapping refit per person. Cross-dataset transfer works partially and better than transferred PCA.

Does this mean LLMs model the brain?

Not by itself. These are linear encoding models - predicting neural responses from embeddings - and cheap alignment can arise from stimulus statistics rather than shared computation. The randomly initialized control exists precisely because alignment alone is weak evidence of equivalence.

Why should organoid computing care?

Two reasons. Practically, if neural predictivity is low-rank, interfaces to living tissue may need far fewer channels and a compact, transferable bridge rather than full-dimensionality coupling. Methodologically, the learn-once, refit-the-final-map protocol is a template for handling batch-to-batch variability across organoid preparations.

What would strengthen or weaken the claim?

Stronger: rank efficiency and cross-preparation transfer replicated outside narrative language, with causal perturbation of the identified subspace. Weaker: failure of transferred subspaces to beat PCA beyond language, or dependence of the 1 to 3 percent figure on the saturation threshold and target-selection choices.

References

  1. Z. Paris and E. Finn. A Low-Rank Space Captures Linear Alignment Between LLM and Human Neural Activity During Naturalistic Language Listening. arXiv preprint 2610.40511. 2026. https://arxiv.org/abs/2610.40511. Accessed 2026-11-15.