Could cortical columns build an attention head?
Two philosophers of science at the University of Cagliari argue that the functional architecture of a Transformer attention head, treated as an abstract system type rather than a matrix recipe, could be physically constructed by groups of pyramidal columns in posterior temporal language regions, using presynaptic GABA modulation and activity-driven spine formation as the hardware for dynamically generated weights.
Source: Pyramidal Columns Acting as Attention Heads, arXiv preprint 2611.16057, 11 Nov 2026. Primary source. Read the full 8-page preprint PDF plus its supplementary figures.
What the work claims
This is a theory paper, not an experiment. Garavaglia and Giunti ask whether the functional organization of a single Transformer attention head could receive a biological construction: could a suitably organized group of pyramidal columns in posterior temporal language regions (PTLRs), the territory traditionally associated with Wernicke's area, constitute the neural hardware of an attention head?1
The claim is deliberately weaker and stranger than the usual "the brain does attention" story. The authors treat the attention head as an abstract system type, defined by its functional architecture and independent of any particular mathematical instantiation. A mechanical realization preserves that organization in an idealized system of components; a neuromorphic one does so in neural-like parts; a biological construction would do it in actual tissue, under mathematical operations that need not be the standard matrix algebra of the Transformer. The paper's own abstract concedes the endpoint: whether the required functional organization actually exists in PTLRs "remains an open empirical question."
What makes the piece bolder than typical AI-to-brain analogy writing is that it commits to specific neurobiological mechanisms and derives at least one quantitative, falsifiable anatomical prediction from the architecture itself.
How it works
The paper builds on the authors' prior mechanical-structural redescription of attention heads as systems of simple neural networks linked by two kinds of interconnection: output-input interconnections, where one component's output feeds another's input, and output-weight interconnections, where one component's output determines how another component transforms its input.2 The latter is the load-bearing feature of attention: during inference, the key and value processing paths generate parameter structures whose values depend on the current input, and those dynamically generated values define the transformations applied downstream. In the standard Transformer this is the query-key-value machinery; functionally, it is computation in which activity configures other computation.
The authors then ask which biological mechanisms could construct such output-weight relations. They nominate two, both well documented in the neurobiology literature they cite:
GABAergic axo-axonic modulation. A GABAergic interneuron forming an axo-axonic contact on the presynaptic terminal of a glutamatergic pyramidal neuron can suppress glutamate release onto a postsynaptic target. Presynaptic GABA receptors reducing glutamate release are experimentally established, including GABA(B)-mediated inhibition through GIRK channels in rat cerebral cortex.3 Functionally, the glutamatergic signal is the quantity to be transmitted and the GABAergic signal plays the role of a dynamically generated weight governing transmission efficacy: a two-input multiplicative gate realized in synaptic physiology.
Glutamate-dependent dendritic-spine formation. A second mechanism uses local interactions between neighboring axo-dendritic synapses on the same dendrite. Glutamatergic activity at an established synapse can promote the emergence of a nearby dendritic spine, providing a potential site for a second synaptic contact whose efficacy is conditioned on the first. Local glutamate stimulation induces rapid, spatially specific formation of new spines in developing cortical neurons, dependent on NMDA receptor activation, and the resulting spines can rapidly acquire functional properties; nascent spines can establish structurally mature, functional glutamatergic contacts on short timescales.45
The proposed construction assigns three functional stages (explicitly not anatomical layers) to query, key, and value processing paths: stage-I pyramidal neurons transmit, GABAergic interneurons modulate established stage-I to stage-II synapses axo-axonically (the query-key interaction), and value-path neurons condition the formation and efficacy of stage-II to stage-III contacts through spine mechanisms. Autoregressive causality is built into the structural organization rather than imposed as a mask, which matters because a biological construction would not need to reproduce the masking operation at all.
Finally, the paper extracts a quantitative prediction. When the Transformer stack is rearranged into a serpentine two-area layout with all attention heads in one area and the remaining per-layer operations in the other, the number of interconnections from the attention area forward exceeds the return count whenever h times d-key equals or exceeds d-model, which standard architectures satisfy. The authors note this asymmetry "suggests a testable neuroanatomical hypothesis": axonal projections through the arcuate fasciculus may be more numerous from PTLRs to frontal language regions than in the reverse direction.1
Where a skeptic should push
The single most load-bearing assumption is timescale. Both candidate mechanisms exist, but the question is whether they can operate on the temporal scale of language processing. Activity-driven spine formation has been demonstrated in developing cortex; the paper itself flags that the dependence of synaptic efficacy on the two glutamatergic signals "remains to be established," as does whether the sequence can run at language-relevant speeds in PTLR circuitry. Presynaptic GABAergic modulation is fast enough in principle, but turning a modulatory contact into a faithful multiplicative weight over a useful dynamic range is an engineering claim the paper asserts rather than demonstrates.
Second, the mapping is underdetermined in the way all functionalist mappings are. The paper acknowledges that the anatomical and functional status of cortical columns remains debated, and leans on recent human data: cross-laminar speech-related organization in human superior temporal cortex and subtype-specific synaptic interactions among human temporal-lobe pyramidal neurons.1 Those findings motivate the investigation; they do not establish the three-stage organization the construction requires. Nothing in the paper measures an actual output-weight computation in any brain.
Third, the Softmax-role block and the rescaling operations get the thinnest treatment; the authors list biological constructions of these as open questions for further research. A hardware proposal whose normalization stage is unresolved should be read as a parts list with gaps, not a working design. Weight the piece accordingly: it is a careful hypothesis that sharpens what would need to be true, not evidence that it is true.
A parts list for dynamic computation in tissue
For organoid intelligence, the non-obvious contribution is the decomposition itself. Most proposals for computing on living neural tissue lean on plasticity as the trainable element: change synaptic weights slowly, read out spikes fast. This paper draws a different and more demanding distinction between transmission and modulation, and identifies a class of computation, dynamically generated weights, that requires modulatory pathways acting on transmission on the timescale of the task itself, not on a Hebbian learning timescale.1 If you want an organoid to do anything attention-like, the assay is not "does it learn?" but "does it possess a fast two-input gate in which one input sets the efficacy of the other's transmission?" Axo-axonic-style presynaptic modulation is exactly the kind of circuit feature one could screen for in cortical organoid microcircuits, and it is a more precise target than the usual hand-waving about spontaneous activity richness.
The timescale assumption cuts both ways for the field, and this is the genuinely interesting asymmetry. The authors' biggest worry, that spine formation may be too slow for language, largely evaporates in dish-scale systems: organoid preparations routinely operate at developmental and activity timescales orders of magnitude slower than intact cortex, so mechanism-speed mismatches that would sink a language claim might be tolerable, even convenient, in a slow biological computer. The threat hiding in the same fact is symmetry: if your organoid's usable dynamics are slow because its microcircuitry is immature or disorganized, then the specific multiplicative organization this paper requires is precisely what stochastic self-assembly probably does not deliver. "The tissue is slow" and "the tissue lacks the parts" are competing explanations for the same observation, and the field currently has no assay that separates them.
There is also a hype-correction here worth importing. By separating functional architecture from mathematical instantiation, the paper gives a vocabulary for claims like "organoids compute like Transformers": such claims should target functional organization (dynamic weighting of transmission) and stay silent about matrix operations, which a biological or organoid instantiation may realize differently or not at all. Anyone benchmarking organoids against attention metrics is testing the instantiation, not the architecture, and the paper shows the two can come apart in principle.
The bottom line
Hypothesis, not result: a rigorous, mechanism-committed hypothesis with one quantitative anatomical prediction. Established: attention heads can be redescribed as systems with output-weight interconnections, and presynaptic GABA modulation plus activity-driven spine formation are real biological processes that could implement two-input gating relations. Unestablished: that PTLR microcircuits actually instantiate the three-stage organization, that the mechanisms run at task speed, and that the softmax-role and rescaling stages have any biological construction at all. What would confirm the framework: directional tractography showing the predicted PTLR-to-frontal asymmetry in the arcuate fasciculus, plus a slice or organoid demonstration that axo-axonic-style modulation implements a usable multiplicative gate. What would break it: evidence that the modulatory mechanisms cannot operate at the required speed, or that the required stage organization is absent from the relevant cortex.
Frequently asked questions
Is this an experimental paper?
No. It is a theory and analysis paper with no new experiments, no recordings, and no model fitting. Its value is a precise functional decomposition of attention heads married to a specific, literature-grounded set of candidate neural mechanisms, plus one testable anatomical prediction.
What is an output-weight interconnection?
A connection in which one component's output does not become another component's input but instead determines how that other component transforms its own input. In attention, the key and value paths generate input-dependent values that set the transformations applied downstream; the paper treats this as the defining functional feature to reproduce biologically.
Why posterior temporal language regions?
The authors map the attention-head-heavy portion of a rearranged Transformer stack onto PTLRs, the cortex traditionally tied to Wernicke's area, which exchanges projections with frontal language regions through the arcuate fasciculus. The mapping yields the prediction that PTLR-to-frontal projections should outnumber the reverse, because standard architectures send more channels forward from the attention area than back.
Could cortical organoids test any of this?
Partially. Organoids will not settle questions about PTLR anatomy or language processing, but they are a plausible venue for the mechanistic core: whether presynaptic inhibitory modulation can implement a two-input multiplicative gate, and whether activity-conditioned synapse formation can serve as a dynamic weight. Those are assayable in dish-scale tissue at its native, slower timescales.
Does the brain need to run softmax?
Not according to this framework. The paper separates functional roles (multiplicative gating, normalization-like rescaling) from particular mathematical functions. A biological construction must preserve the functional architecture, not the softmax formula; the authors leave the biological realization of the rescaling and normalization roles as an explicit open problem.
What is the strongest evidence against the idea?
Timescale mismatch is the most likely killer. If activity-driven spine formation and maturation of new synaptic contacts run far slower than the relevant processing dynamics, the second mechanism cannot serve as a working weight, and the construction loses its value-path hardware, leaving at most a partial realization.
References
- F. G. Garavaglia and M. Giunti. Pyramidal Columns Acting as Attention Heads. arXiv preprint 2611.16057. 2026. https://arxiv.org/abs/2611.16057. Accessed 2026-11-14.
- M. Giunti and F. G. Garavaglia. The Transformer Revolution, Part 1: Dynamic Processing through Output-Weight Interconnections. arXiv preprint 2608.03921. 2026. https://arxiv.org/abs/2608.03921. Accessed 2026-11-14.
- C. Ladera et al. Pre-synaptic GABA(B) receptors inhibit glutamate release through GIRK channels in rat cerebral cortex. Journal of Neurochemistry 107(6), 1506-1517. 2008. https://doi.org/10.1111/j.1471-4159.2008.05712.x. Accessed 2026-11-14.
- H.-B. Kwon and B. L. Sabatini. Glutamate induces de novo growth of functional spines in developing cortex. Nature 474, 100-104. 2011. https://doi.org/10.1038/nature09986. Accessed 2026-11-14.
- K. Zito, V. Scheuss, G. Knott, T. Hill, and K. Svoboda. Rapid functional maturation of nascent dendritic spines. Neuron 61(2), 247-258. 2009. https://doi.org/10.1016/j.neuron.2008.10.054. Accessed 2026-11-14.