Research analysis · Dynamic processing theory

The Transformer as a machine that rebuilds its weights mid-thought

A new conceptual analysis by two University of Cagliari philosophers argues that the Transformer revolution is not a scaling story at all: it is the arrival of a different principle of inference, in which the outputs of some networks become the weights of others while inference is already running, so each input partially constructs the transformations that process it. The paper contains no new experiments. What it contains is a mechanical account precise enough to be falsifiable, and a final section that names axo-axonic synapses and dendritic spine cooperativity as candidate biological realizations of the mechanism. For a field trying to compute on living neural tissue, that final section is the interesting one.

Source: The Transformer Paradigm: Dynamic Processing and a New Principle of Inference, arXiv:2611.17953 [cs.AI], November 2026. Primary source. Read: the full 8 page PDF, including the static versus dynamic transforming concepts analysis, the output-weight interconnection mechanism, the GPT-4.5 Turing test reanalysis with its counts and binomial tests, the research perspectives section on neuromorphic and biological realization, and the reference list.

What the work claims

This is a position paper in the philosophy of AI, and it should be weighted as one: its authors, Marco Giunti and Fabrizia Giulia Garavaglia, are a logician and a philosopher of science, not a machine learning lab, and the argument is a redescription of the Transformer architecture, not a new result about it.1 The claim is that describing the Transformer as a deeper, wider instance of ordinary deep learning misses the discontinuity. The authors distinguish what is transformed (token representations, which they call transformee concepts) from what does the transforming (parameterized transformations, or transforming concepts), and then split the latter into static transforming concepts, whose parameter matrices are fixed at the end of training, and dynamic transforming concepts, whose parameter structures are constructed during inference from the current input. Inference, on this account, is a two stage affair: previously acquired transformations are first adapted to the current input and only then applied to it. They call the resulting mode of processing Sequence-level Interactive Dynamic Parallel Processing, or SIDPP: token representations are processed in parallel at the sequence level, processing streams interact across tokens through attention, and some of the transformations applied are dynamically constructed from the input itself.

The paper's one quantitative exhibit is a reanalysis of the published GPT-4.5 Turing test results of Jones and Bergen. In the PERSONA condition, a 187 word system prompt specifying a personality and linguistic style, GPT-4.5 was judged human in 185 of 254 tests, 72.8 percent, significantly above the 50 percent chance level by an exact one-sided binomial test. In the NO-PERSONA condition, a 21 word prompt merely instructing the model to convince the interrogator it was human, it was judged human in 50 of 141 tests, 35.5 percent, significantly below chance. The authors computed these binomial p values themselves from the aggregated counts across the Prolific and UCSD student samples reported in the original study. A prompt difference of 166 words produced a 37.4 percentage point swing in judged humanness from the same trained model. The authors are careful to say this is not a test of their framework, only a behavioral illustration of the prompt sensitivity that dynamic processing helps explain: change the prompt and you change the dynamically constructed parameter structures, here the attention key and value matrices, and thereby the transformations applied downstream.

How it works

The mechanical heart of the argument is the output-weight interconnection. In a standard dense layer, input units route signals and output units compute weighted sums with weights fixed at training time. The authors redescribe attention in mechanical-structural terms as the opposite arrangement: the output of one group of networks does not become the input of another group but becomes its weights. When the query, key, and value projections generate key and value matrices from the current input, those generated matrices parameterize the attention transformation itself, so the receiving network operates during inference with weights whose values depend on the current input. A static transformation generates input-dependent weights, and the resulting dynamic transformation is then applied to token representations. Crucially, they concede the underlying idea is not new: hypernetworks and fast-weight architectures (Ha, Dai, and Le 2017; Schlag, Irie, and Schmidhuber 2021) explored generated and rapidly changing weights before the Transformer. Their claim is narrower and, to their credit, stated as such: the standard Transformer can be systematically redescribed as an architecture in which output-weight interconnections are constitutive of ordinary inference, not an exotic add-on.

The final section draws the consequence the authors care most about. If SIDPP names a processing principle rather than a particular architecture, other substrates could in principle realize it, and they sketch two. Neuromorphically, the open question is whether output-weight interconnections can be physically realized as local, dynamically modifiable relations between components rather than simulated through massive matrix operations, citing recent neuromorphic self-attention work aimed at cutting the energy cost of Transformer processing. Biologically, they point at mechanisms where the activity of one component modifies how another connection transforms a signal: axo-axonic synapses, in which one axon synapses onto the presynaptic terminal of another and modulates its neurotransmitter release, and local interactions between neighboring synapses on the same dendrite, which modify synaptic efficacy in a context-dependent way. They explicitly call the correspondence programmatic rather than identifying: the biological mechanisms show the required kind of local modulation is compatible with neural organization, not that brains implement attention.

Where a skeptic should push

The single most load-bearing assumption is that the dynamic-versus-static distinction marks a genuine explanatory discontinuity rather than a relabeling of attention. Every expert reader already knows that attention computes input-dependent mixtures, and the fast-weights literature the authors themselves cite made the connection to dynamically generated weights years ago. The paper's contribution is conceptual clarity, not mechanism discovery, and its Kuhnian framing, a new paradigm in a limited but substantive sense, does no empirical work. A skeptic should also note the load-bearing exhibit is a reanalysis of someone else's data: the GPT-4.5 numbers are real and verified, but the aggregated two-sample table flattens population differences the original paper treats separately, and the binomial tests are the authors' own calculation on counts they did not collect.

Second, prompt sensitivity has simpler explanations that the framework must outcompete: persona prompts change behavior because they add behavioral instructions, and no special architecture is needed to explain why instructions change output. The authors' reply, that the prompt changes the transformations and not just the information, is architecturally true but behaviorally indistinguishable from the boring account in this example.

Third, the biological section is speculation by the authors' own labeling, and a peer reviewer should hold it there. Axo-axonic synapses and spine cooperativity are real, well-documented modulatory mechanisms, but showing that local modulation exists is a long way from showing it implements input-constructed transformations of the SIDPP kind, with the formal operation, timescale, and precision that role would require. The paper claims compatibility, not realization, and readers building research programs on it should keep that boundary sharp.

Dynamic inference on living neural tissue

The non-obvious implication is a functional specification, and it cuts against how the organoid field usually frames itself. Most biological computing proposals implicitly treat the tissue as a fixed-function dynamical system: train it, or exploit its spontaneous dynamics, and read out a result. The SIDPP account says the capability that makes modern inference systems powerful is precisely that they are not fixed-function, that a fraction of the transformation applied to each input is constructed from that input during the computation. If that account is even roughly right, then the value of a computing substrate scales with how cheaply and how fast it can modulate its own effective connectivity during a task, not with how complex its static wiring is. That reframes the organoid question from how do we train this tissue to what does this tissue already do to its own effective weights on a millisecond timescale. And tissue has a genuine candidate mechanism that silicon lacks: neuromodulation and short-term plasticity are exactly local, activity-dependent modifications of how one connection transforms a signal. A cortical organoid receiving a patterned modulatory input alongside its drive input is, in SIDPP terms, a substrate in which one signal stream can set the effective transformations applied to another. Nobody has demonstrated that capability usefully, but the conceptual framework gives the field a crisp experimental question it did not previously have: can an external modulatory channel, at biologically plausible amplitude, reconfigure an organoid's input-output function on a timescale faster than any readout retraining?

The opportunity, then, is a measurement program rather than a device. The field should stop benchmarking organoids only on static tasks with frozen readouts and start quantifying dynamic range of transformation: how much can the effective computation be steered within a single trial by an activity-dependent signal, and with what precision. An axo-axonic-style modulatory pathway interfaced through the same electrode array used for recording would be the direct experimental probe, and success would give biological computing a capability argument that has nothing to do with energy efficiency.

The threat is symmetric and worth stating bluntly. The same framework invites the neuromorphic community to build output-weight interconnections in analog electronics, where the parameters are controllable, measurable, and reproducible, and cites work already moving in that direction. If dynamically modifiable crossbars deliver SIDPP-style processing in CMOS, the tissue's remaining advantages are plasticity across hours and days, self-repair, and analog richness it cannot currently control or even monitor. A philosophy paper cannot decide that race, but it defines the terms: the organoid field has perhaps two years of novelty left in the dynamic-modulation story before an engineered substrate claims it, and the only defense is a demonstrated, quantified demonstration that living tissue performs input-constructed transformations better than the analog alternative. The hype correction runs the other way too: the paper's own biological section is labeled programmatic, and any organoid proposal invoking dynamic weight modulation should be held to the same standard, compatibility with a mechanism is not realization of a computation.

The bottom line

Established: the paper gives a coherent, mechanically explicit redescription of Transformer inference in which input-dependent parameter construction through output-weight interconnections is constitutive, and it documents, using verified counts from a published Turing test study, how strongly prompt changes swing behavior. Not established: that this redescription is a new paradigm rather than a sharpened vocabulary, that prompt sensitivity requires the framework, or that axo-axonic synapses implement anything formally equivalent to dynamic transformations. The claim would be confirmed by a substrate, engineered or biological, that physically realizes output-weight interconnections and matches matrix-simulated SIDPP processing at lower energy, and weakened if dynamic-weight analog hardware arrives while biological realizations stay at the compatibility stage. For organoid intelligence the durable contribution is a question and a deadline: measure whether living tissue can reconfigure its own effective transformations mid-computation, before someone builds that capability in silicon.

Frequently asked questions

What is SIDPP?

Sequence-level Interactive Dynamic Parallel Processing, the authors' name for the processing mode they argue the Transformer realizes: token representations processed in parallel at the sequence level, with interaction across tokens through attention, and some transformations applied during inference dynamically constructed from the current input rather than fixed by training.

What is an output-weight interconnection?

The paper's central mechanism: the output of one network becomes the weight of a connection in another network, rather than its input. In attention, the key and value matrices generated from the current input parameterize the attention transformation, so the receiving computation runs with weights that depend on the input being processed.

Is this a new discovery about the Transformer?

No, and the authors say so. Hypernetworks and fast-weight architectures explored generated weights before the Transformer. The paper's claim is conceptual: the standard Transformer can be systematically redescribed as an architecture where such interconnections are constitutive of ordinary inference, which reframes what the architecture's success means.

What evidence does the paper offer?

One quantitative exhibit, a reanalysis of published GPT-4.5 Turing test data: with a 187 word persona prompt the model was judged human in 72.8 percent of 254 tests, with a 21 word prompt in 35.5 percent of 141 tests, both significantly different from chance by exact one-sided binomial tests the authors computed from the aggregated counts. They present this as an illustration of prompt sensitivity, not a test of the framework.

Which biological mechanisms does it point to?

Axo-axonic synapses, where one axon modulates the presynaptic terminal of another and thereby changes how that connection transforms its signal, and local context-dependent interactions between neighboring synapses on the same dendrite. The authors label the correspondence programmatic: it shows compatibility, not that these mechanisms implement SIDPP-style computation.

Why does this matter for organoid intelligence?

It converts a vague ambition, computing like the brain, into a specific capability to measure: whether a substrate can modulate its own effective connectivity during a task, under the control of an input signal. Neuromodulation and short-term plasticity give living tissue candidate mechanisms for exactly that, and analog neuromorphic hardware is moving toward the same target, which sets a clock on the biological claim.

References

  1. M. Giunti and F. G. Garavaglia. The Transformer Paradigm: Dynamic Processing and a New Principle of Inference. arXiv:2611.17953 [cs.AI]. 2026. https://arxiv.org/abs/2611.17953. Accessed 2026-11-19.
  2. M. Giunti and F. G. Garavaglia. The Transformer Revolution, Part 1: Dynamic Processing through Output-Weight Interconnections. arXiv:2608.03921. 2026. https://arxiv.org/abs/2608.03921. Accessed 2026-11-19.
  3. C. R. Jones and B. K. Bergen. Large Language Models Pass a Standard Three-Party Turing Test. Proceedings of the National Academy of Sciences 123(21): e2524472123. 2026. https://doi.org/10.1073/pnas.2524472123. Accessed 2026-11-19.
  4. D. Ha, A. Dai, and Q. V. Le. HyperNetworks. International Conference on Learning Representations (ICLR). 2017. Accessed 2026-11-19.
  5. I. Schlag, K. Irie, and J. Schmidhuber. Linear Transformers Are Secretly Fast Weight Programmers. Proceedings of the 38th International Conference on Machine Learning, PMLR 139: 9355 to 9366. 2021. Accessed 2026-11-19.
  6. N. Leroux, J. Finkbeiner, and E. Neftci. Neuromorphic Principles in Self-Attention Hardware for Efficient Transformers. Nature Computational Science 5: 708 to 710. 2025. Accessed 2026-11-19.
  7. K. K. Cover and B. N. Mathur. Axo-Axonic Synapses: Diversity in Neural Circuit Function. Journal of Comparative Neurology. 2020. https://doi.org/10.1002/cne.25087. Accessed 2026-11-19.
  8. J. P. Weber, B. K. Andrásfalvy, M. Polito, Á. Magó, B. B. Ujfalussy, and J. K. Makara. Location-Dependent Synaptic Plasticity Rules by Dendritic Spine Cooperativity. Nature Communications 7: 11380. 2016. https://doi.org/10.1038/ncomms11380. Accessed 2026-11-19.