A decoder that treats cortex as a program source

What the work claims

This is a methods paper with primary results, single-authored by Morteza Sarafyazd of BrainCo. The claim is that a von-Neumann-style separation between control and execution is a better inductive bias for neural decoding than the standard Transformer. In a normal Transformer the feed-forward block applies the same weight matrix to every token. Here, the feed-forward block is replaced by a low-rank instruction bank: a shared base operator plus a small set of learned low-rank instructions. A per-token instruction code, read from a low-dimensional projection of a carried state-space memory, synthesizes the actual weight matrix used at that token. The slow latent trajectory acts as an instruction pointer: low-dimensional dynamics deciding which operator the token-specific computation runs.¹

The network is trained as a joint neural codec, simultaneously predicting spiking activity and decoding behavior, on three public primate datasets: MC_RTT (DANDI 000129, primary motor cortex, random-target reaching, 130 units), MC_Maze (DANDI 000128, M1 and dorsal premotor cortex, maze reaches, 182 units), and Area2_Bump (DANDI 000127, somatosensory area 2, 65 units), binned at 50 ms. Against a memoryless linear decoder and a matched modern Transformer, decode R-squared at a limited-data budget of roughly 25 percent of the full training budget is, respectively: on MC_RTT, minus 0.236, 0.207, and 0.351; on MC_Maze, 0.549, 0.655, and 0.716; on Area2_Bump, 0.561, 0.632, and 0.696. The model runs at a fixed depth of about 150K parameters, and results are means over three seeds.¹

The boldness is in the diagnostic, not just the margin. The authors sweep the instruction-bank size K from 1 to 32 at fixed model size and measure how much of the program space the network actually uses, via the spectral entropy of the per-token code distribution. The answer: about 2.55 bits on MC_RTT, 2.89 on MC_Maze, and 3.20 bits on Area2_Bump, against a ceiling of log2 of 32 equals 5 bits, corresponding to only 3.4 to 6.5 effective operators. Decode accuracy is essentially flat across the whole sweep. Program capacity, they argue, acts as a control channel rather than an accuracy lever: the network compresses a large bank into a few reusable programs.¹

How it works

Three memory pathways carry information across the sequence. The first is ordinary attention within the current window. The second is a selective state-space model, a diagonal input-dependent dynamical system whose per-channel state evolves slowly and is carried across windows. The third is a fast-weight associative memory, a delta-rule matrix updated at each step. The instruction pointer is a low-dimensional readout of the state-space pathway: a small projection of that slow state produces the per-token code that selects and blends the low-rank instructions synthesizing the SwiGLU feed-forward weights. A small, slow latent thus routes higher-complexity, token-specific computation, which is the von-Neumann hypothesis translated into a differentiable layer.¹

Two results give the mechanism teeth. First, data scaling: at the smallest training budgets the model leads on all three codecs, and the gap is largest where data is scarcest, for example on MC_RTT at 2K bins the model decodes at R-squared 0.295 versus the Transformer's 0.192, and at 8K bins it reaches 0.607 versus 0.522. Second, context scaling: holding the number of optimizer steps constant across context lengths in {16, 32, 64, 128} bins, the plain Transformer's accuracy falls as context lengthens, while the model rises monotonically and overtakes it by roughly 24 bins on the scarcest codec. The carried low-dimensional state integrates long-range structure that attention windows miss. Because the instruction bank synthesizes both projections of the feed-forward block per token, the authors argue the gain is representational, meaning richer per-token operators, rather than a parameter-count effect. A small language-modeling check on two text benchmarks shows the same parameter-efficiency pattern, suggesting the mechanism is not specific to neural data.¹

Where a skeptic should push

The single most load-bearing assumption is that the instruction bank is the source of the gain. The paper reports no component ablation. The model adds, at once, a state-space memory, a fast-weight memory, and the programmable feed-forward block, and the strongest rival explanation is uncomfortable: the carried slow state may be doing most of the work by integrating long-range temporal structure that a fixed-window Transformer simply lacks, with the instruction bank as a garnish on top. The representational argument, that synthesizing both SwiGLU projections makes the gain per-token rather than parametric, does not exclude this, because the memory pathways and the bank are entangled by construction. Until someone removes the bank while keeping the memories, the causal attribution is asserted rather than demonstrated.

Second, scale and provenance. Three seeds on three datasets is respectable but thin; the scarcest benchmark tops out at R-squared 0.351, meaning nearly two-thirds of behavioral variance is unexplained by any model tested. The language check is a sanity probe, not evidence of generality. And a decoding result, however elegant, says nothing about what the tissue computes; it characterizes a readout model conditioned on recordings. Finally, the datasets are anesthetized or behaving primate preparations with hundreds of well-isolated units. Nothing here has been tried on drift-prone, heterogeneous, sparsely sampled signals, which is where the claimed advantages would matter most.

Instruction-pointer decoders for organoid readouts

For organoid intelligence, the relevant constraint is that every training signal is precious. A cortical organoid on a multielectrode array is a one-off preparation: it matures for months, its activity drifts on the timescale of days, it dies, and the next one is a different object. A decoder that leads at the smallest data budgets and keeps improving as context lengthens is the right shape for that economy, because the unit of currency in wetware work is minutes of usable recording from a substrate that cannot be rewound. The finding that a handful of reused operators suffice, 3.4 to 6.5 effective instructions from a 32-slot bank, should also ring a bell: organoid dynamics are widely reported to be low-dimensional and bursty, and a readout architecture whose capacity knob is a control channel rather than an accuracy lever is unusually well matched to a signal that may only ever express a few dynamical regimes.

There is a deeper blueprint here. The paper operationalizes a genuine neuroscience hypothesis: that cortex routes computation through low-dimensional latent dynamics, an instruction pointer rather than a lookup table. If that view is right, then the correct way to talk to living neural tissue, organoid or otherwise, is to treat the tissue as a program source and build decoders that model the pointer, not just the map from spikes to output. Closed-loop organoid training, where stimulation perturbs the state and the readout must track which dynamical regime the tissue is in, is exactly the setting where a carried slow state with a few operators should beat a context-hungry Transformer.

The threats are equally concrete. The unproven causal attribution cuts both ways: if the gain is really the memory pathways, then the honest lesson for the field is that long-range temporal integration, not programmability, is what organoid readouts lack, and that is a hardware and data problem, not an architecture problem. The modest absolute numbers are a warning against reading benchmark wins as capability. And there is a quiet dual-use note: sample-efficient decoders of neural population activity are the core technology of brain-computer interfaces, the author's own field, and an architecture that decodes well from little data lowers the data-collection barrier for invasive interfaces generally. None of that diminishes the result; it bounds it.

The bottom line

Established: on three primate motor-cortex datasets, this memory-augmented Transformer decodes behavior more efficiently than a matched Transformer at every tested data budget, and its accuracy improves with context where the baseline degrades. Plausible but not proven: that the low-rank instruction bank, rather than the companion memory pathways, drives the gain; no ablation separates them. Established: the network uses only about 2.5 to 3.2 bits of a 5-bit program space, so program capacity is a control channel, not an accuracy lever. What would confirm the architectural claim is a bank-only ablation, plus a demonstration on drift-prone, weakly sampled signals such as organoid arrays or clinical recordings. What would break it is evidence that the same gains appear in a model with the memory pathways but a fixed feed-forward block. For organoid intelligence the result is best read as a design brief: build readouts that expect few-shot budgets, long context, and a tissue that changes its mind about which program it is running.