A brain-style division keeps analog memory from shattering, and it quietly sets the design spec for memory in living tissue

What the work claims

This is a modeling and analysis paper: a deliberately minimal architecture, trained on two canonical tasks, read with dynamical-systems tools. The claim has three layers. First, the Recurrent Divisive Normalization Network (RDNN), in which an excitatory working-memory population is divided by an auxiliary inhibitory pool, learns robust, high-fidelity slow manifolds for continuous working memory, avoiding both the fine-tuning fragility of hand-built continuous attractor networks and the discretization that dynamical analyses of trained GRUs and LSTMs keep finding. Second, the division operator shapes optimization itself: during backpropagation through time it applies an activity-dependent scaling that damps parameter updates in highly active regimes, which correlates empirically with a strong self-compression of the network's effective rank. Third, an ablation swapping division for subtractive inhibition shows the two are not interchangeable: subtraction can maintain static memories but division is essential to prevent the manifold from shattering into discrete basins under time-varying inputs.¹

The background is well established in computational neuroscience. Continuous attractor networks are the classical model for analog memory, but they suffer the fine-tuning problem: infinitesimal mis-tuning of the recurrent dynamics destroys the continuous manifold of equilibria and the memory drifts or collapses. Standard gated recurrent networks, for their part, tend to shatter the continuous state space into localized point attractors separated by saddles when trained on the same tasks. What has been missing, the authors argue, is a clean isolation of what the canonical cortical operation of divisive normalization contributes to this problem, independent of the multi-component biophysics of earlier models such as ORGaNICs.

How it works

The RDNN is about as stripped down as a recurrent model can be while still containing the mechanism. An excitatory population R represents the working-memory state; an auxiliary inhibitory pool G provides dynamic gain control; and the recurrent input is divided by the sum of a constant and the inhibitory term, so highly active states automatically shrink their own updates. The dynamics are stochastic differential equations with state noise, emulating intrinsic synaptic noise. Divisive normalization itself is textbook neurophysiology: cortical neurons divide their inputs by a pooled measure of activity, producing contrast invariance and gain control, and variants of the operation recur across visual cortex and beyond.¹

The two tasks probe the two regimes of working memory. The memory-guided saccade task is autonomous maintenance: hold a direction across a delay with no input. The angular velocity integration task is input-driven updating: continuously rotate the internal state in response to a time-varying signal, which is the regime where representations are stressed. Networks were trained at hidden sizes of 64, 128, and 256 with ReLU or tanh activations. In the autonomous task the RDNN's state space is dominated by marginal fixed points, the neutral stability along a continuous attractor that lets the network hold an arbitrary value without drifting toward discrete attractors. In the input-driven task, trained GRUs and LSTMs predominantly form alternating stable fixed points and saddles, effectively discretizing the continuous variable and jumping between basins, while the RDNN maintains rotational dynamics on a smooth manifold.¹

The rank result is the quantitative surprise. Measuring the 99 percent energy effective rank of the merged recurrent weight matrices, the GRU and LSTM baselines scale essentially linearly with hidden size, exploiting nearly all available degrees of freedom and requiring a rank over 240 at 256 units. The RDNN's rank is flat and sub-linear by comparison: at 256 hidden units, approximately 35.4 to 38.0 on the integration task versus over 240 for the baselines. The paper's Table 1 gives, for the ReLU RDNN, 16.8 plus or minus 2.4 at 64 units, 24.0 plus or minus 2.5 at 128, and 38.0 plus or minus 6.7 at 256 on integration, and 19.2 plus or minus 2.5, 27.0 plus or minus 7.2, and 31.0 plus or minus 5.0 on the saccade task. The mechanism linking this to training is the inverse gradient scaling of the divisive term: it attenuates parameter updates along highly active directions, restricting the effective directions of update without formally minimizing any rank criterion, and empirically correlating with the compressed representations. When the authors instead force a low-rank bottleneck explicitly, they hit a higher-rank convergence anomaly in which wider bottlenecks paradoxically train worse, an outcome they attribute to the redundant scaling symmetries and degenerate saddle points of the factorized parameter space.¹

The ablation isolates the operator. A subtractive variant, identical except that inhibition subtracts rather than divides, sustains a stable ring attractor in the autonomous saccade task, matching the RDNN qualitatively when no input arrives. But under the input-driven integration task the subtractive network's manifold shatters into a dense population of discrete stable point attractors on every trained seed, through a sequence of saddle-node bifurcations traced to uncompensated discrete jumps in the Jacobian. Division normalizes the Jacobian smoothly; subtraction cannot. Curiously, the discretized networks can still score well: the appendix's performance comparison on 1024-trajectory test sets shows the subtractive network achieving highly competitive or marginally lower test error than the RDNN even while its underlying representation is highly discretized, because the discrete basins provide a lock-in effect that bounds long-term error, while the RDNN's smooth manifold pays slow diffusive drift, with error growing continuously over 2000 steps.¹

Where a skeptic should push

The most load-bearing assumption is that the mechanism, demonstrated in rate-based recurrent networks trained by backpropagation, is the same mechanism by which biological circuits maintain analog memory. Divisive normalization as a cortical computation is measured in sensory cortex, not in prefrontal or parietal delay circuits where continuous working memory lives, and the inhibitory operations available in real tissue, shunting and subtractive GABAergic currents, map only approximately onto the paper's clean algebraic division. The paper is candid that it is a theoretical and computational model whose spectral and rank predictions require validation against large-scale neural recordings from behaving animals.¹

Second, the rank numbers, striking as they are, are descriptive. The activity-dependent gradient attenuation is derived and the low rank is measured, but the causal chain from one to the other is correlation plus intuition: the paper shows the scaling property of the gradient and notes its empirical alignment with compression, without an ablation that would break the gradient mechanism while preserving everything else. Third, the training is entirely gradient-based, so the optimization story is internal to backpropagation through time. Whether a biologically plausible or externally imposed training rule would find the same manifolds is open. Fourth, the smooth manifold the RDNN preserves is not strictly superior by every metric: over very long horizons the discrete basins of the shattered networks act as error correctors, and the smooth manifold diffuses. Which regime a given application wants is a design question the paper surfaces honestly rather than resolves.

Why living memory needs division, not subtraction

The design spec this hands organoid intelligence is precise and, to my knowledge, new in this form: if you want a cultured network to act as an analog memory that tracks time-varying input, its gain control must be multiplicative, and a stimulation protocol that effectively pushes the tissue's inhibition in a subtractive direction should be expected to shatter the analog representation into discrete basins, whatever the task score says. The converse warning is just as valuable. An organoid decoder that reports smooth, continuous memory behavior may be reading a discretized system whose basins happen to lock in the right values, and an organoid whose error stays flat over very long horizons may be discrete and self-correcting rather than analog and drift-free. The paper's own ablation shows task performance cannot distinguish these two regimes; only the dynamical analysis can. For a field that increasingly scores organoid readouts on task metrics alone, that is a methodological landmine.¹

The rank result carries a subtler implication. If biological-style division forces trained recurrent dynamics into a tight low-dimensional subspace, then the dimensionality of an organoid's task-relevant dynamics is not a free resource to be scaled up by growing the tissue; it is set by the gain-control regime the tissue implements. A 256-unit network doing the work of a 35-dimensional system is a concrete existence proof that effective capacity, not unit count, is the right spec. For organoid systems, where unit counts are large, uncontrolled, and partly unmeasurable, this reframes the benchmarking question: report the effective rank of the trained dynamics, not the electrode count or the neuron count.

The threat is substrate drift in both directions. The opportunity is that division is pharmacologically addressable: inhibitory gain in neural tissue is tunable with GABAergic modulators, and shunting inhibition is at least approximately divisive, so the operator the paper identifies as essential is also a knob a closed-loop rig can turn. The genuine dual-use caution is that a knob controlling how living tissue represents and retains analog variables is a knob controlling the reliability of whatever computation that tissue is trusted with, and the paper's central lesson applies to the experimenter as much as to the network: if you train by pushing harder on the wrong arithmetic, you will not see the damage in the loss, only in the manifold, and in tissue the manifold is the product.

The bottom line

Established in silico: a minimal divisive-normalization recurrent network learns continuous working-memory manifolds on both autonomous and input-driven tasks; the effective rank of its trained dynamics saturates far below that of GRU and LSTM baselines, roughly 38 versus over 240 at 256 units; and subtractive inhibition maintains static memory but shatters the analog manifold under time-varying input, on every seed tested, while sometimes scoring better on the task metric. Hypothesis: that real cortical delay circuits exploit the same division operation and exhibit the same low-rank spectral signature, which the authors note awaits population-recording validation. For organoid intelligence, the durable contribution is a design and diagnosis rule: analog memory in recurrent tissue requires multiplicative gain control, and representational quality must be checked dynamically, not by task error, because shattered basins can look excellent on the scoreboard. What would confirm the mechanism: divisive gain manipulations in cortical tissue preserving smooth manifolds where subtractive manipulations discretize them. What would break it: recordings showing that biological delay circuits hold analog memory through discrete attractors after all, at which point division is a convenience of rate models, not a principle of living memory.