Research analysis · Recurrent architectures

A recurrent network that designs its own memory timescales, and what that demand means for biological computing

Recurrent networks are supposed to be the natural substrate for temporal computation, yet in practice their memory timescales are emergent, fragile, and hard to train around. ADPTNet, a new architecture built from topological conjugates over the orthogonal manifold, makes the Lyapunov spectrum, the complete set of exponential decay rates that defines how long information lives in the state, a designed quantity with theoretical guarantees. A spiking version of it sets a new benchmark on a standard neuromorphic speech task.

Source: ADPTNet: Adaptive with Prescriptive Timescales Non-Linear SSM for Sequence Modelling, arXiv:2609.34034, September 2026. Primary source. Read: full HTML text including theory, benchmark tables, and discussion.

What the work claims

This is a methods paper, an 80-page single-author preprint combining proofs, architecture design, and benchmarks. The claim is that four properties which normally trade off against each other, non-linear recurrence, input adaptability, long-range temporal memory, and GPU-parallel training, can coexist in one architecture, because the mechanism that provides adaptability is constrained to preserve a prescribed global dynamics1.

The core theoretical result is a guarantee: the Lyapunov spectrum of ADPTNet's dynamics is bounded by the logarithm of the magnitudes of its recurrent eigenvalues, a relationship the paper validates empirically across trained networks and tasks. Because the eigenvalues are explicit parameters, this gives parametric, predictable control over every timescale in the network, not just the largest. On benchmarks, non-linear ADPTNet lifts Selective Copying accuracy from 36 to 80 percent by strengthening a single coupling coefficient, outperforms the Hawk model's balance of memory and adaptability, matches linear state-space model accuracy, in the class popularized by Mamba2, on sequential CIFAR-10 with fewer parameters than selective models including a Transformer, and a spiking variant, SpikingADPTNet, reaches 83.56 percent accuracy plus or minus 0.15 over four seeds on Spiking Speech Commands, more than one point above the previous state of the art for non-augmented models1.

The same control pays off in training: because a step-size parameter and the eigenvalues predict convergence behavior, the paper derives Jacobian-free parallel simulation extensions, Conv-DEER with zero computational overhead beyond the forward pass, and Forward-DEER, which retain the same average convergence as the base parallel-in-time method.

How it works

The design starts from a deliberately boring global dynamics: a linear time-invariant system with fixed, explicitly parameterized complex eigenvalues, initialized in the S4D-Inv style, which set the network's decay rates and oscillation frequencies. The interesting part is a data-dependent change of coordinates applied locally in time. Each update computes a similarity transform of the recurrent matrix, but the transform is constrained to the orthogonal manifold, so it can rotate the dynamics without stretching it. These are the topological conjugates: locally they redirect the flow of the state, while globally the conjugacy preserves the spectrum that governs long-term behavior1.

Two ingredients make this tractable. First, optimizing over orthogonal matrices uses Riemannian geometry, specifically a Cayley-map parameterization, rather than generic matrix inverses, which the paper notes are numerically unstable. Second, the conjugates are chained: each one shifts the flow direction while keeping quantities like oscillation frequency constant, so the network can be data-adaptive in where its state moves without surrendering control of how fast memory decays. A step-size parameter sets how strongly the conjugates couple off-diagonal directions; the Selective Copying ablation shows this coefficient moving accuracy from 46.3 to 75.6 percent in the linear variant and from 36 to 80 percent in the non-linear one, monotonically, which is the paper's cleanest evidence that controlled coupling is what buys selectivity1.

The theoretical guarantee, stated as a lemma, is that the entire Lyapunov spectrum of the trained dynamics stays within the bounds set by the logarithm of the recurrent eigenvalue magnitudes, and trained models are shown to satisfy it across layers and tasks, with trained networks that must discard distractors evolving shorter timescales than those trained to retain everything. Decay rates and oscillation frequencies are functionally separated: the Lyapunov spectrum governs exponential memory decay, while complex eigenvalues carry rotation, matching the auditory-cortex motivation that inspired the design, cortical processing operating on fixed intrinsic timescales with adaptation layered on top1.

Where a skeptic should push

The most load-bearing assumption is that bounded, prescribed timescales are the property that matters for sequence modeling at scale. The benchmarks here are deliberately small: Selective Copying at sequence lengths of 128 and 1024, sequential CIFAR-10, and Spiking Speech Commands. The 83.56 percent record is real but scoped: it holds only within the class of models trained without spike-data augmentation, while the literature contains results above 88 percent once augmentation enters, and the small SpikingADPTNet was tested on a single seed. The paper is candid about underparametrizing models to isolate effects, which is good science and simultaneously a reminder that absolute numbers are not the point.

Second, the theoretical guarantees rest on computing Lyapunov spectra from singular values of long-term Jacobians, an implementation choice the paper discusses at length, and the proven bounds describe exponential decay only, not the oscillatory component. The auditory-cortex framing is inspiration, not evidence: no biological data appear. Demonstrated: the eigenvalue-to-Lyapunov-spectrum relationship in trained networks, monotonic selectivity gains from the coupling coefficient, benchmark wins within their stated classes, and Jacobian-free parallel training. Asserted but not demonstrated: behavior at the scales where Transformers actually live, robustness of the guarantees under quantization or spiking non-idealities beyond the benchmarked setting, and any claim that these dynamics resemble real cortex.

The timescale contract organoid memory must meet

Biological computing has one canonical pitch: living neural tissue possesses a rich, self-organized diversity of intrinsic timescales, from fast sodium spikes to slow adaptive and modulatory processes, and that diversity is exactly what engineered recurrent systems lack. This paper inverts the pitch. It shows that in silicon, timescale diversity is no longer something you discover in a trained network but something you specify: pick eigenvalues, get a guaranteed Lyapunov spectrum, then layer adaptability on top without losing the guarantee. The bar for the wetware claim rises from "tissue has rich dynamics" to "tissue offers dynamics you cannot simply parameterize," and this architecture narrows what that residual could be.

The opportunity for organoid intelligence is a concrete experimental program the paper effectively writes. If addressable timescales are the deliverable, then a culture demonstrates value by showing addressable control of its own decay rates, through neuromodulation, stimulation history, pharmacology, or temperature, with a measured Lyapunov or fading-memory spectrum that shifts predictably under command. That is a publishable, falsifiable capability, and far stronger evidence of computationally useful dynamics than power spectra or burst statistics. The hybrid partition also becomes obvious: tissue can supply the non-linear, adaptive part while a silicon front end owns the prescribed slow timescales, an arrangement this paper's separation of decay control from flow direction formalizes.

The threat is the same observation turned around. If organoids cannot control their timescales, and silicon now can, then a culture is a fixed, drifting dynamical system whose memory horizon you inherit rather than design. Batch-to-batch variability in ion-channel expression and maturation becomes variability in the very quantity this paper renders a parameter. And the training story sharpens: the paper shows prescribed timescales make parallel training and Jacobian-free simulation possible, while no equivalent lever exists for living tissue, where every training method must route around an uncontrolled substrate. None of this makes organoid computing hopeless; it makes the field's honest pitch smaller and more specific, which is what a maturing field looks like.

The bottom line

Established within its benchmark scope: a mechanism that keeps a non-linear adaptive recurrent network's Lyapunov spectrum inside designed bounds, monotonic selectivity gains from a single coupling coefficient, competitive accuracy with fewer parameters than selective baselines on sequential CIFAR-10, and a spiking state of the art on Spiking Speech Commands within the non-augmented class. Unestablished: scaling, robustness under deployment non-idealities, and biological plausibility. For organoid intelligence the paper's real contribution is diagnostic: it defines the timescale contract any substrate must meet. What would confirm tissue's role: demonstrations of externally addressable, stable control of a living network's fading-memory spectrum. What would further erode the wetware case: the same guarantees appearing on neuromorphic hardware with sub-milliwatt power, at which point the niche for uncontrolled living dynamics narrows to whatever non-linearity and adaptation tissue can offer that engineered conjugates cannot.

Frequently asked questions

What is a Lyapunov spectrum in plain terms?

It is the full set of exponential growth or decay rates of an infinitesimal perturbation to a dynamical system's state. Negative values mean perturbations, and therefore information, fade; each exponent corresponds to one direction in state space and one memory timescale.

What are topological conjugates here?

Data-dependent similarity transforms of the recurrent dynamics, constrained to orthogonal matrices so they rotate the flow without stretching it. They let the network adapt where its state goes while leaving the global spectrum that governs long-term memory untouched.

What does ADPTNet guarantee?

That the trained network's Lyapunov spectrum stays bounded by the logarithm of its recurrent eigenvalue magnitudes. Since the eigenvalues are explicit parameters, every memory timescale is a designed, predictable quantity rather than an emergent one.

How strong is the benchmark evidence?

Moderate. The spiking variant reaches 83.56 percent on Spiking Speech Commands, over a point above the previous non-augmented state of the art, and Selective Copying improves monotonically with the coupling coefficient. But tasks are small, the record excludes augmented models above 88 percent, and one configuration used a single seed.

Why does this matter for organoid intelligence?

Because tissue's advertised advantage is rich intrinsic timescales, and this work shows silicon can now prescribe timescales with proofs. The field must either demonstrate addressable control of a living network's memory spectrum or redefine what biological substrates contribute.

What is Conv-DEER?

A Jacobian-free extension of a parallel-in-time simulation method for recurrent networks, made possible because ADPTNet's eigenvalues predict its long-term behavior. It adds no computation beyond the forward pass and retains the base method's average convergence.

References

  1. M. I. Stan. ADPTNet: Adaptive with Prescriptive Timescales Non-Linear SSM for Sequence Modelling. arXiv:2609.34034. 2026. https://arxiv.org/abs/2609.34034. Accessed 2026-11-10.
  2. A. Gu and T. Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv:2312.00752. 2023. https://arxiv.org/abs/2312.00752. Accessed 2026-11-10.