A firing floor that sparsity cannot cross

What the work claims

This is a combined empirical and theoretical paper, and its central claim is a reframing: the energy dividend of sparsity is a property of the task, not of spiking neural networks as a technology. The usual case for spiking hardware is that binary, event-driven activity replaces dense multiply-accumulate operations with cheap accumulate operations, so fewer active neurons means less energy. The author argues that whether you can make activity sparse in the first place is decided by what the network has to compute, and builds a controlled protocol to measure it.

The measurements are pointed. On feed-forward perception the firing rate drives cleanly to a five percent target with no accuracy loss against the unregularised spiking baseline, though that baseline near seventy-five percent stays well below the matched conventional network at about eighty-eight percent, and at this sparsest setting the forty-five nanometre energy proxy rises to roughly five times the conventional network rather than falling. On a recurrent character-level language model the same probe, aimed at ten percent, cannot move firing below about fifty percent, down only from sixty-three, because the recurrent state has to stay active to carry information forward. A spiking Transformer, by contrast, sparsifies freely to two percent at no extra quality cost relative to its own spiking baseline, about 3.57 against 3.55 bits per character while the conventional Transformer sits at 3.05, which localises the ceiling to recurrent compression rather than to sequence modeling in general. Attention escapes the firing floor only by storing the entire key-value cache, trading a firing floor for a memory wall. The claim is sharpened into an information-theoretic firing-floor bound stating that the minimum firing rate is at least the inverse binary-entropy of the representational load divided by the state width, so the floor rises with what must be remembered and falls with how wide the state is.

How it works

The protocol is the reason to trust the result. For each task the author builds one architecture and two variants that differ only in the hidden unit, a conventional continuous neuron in one and a leaky-integrate-and-fire spiking neuron in the other, with every weight shape, depth and readout identical, so any energy gap is attributable to spiking alone. On top of that sits a two-sided target-rate regularizer that penalises firing for being either above or below a requested rate. This is the clever part: a one-sided penalty that only pushes activity down tends to collapse the network into a silent, unrecoverable state, whereas the two-sided version lets the experimenter request a firing rate and observe whether the task will actually permit it. Energy is estimated with a standard forty-five nanometre proxy in which a multiply-accumulate costs several times more than an accumulate.

The bound makes the pattern inevitable rather than incidental. Consider a recurrent spiking network whose only channel out of its state is the binary vector of spikes, which is the operative regime on spike-based hardware. If the task must distinguish among some number of memory contents at each step, then that spike vector has to carry enough bits to name them, and a binary vector of a given width with only a small fraction of ones simply cannot index that many distinct messages. Working the counting through gives a minimum firing rate that increases with the representational load and decreases with the state width. A corollary matters for interpretation: memory is not privileged in itself, any source of representational load raises the floor, but memory is special because copying symbols over a horizon makes the number of contents grow exponentially, which is what floors recurrent language modeling while low-class perception stays free.

Three controlled sweeps confirm the structure. On a synthetic copy task the minimum firing that sustains accuracy rises from about four percent to about eight percent as the memory load grows, and accuracy is non-monotone in firing, so excess spikes actually hurt. Widening the hidden state buys the accuracy back, with a load that collapses a narrow network holding in a wider one at the same five percent budget. A memoryless classification with a growing number of classes also lifts the floor, from roughly nine to eighteen to twenty-six percent, which refutes the author's own initial guess that memory alone sets the floor and confirms the more general load-based reading. A layer-wise accounting then exposes an input floor: even with hidden layers at five percent firing, a dense input frame replayed over several timesteps consumes most of the operation budget and holds the total at parity with the conventional network, which points to native event-driven input, untested here, as the setting that could remove this floor and let the hardware genuinely win.

Where a skeptic should push

The single most load-bearing assumption is stated in the proof and is easy to miss: the bound holds when the spike vector is the sole information channel out of the state. The author is explicit that reading the continuous membrane potential loosens the bound, at the cost of forfeiting the spike-based communication that defines the hardware. This is exactly the seam a critic should pull, because it decides how far the result travels. For digital spike-only neuromorphic silicon the bound bites hard. For a system that also carries information in graded, analog, sub-threshold quantities, the premise does not hold as stated, and the clean floor is not guaranteed to apply.

The scope should also be kept honest about scale and proxy. Perception uses a rate-coded standard image benchmark as a stand-in for an event stream, not native event data, which the author flags as future work. The spiking Transformer that sparsifies so freely is single-layer, character-level and small, so its escape from the floor is demonstrated in miniature, and the direct cost of its key-value memory wall is not yet quantified. Energy is a forty-five nanometre proxy rather than a measurement on real neuromorphic chips, the bound is derived for a single step rather than across time, and the trainable memory-load window is narrow. What is demonstrated is a clean, reproducible set of floors under a well-controlled probe and a matching bound; what is not is a large-scale, measured energy comparison. Notably, the author subjects the memory-only hypothesis to a test that refutes it, which is the kind of adversarial self-check that raises confidence in the parts that survive.

Why sparse spikes are not the efficiency story

The most repeated pitch for computing on living neurons is that biology is sparse and sparse is cheap, the twenty-watt brain doing what a data centre cannot. This paper cuts that pitch in two along a single mechanism. First, the energy value of sparsity is task-structural: a computation that must hold state and distinguish many contents forces a high firing floor on any system whose recurrent state is carried by spikes alone, so a biological network that communicated its working memory through spikes alone would face the same information-theoretic constraint as the paper's silicon models. Sparsity is not a substrate gift; it is available only when the task's representational load is low. That alone should retire the blanket claim that living spikes are inherently efficient.

The second half is the more interesting one, because the bound's escape hatch is precisely where tissue differs. The floor applies to spike-only communication, and living tissue is emphatically not spike-only: it carries information in sub-threshold membrane potentials, dendritic processing, graded transmitter release and neuromodulation, alongside spikes. That is the analog regime the paper says loosens the bound, which means the reconciliation of low biological firing with heavy memory is not magic efficiency but a different storage scheme, memory held in synapses, structure and analog state rather than forced through a fixed-width spike vector, backed by an enormous state width. The opportunity for organoid intelligence is to move its efficiency argument off spike sparsity and onto exactly these mechanisms and onto event-driven sensing, the two settings this paper identifies as genuine wins, and to treat the huge effective state width of tissue as the variable that lowers the floor.

The threat is a hype-correction that also functions as an obsolescence warning. Any organoid roadmap that advertises sparse spiking as its efficiency edge is now falsifiable, and it fails the moment the target computation carries real memory load, because the floor rises with load by a bound that holds for any spike-only recurrent system, wet or dry. Worse, the regime that escapes the floor, analog and sub-threshold state, is the regime that is hardest to read out and hardest to engineer with today's microelectrode interfaces, so the very property that would rescue the efficiency claim is the one the field cannot yet observe or control at scale. The honest position is that tissue may indeed sidestep the firing floor, but only through channels it cannot presently access, which converts a marketing slogan into a concrete research problem.

The bottom line

Established, under a controlled probe: feed-forward perception sparsifies to about five percent firing with no accuracy loss against its spiking baseline while its energy proxy rises rather than falls, a recurrent language model is pinned near fifty percent, and a spiking Transformer reaches two percent while paying a key-value memory wall, all matching an information-theoretic floor that rises with representational load and falls with state width.¹ Still open: whether the floors and the forty-five nanometre energy proxy hold at large scale and on measured neuromorphic hardware, and how tightly the single-step bound transfers across time. For organoid intelligence the load-bearing caveat is the finding, not a footnote: the bound assumes spike-only communication, so it retires the naive sparse-equals-efficient pitch for any memory-bearing task, while pointing at analog and sub-threshold state, together with a wider effective state, as the routes by which tissue might beat the floor.² The claim would be confirmed by measured energy on real chips tracking the bound, and challenged if a spike-only recurrent system were shown to carry heavy memory well below the predicted floor.