Deep spiking nets that stay stable without batch normalization

What the work claims

Ma and colleagues at the University of Electronic Science and Technology of China and the Beijing Institute of Remote-Sensing Equipment claim that deep spiking neural networks (SNNs) can be trained to competitive accuracy without any activation normalization layer, by enforcing what they call signal homeostasis through the network topology and the trained weights themselves. This is a methods paper with a full experimental section, accepted at ECCV 2026, so the claims are backed by benchmarks rather than argument alone.

The headline numbers: on ImageNet, their IS-SNN with extended training reaches 68.05% accuracy on ResNet-34 at only 4 timesteps, where the same network without normalization collapses to 32.58%. On CIFAR-10, a normalization-free VGG-9 scores 92.91% where the naive no-normalization baseline scores 10%, effectively chance. The authors report a 96.4% reduction in FPGA lookup table resources for neuron implementations and a 15% increase in training throughput (401 to 461 images per second per GPU) with 17% less memory than a batch-normalized counterpart.¹

How it works

The paper opens with a diagnosis. In a deep SNN without normalization, firing rates decay catastrophically layer by layer: each spiking layer is a thresholding nonlinearity, and without statistics control the fraction of neurons that fire drifts toward zero as depth grows, until the signal is effectively gone. The authors show layer-wise firing rates for VGG-9 on CIFAR-10 and for a 152-layer residual network where the no-normalization baseline simply fails to recover from this decay.

The fix has two parts. First, topology-aware weight standardization: instead of normalizing activations at runtime, the weights of each layer are rescaled offline so that the variance of the signal propagating through the layer stays controlled, using constants derived from the network's own graph structure (a topology-dependent scaling term, plus an empirically estimated output variance for the spiking neuron type). Second, modified residual connections that counteract the residual-branch scaling which otherwise amplifies the decay. Crucially, all of this is folded into static weights before deployment. At inference time there is no statistics tracking, no running mean or variance, and no multiplication by normalization parameters: the datapath is pure accumulation, which is exactly what cheap spiking hardware wants.¹

The comparison points matter. Modern deep SNNs often use time-dependent batch normalization (tdBN) or its more expensive dynamic cousins, which require multiplications at every timestep. The paper's own ImageNet table shows why "we removed normalization" is not by itself a result: IS-SNN reaches 68.05%, but TEBN, a dynamic normalization variant, reaches 68.28%, and TET reaches 68.00%. The claim that survives scrutiny is narrower and more interesting than the abstract suggests: you can get within a fraction of a point of the best normalization-heavy methods while eliminating the normalization machinery entirely.

Where a skeptic should push

The most load-bearing assumption is that accuracy on static vision benchmarks transfers to the settings where normalization-free hardware actually matters. All the headline results are CIFAR and ImageNet classifications, in vitro for a network if you like: curated datasets, fixed weights, a frozen statistics regime. The authors are honest about this in their limitations section. Offline reparameterization assumes weights are fixed after training, so the method is aimed at inference-only deployment; online learning on the device would need statistics tracking back. And they note explicitly that the FPGA analysis covers arithmetic and resource costs for neuron implementations, not end-to-end energy: memory access, routing, and system scheduling are unaccounted. The 96.4% LUT figure is a neuron-implementation accounting, not a system-level energy measurement, and should be read as such.

Second, the "signal homeostasis" framing is a metaphor borrowed from neuroscience, but the mechanism is conventional engineering: variance-preserving weight scaling of the kind normalizer-free ResNets in the ANN literature already use, adapted to the discontinuities of spiking neurons. That adaptation is real work and the spiking-specific variance estimation is the novel piece, but the paper does not discover homeostasis; it ports a known trick across a gap where naive porting demonstrably fails (their own ablation shows naive weight standardization without the topology-aware term is insufficient at depth).

Third, on ImageNet the method as specified (without extended training) reaches 66.61%, below TEBN's 68.28%. The 68.05% figure requires the extended E400 training schedule. "Competitive with or superior to" is fair; "superior" alone would not be.

Organoid computing cannot run batch normalization

Here is the non-obvious implication, and it runs in both directions. The batch normalization that modern deep SNNs depend on is a global statistical operation: it requires a population of activations, running mean and variance estimates, and per-timestep multiplication by learned scale factors. None of that exists in a dish. A brain organoid has no batch, no normalization layer, no runtime multiplier; every regulatory mechanism that keeps its activity from collapsing or exploding is intrinsic to the tissue, implemented through the interplay of neuron-intrinsic conductances, synaptic scaling, and inhibitory feedback. Organoid intelligence was never offered the option that spiking deep nets have been quietly exercising.

What IS-SNN shows is that the silicon world is being pushed toward the same constraint. When the efficiency accounting got honest enough, the normalization crutch became the bottleneck, and the solution was to move the regulation into the substrate: bake the statistics into the weights, choose the topology so decay cannot accumulate, and demand that stability be a property of the trained system rather than a runtime service. That is, almost verbatim, the design brief that living neural tissue already implements. It strengthens the case that intrinsic stability is not a biological curiosity but a load-bearing requirement for any computing substrate that wants to scale in depth without a statistics server attached.

The threat cuts the other way. If silicon engineers can now strip normalization from deep spiking networks while keeping accuracy within fractions of a point, one of the implicit efficiency arguments for wetware gets weaker: the claim that biology is uniquely frugal because it spikes and self-regulates becomes less distinctive when a ResNet on an FPGA does something close. The honest energy comparison for organoid computing will have to be made against this new class of lean, reparameterized spiking hardware, not against normalization-bloated baselines. The authors' own caveat applies doubly here: until someone does the end-to-end system accounting for organoid platforms (media perfusion, environmental control, readout electronics), the efficiency ledger is open on both sides.

There is also a practical blueprint. For people building hybrid systems that train or steer organoids, the paper is a warning about importing training recipes wholesale: any method whose stability depends on global normalization statistics cannot be ported to tissue, because the tissue is the statistics. Closed-loop stimulation protocols for organoids will need the IS-SNN-style property: regulation that lives in the loop's local dynamics, not in a global observer.

The bottom line

Established: deep spiking networks can reach competitive accuracy on standard benchmarks with no activation normalization, by folding stability into the weights, with large claimed savings in neuron-implementation resources. Established within the paper's own evidence: the naive version of this idea fails badly, so the topology-aware formulation is doing real work. Still open: system-level energy on deployed hardware, online learning, and anything beyond static vision benchmarks. For organoid intelligence the paper's deeper message is architectural: stability is a substrate property, and the field that cannot borrow batch normalization has been forced to learn that lesson first. What would confirm the broader claim is the same method holding up on sequence and control tasks with deployed-system energy numbers; what would break it is evidence that the reparameterization only works because ImageNet-scale training statistics are effectively memorized into the weights.