Research analysis · Theory of computation

How much can a single-spike network express?

The sparsest serious proposal for neural computation, biological or silicon, is the single-spike code: every neuron fires at most once, and the answer is read out in spike times. It is energy-frugal, hardware-friendly, and intuitively fragile, but nobody had capped what it can represent. Dominik Dold, Dennis Elbrächter and Philipp Petersen of the University of Vienna now do: the input space of a feedforward single-spike network decomposes into a countable number of causal pieces, and they prove upper bounds on that count ranging from polynomial in network width to exponential in depth times bottleneck dimension, with one neuron model, the non-leaky integrate-and-fire cell with instantaneous synapses, escaping depth dependence entirely. Because the number of causal pieces lower-bounds approximation error, these counts are not curiosities; they are price tags on what sparse-spike computing can compute.

Source: Upper bounds on the number of causal pieces in spiking neural networks, arXiv:2610.22484, October 2026. Primary source. Read: the full 25-page PDF including the combinatorial bound via lonesum matrices, the hyperplane-arrangement bounds of Section 6, Table 1 with the asymptotic orders, and the lower-bound construction of equation 9.

What the work claims

This is a pure mathematics paper, proofs only, no experiments, and it should be weighted as theory: it limits worst-case behaviour, it does not measure typical networks. A causal piece is a region of input space on which the same subnetwork is active, the same collection of neurons and synapses through which the input spikes reach the output. Prior work by Dold and Petersen showed the piece count lower-bounds the uniform approximation error of a target function by a constant over the square of the piece count, so any upper bound U on pieces immediately converts into an approximation floor of constant times U to the minus two.1 This paper supplies the missing ceilings.2

The verified results, in asymptotic order with N the maximum layer width, L the depth, and N0 the input dimension. A model-independent combinatorial argument bounds pieces by a product of poly-Bernoulli numbers, of order e to the 2(L minus 1)N log N plus order N, improving on the naive count of two to the total synapse number by exploiting a nesting constraint: within a layer, a neuron that spikes later must carry a causal set containing all neurons that spiked earlier, which is equivalent to the active-connection matrices being lonesum matrices, a class with exact enumeration. For specific neuron models, hyperplane-arrangement counting gives sharper bounds: order N to the (4L minus 3)(N0 minus 1) for non-leaky integrate-and-fire neurons with exponential synapses and for a ReLU-type spike response model; order N to the L(N0 minus 1) for leaky integrate-and-fire neurons with Dirac-delta synapses; and, most strikingly, a bound depending only on the input dimension, with no dependence on depth or width at all, for non-leaky integrate-and-fire neurons with Dirac-delta synapses. The bounds hold for arbitrary-signed synaptic weights; no Dale's-law constraint is needed.

How it works

The counting rests on two structural observations. The first is temporal: for any fixed input, ordering a layer's neurons by their finite spike times produces nested causal sets, because a neuron that fires later has strictly seen everything earlier neurons saw. That forbids many synapse activity patterns outright and collapses the naive subset count into the lonesum enumeration, giving the general bound. The second is geometric: for the specific models covered, a change of variables in spike-time space makes each causal piece a polyhedral region, so counting pieces becomes counting regions of a hyperplane arrangement, the same classical technique used for ReLU networks, with the authors' refinement that they propagate the reachable dimension layer by layer rather than assuming full rank, which tightens the count whenever a bottleneck layer crushes the representation into fewer dimensions.

The sharp model-dependence is the intellectually load-bearing part. With exponential synapses, every neuron's threshold crossing is a smooth competition among decaying kernels, and the arrangement argument yields the steep depth price, exponent 4L minus 3. With instantaneous synapses the picture simplifies: the leaky model still pays depth linearly, exponent L, but the non-leaky model's spike times are pure sums of input times once thresholds are crossed in a fixed order, so the input weak order fixes everything, and the count collapses to the number of orderings of N0 inputs, the ordered Bell number, independent of everything downstream. The authors also prove these prices cannot be avoided in general: iterating a two-layer block construction with bottleneck dimension d across K blocks, with depth 2K, realises at least 2 to the (d+1)L/4 distinct pieces, so no general bound can be sub-exponential in d times L, and for the Dirac-synapse models the lower bounds match the upper bounds up to constants.

Where a skeptic should push

The gap between lower and upper bounds is the honest weakness, and the authors say so: the exponential-in-depth lower construction matches for Dirac synapses, but for exponential synapses the exponents, 4L minus 3 above against (d+1)L/4 below, are not shown optimal, so the true price of depth in the biologically common regime remains open. The error conversion also carries conditions a casual reader will miss: the c times U to the minus two floor holds for smooth targets meeting a non-affinity condition and says nothing about arbitrary functions, and the constant absorbs target-dependent factors that can be large. Worst-case bounds can be wildly pessimistic for trained networks, which is the standard caveat in the ReLU-region-counting literature this work extends; nothing here measures what a trained single-spike network typically realises. Finally, the setting is feedforward with shared presynaptic spike times per layer for the general bound, single spike per neuron, no recurrence, no noise, no finite time resolution; the authors themselves note that the lower-bound construction's reliance on shrinking relative spike-time intervals may limit its practical realisation on hardware with timing jitter. The paper also discloses that an AI assistant was used iteratively for brainstorming, formalisation assistance, and drafting, with all statements and proofs verified by the authors.

Sparse readouts and organoid capacity claims

For organoid intelligence the paper prices the readout code the field keeps flirting with. If computation on living tissue is ever to be energy-competitive, the readout must be sparse, and the sparsest natural code is time-to-first-spike: each output unit contributes one spike, and the answer lives in the timing. Dold and colleagues do not study organoids, but their counting arithmetic applies to any substrate that computes this way, and its message is two-sided. First, sparsity is not free. Approximation error decays only as the inverse square of the piece count, so a single-spike organoid readout that can distinguish U input regions pays at least c over U squared in accuracy no matter how it is trained, and architectures near the exponential bounds pay enormously for depth and bottlenecks. If a biological computing proposal implicitly assumes a handful of well-timed spikes carry rich computation, these are the numbers that discipline the claim.

Second, and this is the non-obvious implication, the expressivity tax is set by substrate physics, not by the training algorithm. The steep bound belongs to leaky models with exponential synapses, which is what real neurons mostly are; the depth-free bound belongs to a leakless, instantaneous idealisation that no tissue implements. In other words, the biologically realistic corner of model space is exactly where every layer costs the most pieces, and where bottlenecks, the compressed bottlenecks that recurrent tissue is full of, are taxed at the maximum rate the lower bounds allow. For organoid readout design the practical reading is architectural: buy capacity with width, whose cost is polynomial, rather than with depth, whose cost can be exponential; avoid forcing representations through narrow bottlenecks when a function must be approximated finely; and treat the synaptic kernel and leak of the preparation as computational parameters that set the ceiling, since a substrate with faster, sharper effective synapses lives in a cheaper counting class than one with slow exponential filtering.

The threat cuts the other way too. These bounds bind feedforward single-spike computation; recurrence, the native mode of organoid tissue, escapes the counting entirely, which means it also escapes every guarantee the counting provides. A recurrent organoid readout is not thereby more powerful in any provable sense, it is simply unbounded in both directions: neither the approximation floor nor any architecture guidance applies. That is the hype-correction worth printing on the wall: when a biological-computing result shows impressive sparse-readout performance, the first question is which side of these bounds its architecture sits on, and when a result relies on recurrent dynamics, the honest claim is that its capacity is unknown, not that it is large. The opportunity is that the nesting insight, spike order must induce nested causal ancestry, is a property any physical spike system shares, so a wetware analogue of causal-piece counting, regions of stimulus space distinguished by which cells drive the readout, may one day be measurable in an organoid with patterned stimulation. That would convert an organoid from a black box into a counted object, which is a prerequisite for engineering it.

The bottom line

Established, as theorems with matching lower bounds in the Dirac-synapse cases: single-spike feedforward networks have at most poly-Bernoulli-countable causal pieces, order e to the 2(L minus 1)N log N for general models; order N to the (4L minus 3)(N0 minus 1) for non-leaky integrate-and-fire with exponential synapses and ReLU-type response; order N to the L(N0 minus 1) for leaky integrate-and-fire with instantaneous synapses; and input-dimension-only for non-leaky integrate-and-fire with instantaneous synapses; with exponential growth in bottleneck dimension times depth shown unavoidable. Not established: tightness of the exponential-synapse exponents, typical rather than worst-case behaviour, anything with recurrence, noise, finite timing resolution, or multiple spikes per neuron. For organoid intelligence the durable lesson is that the sparse-readout programme has a structural accuracy tax set by substrate dynamics, width is the cheap resource and depth the expensive one, and recurrent tissue's capacity is not large but unquantified. The confirming result would be an experimental sparse-readout organoid whose measured error tracks a counting-based floor as architecture scales; the breaking result would be evidence that biological noise and leak push real preparations so deep into the expensive regime that single-spike codes cannot reach useful accuracy at tissue scale.

Frequently asked questions

What is a causal piece?

A region of input space in which the same subnetwork, the same set of neurons and synapses, carries the input spikes to the output. Single-spike networks partition their input space into such pieces, and the number of pieces quantifies how many distinct input-response behaviours the network can exhibit.

Why does the piece count matter for approximation?

Prior work showed that approximation error of a smooth target is lower bounded by a constant divided by the square of the piece count. Fewer pieces means a coarser partition of input space, and hence a provable floor on how closely any function in that class can be approximated, regardless of training.

What is the nesting constraint?

Within a layer, a neuron firing later must have a causal set containing all causal sets of neurons that fired earlier. This forbids most synapse activity patterns, is equivalent to the active-connection matrix being a lonesum matrix, and is what turns the naive exponential count into the poly-Bernoulli bound.

Which neuron model escapes depth dependence?

Non-leaky integrate-and-fire neurons with Dirac-delta synapses. Their spike times are determined by the weak order of input spikes alone, so the piece count reduces to the ordered Bell number of the input dimension, independent of network depth and width. Leaky models and exponential synapses pay polynomial-in-width but exponential-in-depth prices instead.

Are these bounds relevant to recurrent networks?

Not directly. The bounds assume feedforward architecture with at most one spike per neuron per inference. Recurrent dynamics escape both the upper bounds and the approximation floor, so their capacity is unquantified by this framework rather than provably large.

What does this mean for biological computing?

That the sparsest readout code has a structural accuracy tax set by the substrate's leak and synaptic dynamics, that width is cheap while depth and bottlenecks are expensive, and that claims about computation with very few spikes should state which counting regime the architecture occupies. Tissue-like leaky dynamics with slow synapses sit in the most expensive regime the paper identifies.

References

  1. D Dold, P C Petersen. Causal pieces: analysing and improving spiking neural networks piece by piece. arXiv:2504.14015. 2025. https://arxiv.org/abs/2504.14015. Accessed 2026-11-21.
  2. D Dold, D Elbrächter, P Petersen. Upper bounds on the number of causal pieces in spiking neural networks. arXiv:2610.22484. 2026. https://arxiv.org/abs/2610.22484. Accessed 2026-11-21.