Silicon just learned to spend one spike per neuron, and the wetware ledger shrinks

What the work claims

Zhao, Omidi, Jafari and Naud, with Naud affiliated with the University of Ottawa, claim the first language models built entirely on time-to-first-spike (TTFS) coding, in which each neuron emits at most one spike within a time window and all information is carried by spike timing.¹ Prior TTFS networks could only be mapped exactly from simple ReLU multilayer perceptrons; the components real language models are built from, embedding layers, layer normalization, attention and dropout, have signed or non-spiking structure that broke the mapping. The authors introduce a reference-time mechanism that lets a TTFS neuron represent signed values, then construct spike-native approximations for all four components, train the networks end to end, and scale them to 1.5 billion parameters, which they state is the largest TTFS-coded spiking language model to date.¹

The empirical claim is deliberately split. On natural language understanding and commonsense reasoning, performance is comparable to the equivalent conventional networks and to the best prior spiking models. On open-ended language modeling, measured by perplexity, a clear gap remains, and the authors say so themselves, naming the language-modeling gap and the absence of hardware measurements as the two open limitations. This is a systems-and-benchmarks paper, and its most valuable feature may be that accounting honesty.

How it works

TTFS coding encodes a value in the latency of a neuron's first threshold crossing: earlier means larger, with at most one spike per neuron per window, which makes firing rates extremely low by construction. The obstacle has always been sign and normalization. A conventional TTFS neuron's output timing is bounded below by zero and clipped at a maximum, so negative activations are unrepresentable, and layer normalization requires computing means and variances, operations with no spike-native form. The reference-time fix introduces a reference spike: values are encoded as timing relative to that reference, which recovers a negative range by letting spikes fall on either side of it, and the authors show an exact mapping between their reference-based TTFS layers and a shifted-ReLU linear regime, so linear layers convert losslessly.¹

Components that admit no exact mapping are approximated with fixed-weight spike circuits: the layer-normalization block is expanded into mean-computing, subtraction, and reciprocal-square-root approximator subnetworks whose parameters do not change during training, and attention operations and dropout are similarly expressed in timing logic. The result is a fully spiking pipeline in which, by construction, nearly every neuron spikes exactly once, and the reported spike budget for TTFS-BERT-Base is 14,811 spikes over 18,436 neurons per inference, about 0.80 spikes per neuron.¹

The benchmarks tell a two-part story. On GLUE, TTFS-BERT-Base averages 80.5, far above the early SpikeBERT conversion baseline at 65.1 and statistically in the neighborhood of the strongest prior spiking model, SpikeLM at 81.3; TTFS-BERT-Large reaches 82.3 against 85.3 for full-precision BERT-Large, a 3.0 point gap. On GPT-2, the pattern sharpens: TTFS-GPT-2 Small actually exceeds its conventional counterpart on commonsense reasoning (average 42.5 versus 40.7), and the 1.5-billion-parameter XL model stays close (49.5 versus 51.6), but WikiText perplexity degrades from 20.4 to 26.6, and on LAMBADA, the long-range cloze task, perplexity degrades from 10.6 to 23.8 and accuracy from 51.2 to 39.4. The authors' own diagnosis is that discretizing spike timings accumulates error across a long context, and because LAMBADA accuracy is itself a ranking metric, the ranking of candidate tokens is corrupted on exactly the tasks that hinge on one long-range dependency.¹

Energy is reported as a spike-count proxy, not a measurement: total spikes times a per-spike transmission cost from an established cost model, 0.80 spike-cost units for TTFS-BERT against 1.2 for SpikeLM and 3.2 for SpikingBERT, derived from those papers' firing rates and time steps. The authors explicitly exclude weight storage, off-core memory traffic, inter-core communication, and the timing circuitry needed to resolve spike latencies, noting that on digital asynchronous hardware these terms can dominate, and that their table compares spike activity between spiking models, not system-level energy against conventional networks.¹

Where a skeptic should push

The single most load-bearing assumption is that spike count is the right currency. The paper is exemplary in flagging that it is not: by excluding memory traffic, weight access, and timing-circuitry overheads, the headline energy comparison could be off by an order of magnitude in either direction, and on standard GPUs spiking models often cost more energy than dense ones. Until these models run on physical neuromorphic hardware with instrumented power, the efficiency claim is a hypothesis with a cost model attached, and this paper says so more clearly than most of the literature.¹

Second, the perplexity gap is not a rounding error, and it is mechanistically revealing. A near-doubling of LAMBADA perplexity at 1.5 billion parameters means timing quantization is eating exactly the capability, maintaining precise relative magnitudes across long contexts, that recurrent biological networks are hypothesized to be good at. If a designed, noise-free silicon implementation loses this much by quantizing time, a skeptical reviewer should demand strong evidence before believing a noisy biological substrate with stochastic spike timing preserves long-range dependencies better. Third, the commonsense wins come partly from task structure: short, well-separated candidate completions are robust to timing noise, so the benchmarks where TTFS shines are the least demanding tests of temporal precision. And as with all such scaling papers, the comparison baselines are imported from their own publications rather than rerun under matched conditions.

Single-spike silicon and the shrinking wetware efficiency case

The opportunity and the threat here are the same fact read in two directions. Silicon spiking networks have now reached the one-spike-per-neuron limit, the operating point biology itself lives at, while retaining most language-model capability. That validates the deepest assumption of organoid intelligence, that sparse temporal codes are computationally sufficient at scale, and it hands the field a mature, honest methodology: end-to-end training in the code, component-level spike budgets, and explicit separation of spike-count proxies from system energy.¹

The threat is that the efficiency justification for wetware narrows with every such paper. If the reason to compute on living tissue is that spikes are cheap, and a digital chip now spends 0.8 spikes per neuron per inference, the burden of proof shifts to whatever biology offers beyond spike parsimony: self-repair, learned plasticity, nanowatt metabolic operation, or computational properties no one has named yet. This paper also sets an evidentiary bar the OI field has not consistently met: it reports what degrades (long-range ranking), concedes what it did not measure (hardware energy), and refuses to claim parity where parity does not hold.

The non-obvious implication runs through the failure mode. Where single-spike silicon fails is maintaining fine timing distinctions over long contexts, because time is its only information axis and quantizing time quantizes everything. Biological tissue is not constrained to one spike per neuron, but its timing is noisy and drift-prone, so the same question applies in a different currency: what is the tissue's effective timing resolution, and does it hold over the seconds-to-minutes contexts that LAMBADA-like dependencies require? The reference-time trick itself is a transferable primitive: encoding a signed value as timing relative to a reference event is exactly the kind of coding scheme one could implement when writing into tissue with patterned electrical or optogenetic stimulation, and it is rare that a coding innovation in silicon suggests a write protocol for wetware. A well-designed organoid experiment following this paper's lead would probe precisely the tasks where TTFS silicon degrades, because that is where a biological win would be meaningful and where a loss would be informative.

The bottom line

Established: fully TTFS-coded language models are trainable end to end at up to 1.5 billion parameters; they match or beat prior spiking baselines on GLUE and commonsense reasoning, with TTFS-GPT-2 XL reaching 26.6 WikiText perplexity against SpikeGPT's 39.8 at 216 million parameters; and their spike-related proxy cost is below rate-coded spiking predecessors.¹ Not established: any measured system-level energy advantage, parity on open-ended language modeling (LAMBADA accuracy falls from 51.2 to 39.4), or behavior on physical neuromorphic hardware.

For organoid intelligence, this paper is both a validation and a reprioritization. What would confirm its relevance is wetware demonstrations evaluated on the same long-range-dependency tasks where single-spike silicon struggles, using the same honest accounting; what would weaken the field's position is the efficiency argument remaining its primary public justification while silicon closes the spike-sparsity gap at industrial pace.