Lapis: attention built from first-spike timing instead of dot products

What the work claims

The claim is that self-attention in spiking networks should be defined by spike timing rather than imposed on spikes. Existing spiking transformers inherit query-key scoring from dense networks, either simplifying the dot product or replacing it with masking, addition, or spike-gated operations. Lapis instead uses time-to-first-spike coding, in which each activation is represented by the latency of its first spike, and defines the relation between two tokens as the L1 distance between their latency vectors.¹ A Laplacian kernel converts the distance to an affinity, and the kernel's exponential decay is matched to the leakage dynamics of a leaky integrate-and-fire membrane.

The result is an attention path that needs no multiplication between query and key channels, only subtraction, absolute value, and accumulation, and whose row normalization reduces to a bit shift under power-of-two rounding.¹ This is a primary methods-and-results paper from a group at the National University of Singapore, evaluated on CIFAR-10, CIFAR-100, and ImageNet-1K, with accuracy and energy compared against representative dense, directly trained, and converted spiking vision transformers.

How it works

Time-to-first-spike coding represents each activation by the time at which a neuron first fires, with larger activations producing earlier spikes. For each attention head, Lapis takes the channel-wise first-spike latency vectors of a query token and a key token and computes their L1 distance. A smaller distance means closer agreement between the two first-spike patterns. The affinity is then exp(-D/tau), where D is the distance and tau is a per-head learnable temporal scale. A smaller tau causes faster decay with temporal distance, a larger tau permits larger timing differences.¹

The exponential form is deliberate. The subthreshold dynamics of a leaky integrate-and-fire membrane decay exponentially with time, so the same physical process that determines when a neuron fires can also implement the attention affinity. For each query, the affinities over all keys are accumulated, the normalization factor is quantized to a power of two, and the final scaling becomes a bit shift in fixed-point arithmetic. The value aggregation step still uses standard projections, but the query-key score itself is spike-native.

The authors convert integer activations to TTFS spikes after training. On ImageNet-1K, models are initialized from a pretrained BEiT vision transformer and fine-tuned for 60 epochs before a short quantization-aware fine-tuning pass. The Lapis-S variant for CIFAR uses a 15-step TTFS window; Lapis-B and Lapis-L on ImageNet use 15-step and 20-step windows respectively.¹

Where a skeptic should push

The most load-bearing assumption is that time-to-first-spike coding captures enough information. The method discards every spike after the first within the window, so all temporal structure beyond initial latency is thrown away. The CIFAR-10 ablation shows that a Hamming relation, which retains only binary match between latencies, drops accuracy to 87.92%, while the graded Laplacian L1 distance reaches 96.56%.¹ That validates graded distance, but it does not validate using only the first spike.

Second, the energy numbers are estimates from a 45 nm operation-level model, not silicon measurements. The attention-path energy of 100.01 mJ for dense dot-product attention versus 6.92 mJ for Lapis is a large ratio, but it depends on the accounting protocol and excludes data movement, memory, and the rest of the inference pipeline. The full-model estimates, 13.20 mJ for Lapis-L W32 and 3.28 mJ for W6, are more representative of deployment but still modeled, not measured.

Third, the ImageNet models start from a pretrained BEiT encoder. The accuracy therefore reflects how well Lapis can approximate an existing attention mechanism, not whether TTFS attention can learn powerful representations from scratch. The fair comparison is against conversion-based methods, and Lapis matches SpikeZIP-TF to within 0.57 points while using a shorter temporal representation.¹

Fourth, the accuracy gap to dense attention is small but real. On CIFAR-10 the softmax dot-product baseline reaches 97.09%, while Lapis reaches 96.56%, a 0.53 point gap. Whether that gap is acceptable depends on whether the energy reduction is large enough to justify it, and that trade-off will vary by application.

What timing-based attention means for biological computing

The non-obvious implication for organoid intelligence is that spike timing can define the computational primitive, not merely encode a value computed elsewhere. In most neuromorphic systems, spikes are outputs of a neuron model that approximates a continuous activation. Lapis treats the first-spike latency as the native representation and builds the relation function out of the same leakage dynamics that generate the spikes. For a biological substrate, this is the difference between using spikes as a readout and using spikes as the computation itself.

The specific mechanism is the Laplacian kernel. Because exp(-D/tau) matches the impulse response of a leaky integrate-and-fire membrane, an array of neurons receiving query and key spike trains can implement the affinity by letting one train reset a membrane and the other sample the resulting decay.¹ That is not a metaphor; it is a direct mapping from the attention score to a biophysical process. In an organoid, one could in principle deliver spike trains representing query and key latencies through patterned stimulation and read the resulting membrane potentials or firing probabilities as attention weights. No one has demonstrated this, but Lapis provides the algorithmic template.

The opportunity is a move from rate-coded to timing-coded organoid readout. Most current organoid interfaces treat firing rate as the signal and time as noise. Lapis suggests that latency structure is itself information-rich and can be exploited for relational computation. If organoids can be taught, or wired, to produce latency-coded tokens, then attention-like selection could emerge from the same membrane dynamics that already exist in the tissue, without a digital attention layer bolted on top.

The threat is fragility. Time-to-first-spike coding is sensitive to latency jitter, missing spikes, and baseline firing-rate drift, all of which are first-order phenomena in living tissue. The 0.53 point accuracy gap on CIFAR-10 in a controlled simulation could become much larger when latencies are perturbed by biological variability. The paper's energy advantage also assumes clean single-spike latencies; in a noisy culture, the same coding scheme might require averaging over many trials, erasing the efficiency gain.

A second threat is representational poverty. Throwing away all spikes after the first within a window is a severe compression. Biological neurons often use spike count, burst structure, and precise inter-spike intervals to convey information. A timing-only attention mechanism may therefore be a better fit for engineered silicon than for living tissue, even though both are spiking. The correct organoid analogue may not be Lapis as written, but a generalized version in which attention weights depend on a richer temporal feature vector.

The bottom line

Established: a spiking attention mechanism can be built directly from first-spike latency vectors and leakage dynamics, achieving ImageNet-1K accuracy above 83% with a large estimated energy reduction over dense attention, and staying within 0.53 points of dot-product scoring on CIFAR-10. Not established: that the energy numbers hold in silicon, that single-spike latency coding is robust enough for noisy biological tissue, or that the same idea can learn representations from scratch without a pretrained encoder.

What would strengthen the claim: a silicon implementation measuring real energy and latency, and a robustness study adding jitter and spike failures to the TTFS coding. What would break the biological application: showing that biological latency variability destroys the accuracy advantage, which would mean timing-based attention is better suited to clean neuromorphic silicon than to living neural tissue. For organoid intelligence, Lapis is a blueprint for how attention could be implemented by membrane dynamics, but the blueprint needs biological stress-testing before it can be called a substrate match.