The cart, the pole, and the learner, all on one chip

What the work claims

This is a systems paper with a hard, falsifiable boundary claim: the entire reinforcement-learning loop, the Q-learning update, action selection, reward and state handling, and the nonlinear physics of the CartPole-v0 environment itself, executes on one Intel Loihi 2 neuromorphic processor using Lava microcode, with no host computation between environment transitions¹. Prior work had put RL policies or value networks on neuromorphic chips while keeping the environment and much of the bookkeeping on a conventional computer; the authors' stated contribution is closing that boundary. Their Q-learning circuit trained the same absolute number of successful agents as a CPU implementation in 51 percent of the wall-clock time, with measured energy efficiency of over 1000 times the CPU's for equivalent training success.

The result is deliberately anchored to a toy problem. The state space is a 72-bin one-hot encoding of cart position, cart velocity, pole angle and angular velocity, the action space is binary, and the value function is a literal Q-table of 8-bit weights rather than a function approximator. The authors say plainly that they kept the standard CartPole-v0 formulation because the objective is not state-of-the-art control but a test of whether the complete loop can execute on a single asynchronous spiking substrate.

How it works

The engineering heart of the paper is a nine-step algorithmic loop synchronized by a synfire-gated gating chain. Synfire gating is a technique for routing information through a spiking network in discrete, ordered phases: a pulse of synchronous activity opens a functional pathway for one step, then the next pulse opens the next. Because a globally clocked step counter does not exist on Loihi 2, the authors chain these gated phases so that the CartPole state update, the Q-table lookup, the greedy or random action choice, the temporal-difference target computation and the weight update each happen in their own step, with relay neurons holding values between steps. Reward is delivered as a scaled rate signal, multiplied by 8 to give plus or minus 8, and the exploration rate and learning rate are themselves computed on-chip from a running episode score, with training halting early when an episode score of 500 is reached.

The environment is the part that forces compromise. CartPole physics needs sine, cosine and division, none of which Loihi 2's microcode instruction set provides, so the authors approximate: sine by a straight line with maximum absolute error 0.00229, cosine by a quadratic with error 0.00020, and division by Newton's method on the reciprocal with error 0.00291, compounding to a maximum error of 0.03632 rad/s² in pole angular acceleration. Random initialization uses a linear congruential generator with period 2^23 rather than the Mersenne Twister, called at most 400,000 times per training run. Everything runs in 24-bit fixed point where the CPU comparison runs 64-bit floating point. These are not footnotes; they are the reason for the headline gap in the results.

Where a skeptic should push

The success ceiling is the load-bearing number, and it is lower on the neuromorphic side. Training 1,000 agents on each platform, 100 agents at each of ten weight-update counts from 10,000 to 100,000, the CPU reached a maximum success rate of 62 percent at 90,000 updates while Loihi 2 peaked at 36 percent at 50,000 updates and plateaued near 30 percent thereafter. The authors are admirably direct about the attribution: 24-bit fixed point versus 64-bit floating point, trigonometric accuracy of 2 to 3 decimal places versus 14 to 15, and a weaker random number generator. Their reframing, that Loihi 2 trained the same number of successful agents in half the wall-clock time, is honest arithmetic rather than spin, since the energy-per-successful-agent comparison is the one that matters for embedded applications. But a reader should hold both facts: the substrate learned, and the substrate's own numerical imperfections halved its ceiling on a task where the correct answer is fully known.

Three more cautions. First, the energy accounting: the Loihi 2 board draws 1.78 W of its 1.80 W training total as static power, so the celebrated advantage is about dynamic power, 0.02 W against the CPU's 12 W, roughly 600 times less, compounding with a 2.8 times speedup per weight update into the over-1000-fold energy-per-action margin in Table II. Whole-board energy per action is 104.75 microjoules against 7,024.44 for the CPU, but almost all of the Loihi figure is idle silicon, which would not scale down if you tiled many boards. Second, the closed loop is closed against an approximate world: the agent learns to balance a pole in physics that has been deliberately degraded, which is fine for a feasibility demonstration and says little about coupling to a real plant. Third, the comparison CPU is one laptop chip and the GPU datapoint, three orders of magnitude more energy than Loihi 2 at 67 percent success, is offered by the authors themselves as an artifact of library inefficiency, not a fair fight.

The closed loop organoids cannot build alone

The non-obvious implication for organoid intelligence is that this paper defines, with unusual precision, the half of the closed loop that living tissue can never own. An organoid has no world inside it. Whatever an organoid culture learns to maximize, the task, the reward and the state transitions must be supplied from outside, by silicon. What the Los Alamos group shows is that this external partner need not be a GPU server on an unconstrained power budget: the entire scaffold, a phased gating mechanism, reward routing as a scaled spike-rate signal, on-the-fly rate adaptation, and a weight-update stream sustaining over 17 kHz, fits in 1.8 W with headroom against the 100 to 1,000 Hz sensing and actuation rates of real control tasks. For chronic in vitro reinforcement learning, where the training session lasts weeks and the culture incubator, electrode array and closed-loop controller must run continuously in the same room, that power and latency envelope is the difference between a plausible apparatus and a datacenter fantasy.

The second implication cuts deeper, because it gives the field a quantitative frame for its favorite excuse. Biological computing proponents, including this site, routinely argue that tissue is noisy, analog and imprecise, and that this is a feature or at worst tolerable. Here is a substrate with exactly specified imperfections, 24-bit arithmetic, 0.00229 maximum sine error, a 2^23-period random generator, and the measured cost of those imperfections is a learning ceiling of 36 percent against 62 percent, with learning itself fully intact and convergence actually faster early on. That is the kind of number the organoid field has never produced for its own substrate: nobody has calibrated how much of a culture's failure to learn in a DishBrain-style loop is attributable to interface noise, imprecise reward delivery or uncontrolled variability, because nobody has a noise-free reference. The paper's architecture is a template for getting one: run an identical loop against a simulated agent and against tissue, and the performance gap becomes a measurement of the tissue's effective precision, not a rhetorical question.

The threat is the steadily rising floor. A single neuromorphic chip now executes the complete sense-decide-learn-act cycle, world model included, at millisecond latency and microwatt-class dynamic power. Any claim that organoid computing earns its keep on closed-loop tasks must now clear a bar set by this result: the culture must contribute something a one-hot Q-table on Loihi 2 cannot, which in practice means a continuous, high-dimensional, self-organizing state representation, since discrete toy tasks are now silicon's home turf. The opportunity is symmetric: the gating-chain mechanism is exactly the kind of algorithm scaffold that could orchestrate stimulation protocols for tissue, sequencing sense, target computation and neuromodulatory delivery in phases a living network can absorb, with the silicon partner small and cool enough to live beside the incubator instead of behind it.

The bottom line

Established: a complete Q-learning loop, including the CartPole environment, runs on Loihi 2 with no host in the cycle, and the on-chip agents learn successfully, up to 36 percent of 1,000 trained agents solving CartPole-v0, with the best agent reaching a perfect mean score of 200 after 40,000 updates. Established, with the caveat of static-power dominance: whole-board training energy of 104.75 microjoules per action against 7,024.44 on the comparison CPU, and half the wall-clock time per successful agent. Not established: transfer beyond a discretized toy task, robustness of the approximate on-chip physics, or any advantage over conventional hardware once function approximation and continuous state spaces arrive. What would confirm the approach: the same fully on-chip loop on a task with continuous state and a learned value function, where the precision story either closes the gap or quantifies it. What would break it: evidence that the synfire-gating overhead and fixed-point ceiling grow with task complexity faster than the energy advantage does. For organoid intelligence, the durable takeaway is the specification: a self-contained learning loop fits in watts, moves at kilohertz, and pays a measurable, survivable price for substrate imprecision. Tissue now has a number to beat.