The General Purpose Neural Computer

The signature artifact of machine intelligence today is the language model: a system that samples every output from a distribution, cannot repeat itself across runs, and hides its behavior across billions of entangled weights. For open-ended prose those traits are tolerable and often welcome. For work that is fundamentally algorithmic - ordering a list, doing arithmetic, parsing input, carrying out a specified procedure to the letter - they are fatal. A ledger, a compiler, or a control loop that hands back two different answers to the same question twice is worthless.

The claim of this work is that the algorithmic regime deserves an artifact of its own, one that is neural yet not stochastic. A conventional computer draws a hard line between its program and the machine that runs it; a neural network erases that line, which is at once the source of its generality and of its unpredictability. The design here restores the line without giving up learning: retain a genuine discrete instruction set and a deterministic execution loop, but learn what each instruction means and make the loop differentiable. What emerges is a machine whose microcode is itself a trained network.

The machine is built to mirror a real register-and-stack computer. A configuration comprises a value stack, a linear memory, a bank of local registers and a program counter, and a fixed micro-sequencer drives it forward by repeating fetch, decode, execute, pin and advance. Each opcode is handled by a small decoder that emits a five-tuple microcode: the number of operands popped from the stack, the rule computing the pushed value, whether a local is written, the effect on the program counter and the memory effect. Learning the interpreter reduces to learning this finite table; because the weights are frozen and the program lives on the tape, one parameter set executes any program.

Two pieces are deliberately kept out of learning. Arithmetic - addition, subtraction, comparison, equality - is evaluated symbolically and exactly by an integer unit rather than approximated by a network, so the machine handles operands of arbitrary magnitude without error. The second is the pinning membrane, which at the close of every cycle projects each soft cell onto the nearest integer of the instruction set's lattice. Absent pinning, tiny per-cycle perturbations accumulate without limit; with it, as long as the single-cycle error stays within the lattice basin, exactness is reestablished at every step instead of merely being slowed down.

A paired ablation that feeds identical noise into a pinned and an unpinned arm sharpens the point. The unpinned datapath does not degrade gradually - it fails through tie-breaking, and it fails at any perturbation no matter how tiny. At exactly zero noise it is perfect, yet at 10^-12 it is already wrong half the time, each failure a control-flow error in which a loop guard that compares two exactly equal quantities is flipped by an infinitesimal nudge. Ordering and equality are decisions that jump rather than vary smoothly, so exactness has a genuine discontinuity at zero - an argument for pinning stronger than any story about drift.

The strictest bar for a learned interpreter is that instruction meanings emerge from execution alone, with no human-supplied labels. The paper casts this as a label-free identifiability theorem: given only before-and-after state pairs from a reference machine, together with a probe distribution that is 'discriminating' (any microcode not observationally equivalent to the reference is separated with positive probability), every opcode's semantics is recoverable, and an explicit sample-complexity bound quantifies how much data suffices. Distributions that violate the condition are shown alongside the exact confusions they cause - for example, when the top of the stack is never zero, a conditional branch becomes indistinguishable from an unconditional one.

Because the substrate is symbolic and the microcode space is finite, equivalence is decidable, and the paper decides it rather than estimating it. Encoding both the reference and the recovered interpreter for an SMT solver over 32-bit bitvectors establishes total equivalence for 40 of 41 opcodes across the entire unbounded state space - closed by a locality lemma and an opaque-tail encoding that discharges all stack depths of at least five in a single query. The lone exception, local.tee on an empty stack, is a real defect that only a proof could surface: no sample from a distribution that never yields an empty stack could ever expose it. Enumerating all 133,783 wrong (opcode, microcode) pairs settles every one with none unknown, of which 71 are proved genuinely observationally equivalent to the reference.

Running an array-summation program on a 1024-cell memory, the recovered interpreter is exact on 400/400 trials at every trip count from 2 to 1000, with the cycle count following the closed form 14n+10 exactly, and on a 4096-cell memory it remains exact out to n=4000 over 56,010 cycles.

The recovered 41-opcode interpreter then runs eight recognizable algorithms - Euclid's GCD, in-place bubble sort, an FNV-1a hash, bit-serial CRC-32, binary search, bignum addition with carry, a compiled two-counter Minsky machine and a Brainfuck interpreter - trace- and oracle-exact on 980/980 instances over 1,189,592 cycles. The Minsky machine converts the universality reduction from a written claim into an executed one, with the measured cost never exceeding the proved bound of 12T+1 cycles. At extreme scale the FNV-1a hash runs cycle-exactly out to 16,000,010 cycles on a 2^20-cell memory, every closed-form cycle law that held at a thousand cycles still holding with zero residual at ten million.

To turn the positioning against prior differentiable computers into a measurement, four learned architectures - a Neural Turing Machine, a Universal Transformer with adaptive computation time, and Transformers with rotary and with random positional encodings - were trained on the same traces under one protocol and graded by the same exact-sequence rule. The results table scores three arms by name: a Transformer, a Neural GPU and a Neural Turing Machine. At ten times the training length the recovered interpreter is exact across all six tasks while no learned arm crosses 0.99 anywhere.

The sharper result appears under iteration. The strongest learned arm, the Transformer, is exact on 0.819 of single steps - a competent step evaluator - yet 0/40 on whole executions at every trip count, because a per-step rate of 0.819 decays toward zero once compounded across twenty-three steps (the Neural GPU sits at 0.2792 per step and the Neural Turing Machine at 0.1840, both also 0/40 on whole runs). The recovered table is 4000/4000 on single steps and 40/40 on whole executions at 1175 states, including at memory sizes never seen in training. An interpreter is measured by how its steps compose, not by any single step in isolation, and that is exactly the axis on which the systems part. Because the learned content surviving to inference is merely a table of integer five-tuples, at inference the machine holds no neural network at all - confirmed by a single SHA-256 hash that is identical across 43 environment and dtype conditions.

A method that does not care about machine architecture is general; one welded to a single machine is not. The construction is carried to five instruction sets spanning four machine shapes: the complete WebAssembly integer MVP (104 opcodes, a stack machine), RISC-V RV32I and RV64IM (register machines), ARMv7-A with condition flags, predicated execution and a barrel shifter (109 opcodes), and the MOS 6502 accumulator machine with variable-length instructions (151 opcodes) - 476 opcodes in all. Each reference table is differentially tested against a production implementation before recovery, agreeing with unicorn on 1,080,000/1,080,000 ARM post-states and with py65 on 1,500,000/1,500,000 for the 6502, and across the five tables 1,052,266,133 wrong pairs are decided with none unknown.

The theorem's hypotheses are re-derived per shape, and the shapes disagree about what they demand: predication tightens the probe condition into one that no configuration can satisfy, while the 6502 dispenses with it entirely. The two halves of the procedure diverge honestly - on RV32I the search half transfers, producing a table trace-exact on 120 held-out programs, but the gradient half does not, and both outcomes are reported against pre-registered criteria. Grounding on real compiled code is measured head-on: the recovered WebAssembly table runs 1669/1669 real call-graph closures lifted from SQLite, zlib and a Rust binary over 554,522 instructions, agreeing with wasmtime throughout, and executes compiled kernels cycle-exactly on 3900/3900 trials.

The last and most suspect ingredient - the hand-designed probe distribution whose branches exist expressly to separate the hard opcodes - is deleted, and the semantics is recovered from program execution traces alone. From 200 programs generated by an ISA grammar (no labels, no microcode dictionary, no engineered states), recovery narrows 38 of 41 opcodes to their proved floor and machine-checks them totally equivalent. On eight real algorithms the outcome is a clean negative more instructive than the positive: 21 opcodes never execute at all and only 16 reach the floor, because real code keeps stacks shallow, never divides by zero, and mostly compares small non-negative integers. A corpus of genuine algorithms discriminates more weakly than a purpose-built sampler. In parallel, a decoder network trained purely by gradient on state pairs reaches 38/41 from the designed distribution - matching the 33 singleton identifications the consistency search finds - and 36/41 from program traces alone, showing that the microcode is learnable by gradient at full ISA-41 scale and not only by enumeration.

The paper is candid about its limits. The fifteen- and forty-one-opcode cores are Wasm-inspired but are not WebAssembly: branches take flat indices instead of structured label depths, memory wraps rather than traps, and a single global stack replaces a per-block frame, so 40% of otherwise-legal flat programs are not even type-correct Wasm. The learned content is confined to dispatch and state plumbing - the arithmetic is handed to a verified exact unit, itself compiled down to 799 logic gates. Pure-gradient recovery of the coordination-bound opcodes, synthesis through learned control flow, and folding the spectral addressing operator into the loop all remain open, each carried forward into an experiment whose criterion is fixed in advance.