The General Purpose Neural Computer

The dominant computational artifact of the moment is the language model: a system whose every output is sampled from a distribution, cannot be repeated across runs, and is smeared illegibly across billions of weights. For open-ended prose these properties are tolerable and often welcome. For work that is fundamentally algorithmic - sorting, arithmetic, parsing, the faithful execution of a specified procedure - they are disqualifying. One does not want a ledger, a compiler, or a control loop that returns a different answer when asked twice.

The thesis of this work is that the algorithmic regime deserves an artifact of its own, one that is neural without being stochastic. A classical computer keeps its program distinct from the machine that runs it; a neural network collapses that separation, which is at once the source of its generality and of its indeterminism. The proposal restores the separation without abandoning learning: retain a real, discrete instruction set and a deterministic execution loop, while learning the meaning of each instruction and rendering the loop differentiable. The result is a computer whose microcode is a neural network.

The machine mirrors a real register-and-stack computer. A configuration is a tuple of a value stack, a linear memory, a bank of local registers and a program counter, and a frozen micro-sequencer drives it forward by repeating fetch, decode, execute, pin and advance until a learned halt. For each opcode a small decoder emits a five-field microcode: how many operands are popped from the stack, how the pushed value is computed, whether a local register is written, the effect on the program counter, and the memory effect. Learning the interpreter is learning this finite table; because the weights are frozen and the program lives on the tape, one parameter set runs any program.

Two components are deliberately withheld from learning. Arithmetic - addition, subtraction, comparison, equality - is computed symbolically and exactly rather than approximated by a network, so the machine generalizes to operands of any magnitude with no error; the network learns where operands are, which operation applies, how control flows and when to halt, not what a sum is. The second withheld component is numerical stability, supplied by the pinning membrane, which after each cycle projects every soft cell onto the nearest element of the discrete symbol lattice. Without pinning the per-cycle perturbations compound without bound; with it, provided the single-cycle error stays within the lattice basin, the projection restores exactness every cycle, so the error recurrence is broken at every step rather than merely damped.

The mechanism that confers length generalization is how the machine addresses memory. Rather than a fixed positional lookup, the read and write heads move by a learned shift operator: the soft address advances by circular convolution with a decoder-emitted kernel over the relative displacements minus-one, zero and plus-one. The essential property is that the operator's only parameters are those three taps, independent of the memory size N. A positional table encodes 'the value is at index seven' and is silent at index seven hundred; a shift operator encodes a size-independent rule and therefore transports unchanged to inputs of any length. By the convolution theorem the operator is diagonalized by the discrete Fourier transform, exactly as a Fourier neural operator applies a layer in the spectral domain.

The invariance is measured, not merely argued. On a pointer-walk task supervised only on the read value, the spectral operator reproduces the integer circular shift bit-exactly - a maximum discrepancy of 2.2x10^-16, machine precision - and an addressing rule learned at N=16 reads with perfect exactness at sizes it never saw, including N=64, 256, 1024 and 4096. The same operator serves as a token mixer for sequence models: on delayed copy trained at length 32, the offset-domain operator is exact at every length to 512, while a conventional learned dense mixing table collapses to 11.5% near chance. The paper records a sharp negative result alongside: a free-modes Fourier operator that learns an independent multiplier per mode fails out of distribution at 7.7%, because a shift's spectral response depends on length at fixed mode index. Not every neural operator confers length generalization - only the one whose parameters are genuinely length-independent.

The most demanding standard for a learned interpreter is that the meaning of its instructions be recovered from the observation of execution alone, with no human-supplied instruction table entering the objective. The paper formalizes this as a label-free identifiability theorem: given only single-step transitions produced by a reference machine, and a probe distribution that is discriminating - one that, for every opcode and every wrong candidate microcode, assigns positive probability to some configuration on which they differ - label-free recovery returns the reference microcode for every opcode of the fifteen-opcode WebAssembly integer core. The discriminating condition is exactly what excludes the degenerate case in which, say, a conditional branch is only ever seen with a true condition and is indistinguishable from an unconditional jump. The result is an exact, finite recovery guarantee rather than an identifiability-up-to-equivalence statement.

Recovery over the finite microcode space realizes the theorem in practice, returning all ten stack opcodes and all fifteen opcodes of the full instruction set with no labels, and the recovered interpreter is bit-exact on held-out configurations with operands as large as 10^6. Fully differentiable recovery - learning the microcode by gradient descent through a soft substrate rather than by search - is harder and its analysis is instructive: a single gradient run recovers only five to six of the ten stack opcodes, because the comparison opcodes carry almost no gradient and the select opcode requires two microcode fields to be set correctly at once. Two label-free measures close the gap. Selecting, per opcode, the best of several independent restarts by held-out bit-exact score raises recovery to nine of ten, and a restricted exact search over the residual opcode's seventy-two-candidate space yields the full ten of ten at per-step exactness of 1.000.

The payoff of a recovered interpreter is that it runs real, unbounded programs deterministically. Validation proceeds in self-contained stages, every exactness figure a bit-exact comparison against a reference oracle: shift-operator addressing on sequence transduction, a learned ten-opcode stack machine bit-exact across held-out lengths and operand magnitudes to 10^6, and then structured control flow with linear memory, which supplies real loops. An array-summation program and a countdown program execute bit-exactly and generalize to loop trip-counts from sixteen to one thousand after training on trip-counts of at most eight - acquiring an algorithm, not a table.

As a concrete demonstration, the recovered Turing-complete interpreter computes the sum of a specific eight-element array as thirty-one, correctly, in one hundred and twenty-two deterministic cycles, with the program counter looping and the accumulator accreting each loaded element. Universality itself is argued honestly by reduction: a GPNC with the WebAssembly integer core, an external discrete memory of extendable size, and an unbounded cycle count is Turing-complete in the finite-precision sense, with unboundedness carried by the discrete external memory rather than smuggled into the mantissa of a real-valued activation. The paper is explicit that network universality is settled prior work and not the contribution; the contribution is the intersection - learned, grounded, length-generalizing, and deterministic at once, with a label-free identifiability guarantee.

Because the recovered machine's arithmetic is a fixed-precision integer function over a finite alphabet, it is by the founding theorem of digital design a Boolean function, hence a logic circuit. Using the verified gate compiler of the companion binary-circuit work, the paper compiles each arithmetic and logic opcode of the core into a gate-level netlist and checks it bit-exactly against the reference interpreter: i32.add to a 160-gate ripple adder, i32.sub to 192 gates, i32.eq to 63, i32.lt_u to a 225-gate comparator, and the ternary select to a 159-gate multiplexer - the entire data-plane in 799 verified gates, correct exhaustively at four bits and on random inputs at the full thirty-two. The learned control that recovers the microcode and the compiled circuit that computes it are two views of one object.

The paper is equally precise about what remains open, treating the items as a roadmap rather than a wall. Three matters are named: pure-gradient recovery of the coordination-bound select opcode without any search fallback; program synthesis through learned loops and branches, which needs a smooth relaxation of control flow followed by discretization and exact verification; and grounding on real compiled machine code, via a harness that traces a reference interpreter over programs compiled to WebAssembly. A tractability map frames the last of these: the instruction set validated here is real but deliberately small, and the authors argue the identical method should transfer to the fixed-width RISC-V integer base as the natural next target, with variable-length, flag-bearing instruction sets such as x86-64 left as the research frontier.