GCPU: A CPU-First Training Runtime and Morpheus, a Categorical Deep-Learning Language

The system starts from a blunt premise recorded in its own design conversation: GPUs are scarce and expensive, and most people who want to train models cannot afford them. The authors reject the naive framings (use a graphics API, pretend a CPU has hidden tensor cores) and settle on a sharper thesis carried throughout the specs: on a CPU, training is bottlenecked by bytes moved before it is bottlenecked by floating-point throughput. The winning move is therefore not to imitate CUDA but to redesign the workload so CPU economics make sense, borrowing discipline from game engines: hot loops, cache-sized tiles, persistent job systems, asset streaming, and aggressive reuse.

This yields two distinct but linked artifacts. Forge is the systems answer: a runtime that makes a commodity CPU behave like a training device, optimizing for resident bytes per parameter update rather than raw GEMM throughput. Morpheus is the semantic answer to a second frustration voiced in the specs, that deep learning is practiced as alchemy: it is a language in which the type system is category theory, so that an architecture is meant to be checked, and in places proven, to respect the structure of its data before any training run consumes a CPU cycle.

Forge presents itself to PyTorch as a real device. It calls torch._C._rename_privateuse1_backend("gcpu") so that tensors move with .to("gcpu"), device "gcpu:0" maps to socket 0, and torch.gcpu.memory_allocated() reports usage. A GCPUTensor subclass implements __torch_dispatch__, intercepting every ATen operation; in the form the architecture doc shows, it unwraps GCPUTensor arguments to plain CPU tensors, executes the op (or routes to a C++ kernel), and rewraps the result, giving tensors a genuine gcpu device type while compute stays on the CPU until the native extension takes the hot paths. On top of this sit CUDA-shaped abstractions with no GPU underneath: ForgeStream, ForgeEvent, and ForgeDeviceGuard backed by a CPU thread pool, plus a NUMA-aware allocator that gives each socket its own arena so tensors are allocated near the thread that will compute on them.

Kernel dispatch is chosen at initialization by ISA detection: AVX-512 or Intel AMX BF16 on recent x86, AVX2 on older x86, and NEON on ARM (Apple Silicon, Graviton), with a scalar fallback. The kernel surface described covers GEMM, flash-style attention with online softmax that keeps memory at O(N) rather than materializing the N-by-N score matrix, fused activations (GELU, SiLU, Mish), and RMSNorm/LayerNorm with FP32 accumulation. A torch.compile backend is also specified, using AOTAutograd to trace the forward-plus-backward graph once and apply passes for fusion, tiling, layout, rematerialization, optimizer-state quantization, and ZeRO-3 sharding.

The five innovations the docs name as core are trainable packed weights (weights stay in their kernel-ready layout across steps instead of being repacked), tile-local optimizer state, render-graph activation planning that aliases transient buffers by computing exact tensor lifetimes, out-of-core optimizer streaming to NVMe modeled on game asset streaming, and the virtual-accelerator semantics above. ZeRO-3 across sockets lets a two-socket machine hold a model roughly twice as large as one socket's DRAM.

The mechanism the whole runtime is organized around is optimizer-state compression, because for Adam-style training the moment buffers, not the matmuls, are often what exhausts RAM. AdamW8 quantizes both moments blockwise: for each block of 256 parameters it stores the first moment m and second moment v as INT8 with a shared FP32 scale, which the architecture doc accounts at about 1.016 bytes per parameter each, and keeps the parameters themselves in BF16 at 2 bytes. That totals roughly 4 bytes per parameter against about 12 for FP32 AdamW, a factor near 3, with dequantization happening inside the tile kernel at each step. The doc estimates the resulting quantization error at about 0.4% per moment, compounding to about 0.5% over a run, which it treats as within convergence tolerance.

The README pushes this further with AdamW4, quoted at a 7.7x optimizer-memory reduction versus FP32 AdamW, in a memory table labeled as measured on an Apple M1 Pro: a 70M-parameter model's optimizer state falls from 413 MB (FP32) to 105 MB (AdamW8) to 54 MB (AdamW4), a 7.72x ratio, holding near 7.73x and 7.74x at 350M and 760M parameters. These are the stated numbers; the projections extending them to LLaMA-2 7B/13B/70B fitting in 32/64/128 GB machines are explicitly labeled projections rather than measurements.

The one end-to-end benchmark reported in full trains a 25,231,872-parameter transformer (batch 8, sequence 256, 50 steps after 5 warmup) on a machine the tool reports as 2 sockets x 24 cores with AVX-512 and BF16. Baseline PyTorch eager CPU with FP32 AdamW runs at 1240 ms/step (p95 1298 ms), 6.5 samples/sec, 1650 tokens/sec, peak 1842 MB, final loss 6.92. Forge with AdamW8, BF16 autocast, and the virtual accelerator runs at 890 ms/step (p95 920 ms), 9.0 samples/sec, 2293 tokens/sec, peak 1120 MB, final loss 6.94. That is a 1.39x speedup, 1.64x less memory, and +39% token throughput with essentially matched loss. A separate low-level GEMM microbenchmark reports PyTorch FP32 mm at 701.5 GFLOPS versus GCPU BF16 mm at 1328.1 GFLOPS.

The docs are candid about where this does and does not hold: on models under about 1M parameters, Python dispatch overhead can make Forge look slower in microbenchmarks, and expected speedups are laid out as a function of scale (1.1 to 1.3x at 10 to 100M, larger at 100M to 1B, with ZeRO-3 the enabling rather than accelerating factor above 10B). Forge's correctness suite is reported as 91 passed and 1 failed (an ONNX version mismatch). No number beyond these is claimed here, and the aggressive scaling and cluster tiers in the spec are presented as design targets.

Morpheus makes the second bet: that neural-architecture design should be pushed into a type system so it can be checked and automated. Its intellectual basis is the ICML 2024 position that a neural network layer is an algebra homomorphism h for an endofunctor F, meaning the square relating F(A), F(B), A, B commutes (h after alpha equals beta after F(h)); this commutativity is exactly the equivariance/fold-alignment equation, and the endofunctor (list, set, tree, graph) names the data structure the model must respect. The spec's Rosetta table maps category-theory constructs onto keywords and ML concepts: objects are tensor spaces, morphisms and para-morphisms are functions and learnable layers, >> is composition, endofunctors are data-structure types, and 2-morphisms (the reparameterise keyword) express weight tying.

The compiler's Rust source in the bundle is a single module list (lexer, parser, types, ir, codegen, lean, imports, testing, search, symbolic_search, error, span); the design is otherwise described by a specification the document self-labels v0.1, so behavior below is spec intent rather than inspected implementation. The specified pipeline parses .morph files, then category-checks compositions by requiring that the output type of f match the input type of g (the README states this rule as f.target = g.source), with an intent that mismatches are rejected as composition errors instead of deferred to runtime shape crashes. The spec's example layer constructors (Linear, Attention, FeedForward, plus GRU and SSM appearing in the search-space example) carry explicit shapes so that a composed model's type is meant to be known statically. The pipeline then lowers to a serializable Categorical IR whose objects, morphisms, parameters, algebra annotations, commutativity obligations, and 2-morphisms form a commutative diagram, and targets backends including PyTorch nn.Module and native Rust; the bundle does not contain the codegen source, so specific toolchain details are not asserted here.

For formal verification, the spec describes an optional mode in which the compiler emits Lean 4 proof obligations for categorical properties (functor laws, algebra homomorphisms, monad laws) against a Lean library the README calls MorpheusProof, emitting a certificate when a proof passes and a counterexample when it fails. The README reports 11 theorems proven with zero sorry and 23 of 23 example programs compiled to arm64 binaries. That is the extent of what the bundle supports: it contains no Lean proof source, so no claim can be made about which laws the individual theorems encode or how they are discharged, and the spec is explicit that verification is optional and that some obligations are undecidable, in which case the compiler warns and proceeds without guarantee. The CatDL transcript underlining the project reinforces the boundary: it notes that these categorical constraints do not by themselves tell you what a solution looks like in neural-network weight space, so structural validity is a property of the architecture, not of the trained weights.

Two search facilities extend the same idea, both present in the source only as module names (search and symbolic_search). The spec presents search as a language-level primitive that enumerates a configuration space (dimensions, head counts constrained by d % heads == 0, depth, weight-tying recursion, and block patterns mixing attention, GRU, state-space, and MLP blocks) and prunes categorically invalid architectures before any training, with an evolutionary strategy in the example. The bundle does not include the implementation of either module, so no operator alphabet or search algorithm is attributed here. Together the two halves of GCPU are an ambitious, internally coherent attempt to democratize training: Forge lowers the hardware bar with quantized state and a CPU virtual accelerator, and Morpheus aims to raise the correctness floor by making structural validity a compile-time property. The bundle's own framing (design targets versus measurements, one honest end-to-end benchmark, a dispatch layer still routing through CPU tensors, and a verification mode that is optional and can proceed without guarantee) is what keeps the claims legible.