A RISC-V CPU Compiled Into a Language Model's Weights: What Qwenputer Shows
A developer has published Qwenputer, a stock Qwen 3.5 4B with a 32-bit RISC-V CPU compiled into its weights. The CPU uses the model's Gated DeltaNet layers as RAM, runs inside its forward passes, and passes riscv-tests, but the checkpoint is twice the size and needs a driver to boot.
(See the reader poll and source context below).
What do the sources establish?
| Platform / Source | Core Stance / Quote | Engagement Metric | Timestamp |
|---|---|---|---|
| Divyansh Agrawal | Reported context: Qwenputer architecture and Gated DeltaNet RAM CPU design, riscv-tests and qemu match benchmark numbers filesystem results stated limits and hardware requirements | Not recorded |
A developer has built a working RISC-V CPU inside a language model's weights and published the details, including the parts that do not work well yet. The project, called Qwenputer, is a stock Qwen 3.5 4B model with an extra computer compiled into it: the CPU runs inside the model's forward passes, uses the model's own attention state as memory, and can execute arithmetic, C and MicroPython without calling an outside tool.
The write-up is by Divyansh Agrawal, dated 22 October 2026. It describes a model that is available as a download, with code on GitHub, and it spends as much space on limits as on results. That balance is why the demo is worth reading closely rather than filing under stunts.
What the thing actually is
Qwenputer is a stock Qwen 3.5 4B with one addition: a RISC-V CPU embedded within the weights that uses the model's own Gated DeltaNet attention layers as RAM. The CPU executes inside the model's forward passes, so arithmetic, C and MicroPython run without an external runtime, a Python kernel or a tool server.
Qwen's original weights stay exactly as they are. The computer weights are added on top, in extra dimensions and layers.
How an attention layer becomes memory
Three out of four attention layers in Qwen 3.5 use Gated DeltaNet, a linear RNN that keeps a state matrix per head and updates it with the delta rule.
Three settings turn that state into addressable memory. Set the gate to one and nothing fades. Set the write strength, beta, to one and a write replaces the old value completely. Use one-hot keys and every write overwrites exactly one row while every query retrieves an exact value. The GDN layers then work as RAM, at a cost that does not grow with context the way attention's KV cache does.
The post credits Schlag, Irie and Schmidhuber's 2021 paper on linear transformers as fast-weight programmers for the memory idea, and claims the new part is making it exact and addressable inside a production model.
Compiling a CPU into layers
The compiler sets every weight directly, with no training. The author describes the CPU as a circuit, a list of small steps such as reading a register or jumping if zero, and the compiler turns each step into part of a transformer layer: attention heads for lookups, feed-forward units for arithmetic, DeltaNet heads for RAM.
Instructions become generated tokens. One names the memory row, the next carries the value, and a STEP token ends the instruction, so generating the trace is running the CPU. The core is a 32-bit RISC-V design, RV32IM plus the Zba extension and a small bignum unit, occupying 11 layers in front of Qwen's 32. It passes the official riscv-tests, its programs match qemu, and it has run 18 million instructions free-running without a single error.
How the model hands work to it
Qwen 3.5 4B uses the CPU without post-training. The model piggybacks on the tool-call format it already knows to send instructions and read results back, and a small driver called RvChat loads the firmware into the GDN state and keeps the trace tokens out of Qwen's own layers. No external tool calls or runtime are involved, and Qwen's original weights stay unchanged.
Does it work, and how well
The post reports arithmetic benchmarks against both base Qwen and Qwen given a Python tool.
On div_big, Qwenputer scores 97.5%, against 95% for Qwen with a Python tool and 72.5% for base Qwen. On mul_big it scores 97.5%, against 100% with a Python tool and 55% for base Qwen. On mul_div_multiple_longer it scores 100%, against 97.5% and 90%. On add_or_sub_big it scores 97.5%, the same as the Python tool, against 87.5% for base Qwen.
On NUPA's long-integer arithmetic, the score goes from 48.6% for base Qwen to 100%, the same as with a Python tool.
The accuracy claim is not only about scored answers. The author replayed about 230,000 of the CPU's output positions against a reference machine and found zero mismatches. Normal text is said to be unaffected: the model's conversational output stays within noise of base Qwen, with a KL divergence of 6e-4 against a noise floor of 5e-4.
A filesystem, because why not
Having built RAM, the author added a filesystem, exposed through internal put, append, get, list and delete operations. When the model saves a file, compiled layers hash its name into one of 1,024 slots in a directory kept in the DeltaNet state, and store the text in other heads with eight tokens per row, chained together. It holds up to 512 files, about 53,000 tokens.
The persistence result is the interesting part. The state can be saved at the end of a conversation and loaded into the next one with files intact. Using that filesystem, Qwen gets facts that changed between chats right 90% of the time, against 30% when old chats are pasted back in. The author calls this experimental and keeps the code on a separate branch.
The limits, in the author's own account
The checkpoint is 18.8 GB, roughly double base Qwen, though most of that is zero padding. A small driver called RvChat has to boot the firmware and keep trace tokens out of Qwen's layers, so the model does not run with a plain generate() call yet. Speed is the other cost: every CPU instruction costs a few decoded tokens, so a big calculation takes seconds, with the benchmark executor running about 3,200 CPU tokens a second on an L4 GPU. This is also a single-author experiment, and independent reproduction and production relevance are not established.
What it does and does not prove
It proves that a transformer can host exact, addressable memory, and that a compiler can turn that memory plus arithmetic layers into a working CPU that executes a standard instruction set. That claim is checked against riscv-tests, qemu and a reference machine.
It does not prove that the model reasons better or that it has acquired a general capability. The CPU is compiled circuitry, not learned skill, and the benchmark gains come from exact computation replacing approximate arithmetic. The author has written two follow-ups on what exact control over the RNN layers could mean for safety research, including models that hide reasoning in RNN state and compiled weights as monitoring tools.
For engineers, the practical takeaway is narrower than the headline. The interesting object is not a model that can multiply. It is the demonstration that a linear-attention state, forced into exact overwrites, can serve as memory for compiled computation inside a production model's layers, with the trace visible in generated tokens. Whether that becomes useful depends on speed, tooling and reproduction, all of which the author lists as open.