8bit-threshold-computer

A Turing-complete CPU implemented entirely as threshold logic gates. Every gate, from Boolean primitives to arithmetic to control flow, is a single threshold neuron of the form:

output = 1 if (Σ wᵢ·xᵢ + b) ≥ 0 else 0

Every weight in the file is in {-1, 0, 1}. Biases are integers. Activations are the Heaviside step. Nothing else. The library was originally built with positional weights up to ±2³¹ for wide single-layer comparators; those have all been replaced with bit-cascaded multi-layer equivalents that use only ternary weights and small integer biases. Threshold-gate evaluation reduces to a popcount minus a popcount plus a bias, which is exactly what neuromorphic chips and FPGAs natively support.

The repository ships eighteen prebuilt configurations spanning three data-path widths (8, 16, 32 bits) and six memory sizes (0 B to 64 KB). The canonical file at the repo root is the largest of these: a 32-bit data path with a 64 KB address space and ~~8.61 M tensor elements (~~8.46 M gate weights and biases; the rest is .inputs routing metadata).

neural_computer.safetensors        32-bit data, 64 KB memory, ~8.61M params (canonical)
variants/neural_computer{8,16,32}.safetensors                    full memory (64 KB)
variants/neural_computer{8,16,32}_reduced.safetensors            4 KB memory
variants/neural_computer{8,16,32}_small.safetensors              1 KB memory
variants/neural_computer{8,16,32}_scratchpad.safetensors         256 B memory
variants/neural_computer{8,16,32}_registers.safetensors          16 B memory
variants/neural_alu{8,16,32}.safetensors                         pure ALU, no memory
variants/neural_subleq8.safetensors                              one-instruction machine (SUBLEQ)
variants/neural_rv32.safetensors                                 RV32IM + F-subset RISC-V processor

Two further machines mark the endpoints of the family, each detailed in its own section below: neural_subleq8, a Turing-complete one-instruction computer whose entire control flow is a single threshold neuron; and neural_rv32, a RISC-V processor (RV32IM + an F subset) that runs stock-compiler C, with dual-issue execution, memory-mapped I/O, and a self-referential NEUR opcode.

Quick start

import torch
from safetensors.torch import load_file

tensors = load_file("neural_computer.safetensors")

def heaviside(x):
    return (x >= 0).float()

# AND gate: fires when both inputs are 1
w = tensors['boolean.and.weight']  # [2]
b = tensors['boolean.and.bias']    # [1]
for a, c in [(0, 0), (0, 1), (1, 0), (1, 1)]:
    out = heaviside((torch.tensor([a, c], dtype=torch.float32) * w).sum() + b)
    print(f"AND({a}, {c}) = {int(out.item())}")

Run the full circuit verification suite against any variant:

python eval_all.py variants/                              # all 18 in one pass
python eval_all.py neural_computer.safetensors            # the canonical file
python eval_all.py --cpu-program variants/                # also run an assembled
                                                          # program through the
                                                          # threshold-gated CPU

eval_all.py reads each variant's manifest, runs a gate-level fitness suite (13,900–15,900 tests per variant covering Boolean, arithmetic, ALU, control, modular, error-detection, threshold, and float circuits, including end-to-end evaluation of the composed float pipelines from the shipped wiring metadata — see the Verification table), and optionally executes a small assembled program through a manifest-sized threshold CPU plus a chained 16- or 32-bit ALU sequence on wider variants.

For an interactive walkthrough that exercises Boolean gates, the 8-bit ALU, mod-5 divisibility, and a CPU loop end-to-end:

python play.py                                            # 1 KB demo, runs in seconds
python play.py --model neural_computer.safetensors        # 64 KB, slower

For end-to-end CPU validation (Fibonacci, sum 1..N, bubble sort, self-modifying JMP, all eight conditional jumps, CALL stack semantics, MUL cross-checked against repeated ADD, DIV cross-checked against repeated SUB, a bitwise AND/OR/XOR/SHL/SHR chain, and the architectural flag policy):

python test_cpu.py                                        # default: 1 KB, ~2 s
python test_cpu.py --model neural_computer.safetensors    # 64 KB canonical, ~2 min
python test_cpu.py --only fib,sum_n                       # subset of suite

Each program is assembled by a small Python assembler (cpu_programs.py) and run through the threshold-gated CPU; the driver verifies expected memory contents at HALT.

Execution model

A self-contained machine. State goes in, state comes out:

Pure tensor computation: state in, state out
Frozen circuits: integer weights, Heaviside activation
ACT execution: internal loop until HALT
No external orchestration: one forward pass equals one complete program execution

Every datapath operation runs through threshold gates: ALU arithmetic (ADD/SUB via ripple-carry, MUL via partial-product AND gates and shift-add, DIV via per-stage bit-cascade comparators), bitwise logic, shifts, comparisons, conditional-jump PC muxing, stack-pointer stepping, and all memory reads and writes. Instruction decode (bit-field selection), PC sequencing, and the bit complements feeding negated-condition selects are fixed wiring in the runtimes — structural routing, not gates.

            ┌─────────────────────────────┐
            │      Initial State          │
            │  [PC|Regs|Flags|Memory...]  │
            └─────────────┬───────────────┘
                          ▼
            ┌─────────────────────────────┐
            │   Threshold Circuit Layer   │
            │  ┌───────────────────────┐  │
            │  │   Fetch: PC → Instr   │  │
            │  ├───────────────────────┤  │
            │  │   Decode: Opcode/Ops  │  │
            │  ├───────────────────────┤  │
            │  │   Execute: ALU/Mem    │  │
            │  ├───────────────────────┤  │
            │  │   Writeback: Results  │  │
            │  ├───────────────────────┤  │
            │  │   PC Update           │  │
            │  └───────────┬───────────┘  │
            │              │              │
            │         ┌────▼────┐         │
            │         │ HALTED? │         │
            │         └────┬────┘         │
            │       no ────┴──── yes      │
            │        │           │        │
            │        ▼           ▼        │
            │     [loop]      [exit]      │
            └─────────────┬───────────────┘
                          ▼
            ┌─────────────────────────────┐
            │       Final State           │
            └─────────────────────────────┘

Instruction set

Opcode	Mnemonic	Operation
0x0	ADD	R[d] = R[a] + R[b]
0x1	SUB	R[d] = R[a] - R[b]
0x2	AND	R[d] = R[a] & R[b]
0x3	OR	R[d] = R[a] \| R[b]
0x4	XOR	R[d] = R[a] ^ R[b]
0x5	SHL	R[d] = R[a] << 1
0x6	SHR	R[d] = R[a] >> 1
0x7	MUL	R[d] = R[a] * R[b]
0x8	DIV	R[d] = R[a] / R[b]
0x9	CMP	flags = R[a] - R[b]
0xA	LOAD	R[d] = M[addr]
0xB	STORE	M[addr] = R[s]
0xC	JMP	PC = addr
0xD	Jcc	PC = addr if cond (imm8[2:0]: 0=Z, 1=NZ, 2=C, 3=NC, 4=N, 5=P, 6=V, 7=NV)
0xE	CALL	push PC; PC = addr
0xF	HALT	stop execution

Flag policy. Only ADD, SUB, MUL, and CMP write the FLAGS register; AND, OR, XOR, SHL, SHR, DIV, LOAD, STORE, and all control transfers leave it unchanged. Branch on the arithmetic op that set the condition — intervening bitwise or shift instructions do not disturb it. The flags_policy program in the CPU suite pins this behavior.

Flag	ADD	SUB / CMP	MUL
Z	result == 0	result == 0	result == 0
N	result bit 7	result bit 7	result bit 7
C	carry-out	1 when no borrow (`a >= b`)	0
V	signed overflow	signed overflow	0

State tensor layout

The state tensor uses MSB-first bit ordering: index 0 of each multi-bit field is the most-significant bit. So R0[0] is bit 7 of the architectural register, R0[7] is bit 0.

[ PC[N] | IR[16] | R0[8] R1[8] R2[8] R3[8] | FLAGS[4] | SP[N] | CTRL[4] | MEM[2^N][8] ]

N is the address width (configurable, 0–16). Flags are ordered Z, N, C, V. Control bits are ordered HALT, MEM_WE, MEM_RE, RESERVED.

Bit ordering, one rule per scope

The state tensor's MSB-first convention does not propagate to subcircuit ports. Each subcircuit names its operand bits in its own scope:

Scope	Convention	Example
State tensor	MSB-first (index 0 = MSB)	`R0[0]` is bit 7 of register R0
Integer subcircuit ports (`$a[i]`, `$b[i]`)	LSB-indexed (index 0 = LSB)	`$a[0]` is bit 0 of operand `a`
Ripple-carry full adders (`fa0..fa7`)	LSB-first (fa0 = LSB)	`fa0` consumes `$a[0]` and `$b[0]`
Composed float circuits (`float16.add` etc.)	MSB-first operand words	`$a[0]` is the sign bit, `$a[1..E]` the exponent
Float result gates (`exp_out.bit{k}`, `frac_out.bit{k}`)	LSB-first field bits	`frac_out.bit0` is the fraction LSB
Instruction word	MSB-first (bit 15 = opcode high)	bit 15 is `opcode[3]`

Worked example for arithmetic.ripplecarry8bit:

Inputs: $a[0]..$a[7] and $b[0]..$b[7] where $a[0] is the LSB of a. To add a = 0x05 = 0b00000101 and b = 0x03, drive a[0]=1, a[1]=0, a[2]=1 (rest 0) and b[0]=1, b[1]=1 (rest 0).
Outputs: fa0.ha2.sum.layer2..fa7.ha2.sum.layer2 are sum bits 0..7 (LSB to MSB), and fa7.carry_or is the final carry-out. The 8-bit result is {fa7..fa0} reading high-to-low.

This is also how safetensors2verilog's threshold-logic frontend exposes the ports of any extracted subcircuit. See the project's testbench at tests/threshold_alu/run.py for a worked end-to-end example, or use python -m safetensors2verilog ... --inspect to print the port contract for any extracted circuit.

Instruction encoding (16-bit, MSB-first)

15..12  11..10  9..8  7..0
opcode  rd      rs    imm8

Interpretation:

R-type: rd = rd op rs (imm8 ignored)
I-type: rd = op rd, imm8 (rs ignored)
Address-extended: LOAD, STORE, JMP, Jcc, CALL consume the next word as a 16-bit address (big-endian); imm8 is reserved and the PC skips 4 bytes when the jump is not taken.

Circuit categories

Category	Circuits	Examples
Boolean	9	AND, OR, NOT, NAND, NOR, XOR, XNOR, IMPLIES, BIIMPLIES
Arithmetic	18+	half/full adder, ripple-carry (8/16/32-bit), comparators (8/16/32-bit), 3-operand adder, A+B×C and (A+B)×C expressions
ALU	8/16/32-bit	shifts, multiply, divide, INC/DEC, NEG, ROL/ROR, bitwise
Combinational	10+	MUX (2:1, 4:1, 8:1), DEMUX, 3-to-8 decoder, 8-to-3 encoder, barrel shifter, priority encoder
Control flow	16	JMP, conditional jumps (JZ/JNZ/JC/JNC/JN/JP/JV/JNV), CALL, RET, PUSH, POP
Memory	3	N-bit address decoder, read mux, write cells (packed)
Modular	11	divisibility by 2–12 (multi-layer for non-powers-of-2)
Threshold	13	k-of-n gates, majority, minority, exactly-k
Pattern	10	popcount, leading/trailing ones, symmetry
Error detection	11	parity (XOR tree), checksum, CRC, Hamming
Float (IEEE 754)	half + single	composed ADD, MUL, DIV, EQ/LT/LE/GT/GE pipelines plus unpack/pack/classify/normalize stages

The float ADD/MUL/DIV/CMP circuits are self-contained composed pipelines: their complete internal wiring ships in the files as .inputs metadata, their external inputs are the raw operand words ($a[0] sign, $a[1..E] exponent, then mantissa; MSB-first), and the eval suite reconstructs and executes each netlist end to end from that metadata alone (NetlistEvaluator). Comparisons are fully IEEE (NaN unordered, +0 == -0, subnormal ordering, mixed signs). The arithmetic is round-to-nearest-even, bit-exact to IEEE hardware on the normal range, with exact specials (NaN, infinities including inf - inf/inf * 0/0/0/inf/inf → NaN and x/0 → inf, signed zeros) and flush-to-zero for subnormal operands and results. Because the tests run from each file's own routing metadata, they prove the artifact is self-contained: safetensors2verilog-style extraction of float16.add and friends has everything it needs to emit a single composed module. Remaining gaps: gradual underflow (subnormal results) and FMA/FCVT for a full F extension.

Tensor naming

{category}.{circuit}[.{layer}][.{component}].{weight|bias}

Examples:
  boolean.and.weight
  boolean.xor.layer1.neuron1.weight
  arithmetic.ripplecarry8bit.fa7.ha2.sum.layer1.or.weight
  modular.mod5.eq.k15.bit3.match.weight
  error_detection.paritychecker8bit.stage2.xor1.layer1.nand.bias

Memory circuits are stored as packed tensors so the safetensors header stays manageable (memory.addr_decode.weight, memory.read.and.weight, memory.write.and_old.weight, etc.).

Bit widths and memory profiles

The build tool emits one of 51 functionally distinct configurations: three data-path widths × seventeen address widths (0–16, where 0 means no memory).

Bit widths (--bits):

Width	Range	Use case
8	0–255	full CPU, legacy compatibility
16	0–65,535	extended arithmetic
32	0–4,294,967,295	practical arithmetic ranges

Memory profiles (-m):

Profile	Size	Addr bits	Filename suffix
`none`	0 B	0	(uses `alu` instead of `computer`)
`registers`	16 B	4	`_registers`
`scratchpad`	256 B	8	`_scratchpad`
`small`	1 KB	10	`_small`
`reduced`	4 KB	12	`_reduced`
`full`	64 KB	16	(none)

Auto-generated filename: neural_{alu|computer}{BITS}[_{MEMORY}].safetensors. Custom address widths via -a N produce _addrN.

python build.py --bits 32 --apply all              # neural_computer32.safetensors
python build.py --bits 8 -m none --apply all       # neural_alu8.safetensors
python build.py --bits 16 -m small --apply all     # neural_computer16_small.safetensors
python build.py --bits 32 -a 6 --apply all         # neural_computer32_addr6.safetensors

To regenerate every named variant in one pass:

python build_all.py

This populates variants/ with all 18 builds, quantizes each one to the smallest signed integer dtype that exactly represents its weights (~4× reduction in tensor data, with file size dominated by the safetensors header on the smaller profiles), verifies the strictly ternary weight invariant (--ternary --strict, so a build with any non-ternary weight fails loudly), stamps the weight_quantization metadata field, and runs eval.py on each as a sanity check.

The quantizer is also available standalone:

python quantize.py path/to/file.safetensors           # in-place
python quantize.py variants/                          # whole directory
python quantize.py model.safetensors -o quantized.safetensors
python quantize.py file.safetensors --ternary         # push toward {-1, 0, 1} weights
python quantize.py file.safetensors --ternary --strict  # error if any weight is non-ternary

Every weight and bias tensor in the canonical model fits in int8. The eval pipeline promotes weights to float32 on load, so integer storage is exact and transparent.

Ternary mode. build.py emits only ternary weights: identity buffers are weight=1, bias=-1 (H(x - 1)), and the comparators, modular detectors, and division stages that previously required positional weights up to ±2³¹ are bit-cascaded multi-layer equivalents. With --ternary, the quantizer verifies this and repairs legacy files: it rewrites historical single-input weight=±2 buffers as weight=±1 with the bias adjusted to preserve the heaviside output for binary inputs (H(2x - 1) ≡ H(x - 1)), and rebuilds pre-bit-cascade modular detectors (moduli already in bit-cascade form are left untouched, routing metadata included). --strict fails if any weight tensor remains non-ternary. Every shipped file carries the metadata field weight_quantization: ternary; a repaired file with remaining violations would be stamped ternary_partial.

Verification

Category	Coverage	Notes
Boolean gates	exhaustive	all 2^n input combinations
Arithmetic (8-bit)	strategic sampling	edge values + diagonal pairs; ~50 cases per circuit
Arithmetic (16/32-bit)	strategic sampling	width-scaled extremes, alternating patterns, byte-boundary carries
ALU primitives (8/16/32-bit)	strategic sampling	edge inputs per operation; DIV comparators driven along real restoring-division traces
Control flow	exhaustive	all 2^3 input combinations per Jcc, per address bit
Threshold k-of-n	exhaustive	all 256 8-bit popcounts
Modular (all moduli, 8-bit input)	exhaustive	every value in [0, 255]
Parity	exhaustive	every value in [0, 255]
Pattern recognition	exhaustive	every value in [0, 255]
Combinational logic	exhaustive	full input space per gate
Float unpack/pack	exhaustive, functional	every bit gate driven with 0 and 1 (identity)
Float classify	functional	IEEE 754 categories (zero, subnormal, normal, inf, NaN) at edge encodings, both widths
Float CMP (composed)	functional, exact IEEE	full netlist rebuilt from the shipped `.inputs` metadata and evaluated end to end; NaN unordered, signed zeros, subnormal ordering, mixed signs — all five predicates, both widths
Float ADD/MUL/DIV (composed)	functional, bit-exact to IEEE hardware	same metadata-driven evaluation: exact specials, flush-to-zero, round-to-nearest-even; cross-checked against native float arithmetic
Memory / manifest	structure checks	packed-tensor shapes against the manifest
CPU integration	program-level	ten assembled programs (Fibonacci, sum, sort, self-modifying JMP, all eight Jcc, CALL stack push, MUL vs repeated ADD, DIV vs repeated SUB, a bitwise AND/OR/XOR/SHL/SHR pipeline, and the flag-policy pin)

The 8-bit arithmetic and ALU tests use strategic sampling rather than the full 65,536-case sweep because exhaustive coverage at 8-bit is feasible but not necessary given that the circuits are constructed gate-by-gate. The 16-bit and 32-bit arithmetic tests sample edge cases only; full exhaustive coverage at those widths is infeasible without specialized hardware.

eval_all.py runs the unified suite. Exit code is the number of failing variants (0 means all pass). Testing is evaluation, not rebuilding: python eval_all.py variants/ scores all 18 fitness variants against the shipped weights in about two minutes (~6 s each, the composed float netlists evaluated in NetlistEvaluator's leveled mode) and cleanly skips the two standalone machines. Rebuilding the models (build_all.py, ~50 min for all 18) is a separate step, needed only when the circuit constructions in build.py change — routine verification never rebuilds. The batched evaluator is population-safe: every chained intermediate (carry, borrow, mux select) is computed per population slot, so prune_weights.py's parallel fitness screens are exact rather than slot-0 approximations.

neural_subleq8 — the one-instruction machine

The minimal member of the family: a Turing-complete computer with no instruction decoder (there is only one instruction), no registers, and no flags — 194 logic gates totalling 548 ternary parameters. Its one architectural decision, branch or fall through, is a single threshold neuron (subleq.leq) over the sign and zero of the subtraction result.

8-bit data, 8-bit addresses, 256 bytes of packed threshold memory, a program counter and nothing else.
An instruction is three bytes A B C. Step: M[B] = (M[B] - M[A]) mod 256; if the result is <= 0 in two's complement (zero or bit 7 set), PC = C, else PC += 3. PC = 0xFF halts.
Circuits (all wiring shipped as .inputs metadata): an 8-bit two's-complement subtractor, the branch neuron, a PC + 3 incrementer, a branch mux, and the packed 256-row memory.

python build.py --apply subleq          # -> variants/neural_subleq8.safetensors
python machines.py subleq               # exhaustive datapath + lockstep programs

machines.py subleq evaluates the datapath from the shipped wiring metadata over all 65,536 operand pairs (result and branch decision both exhaustive), checks the PC mux, and runs a program suite (clear, negate-copy, add-by-double-negation, countdown loop) in full-state lockstep against a reference emulator. Third-party SUBLEQ toolchains target this machine directly once the 0xFF halt convention is mapped.

neural_rv32 — a RISC-V processor as a threshold network

The most capable member: a RISC-V CPU whose entire datapath is ternary threshold gates (variants/neural_rv32.safetensors, ~8.6 M parameters, 64 KB memory, strictly ternary).

RV32I base: LUI, AUIPC, JAL, JALR, all six branches, LB/LH/LW/LBU/LHU, SB/SH/SW, and the full OP-IMM/OP groups. 32 × 32-bit registers (x0 zero), little-endian memory through the packed threshold circuits. ECALL halts.
M extension: MUL, MULH, MULHSU, MULHU (full 64-bit product through a shift-add array with gate-level sign correction), DIV, DIVU, REM, REMU (32 restoring stages, spec-exact divide-by-zero and overflow).
F subset: FLW, FSW, FMV.W.X, FMV.X.W, FADD.S, FSUB.S, FMUL.S, FDIV.S, FEQ.S, FLT.S, FLE.S, FSGNJ[N/X].S — the arithmetic executed by the composed float32 pipelines (round-to-nearest-even, bit-exact to hardware; specials and flush-to-zero as above).
NEUR (custom-0): neur rd, rs1, rs2 evaluates one threshold neuron — rd = H(popcount(rs1[7:0] & rs2[7:0]) - popcount(rs1[7:0] & rs2[15:8]) + sext(rs2[20:16])). Networks of NEUR instructions are neural networks running as software on the neural network; the test suite computes XOR with a two-layer NEUR net.
Dual issue: two adjacent OP/OP-IMM/LUI/AUIPC instructions retire in one cycle when the gate-level hazard comparators (rv32.hazard.*) clear RAW and WAW dependences.
MMIO: stores to 0xFF00 append a character to the console.

Signed comparisons ride the unsigned bit-cascade with sign bits complemented through NOT gates; SLL uses the barrel shifter, SRL is bit-reversal wiring over it, SRA a gate mux over the complement form. Instruction decode, immediate extraction, register-file indexing, and PC sequencing are fixed wiring, per the family convention.

python build.py --apply rv32                            # build the file
python quantize.py variants/neural_rv32.safetensors --ternary --strict
python machines.py rv32                                 # eight-program lockstep suite
python machines.py rv32-c                               # stock-compiler C, end to end

Running compiled C. machines.py rv32-c compiles a freestanding C program (gcd, Fibonacci, insertion sort; rv32im, so real mul/rem) with an unmodified clang rv32im toolchain, loads the relocatable object with an in-repo loader (no external linker — it resolves the R_RISCV relocations of one translation unit and lays the sections out flat), executes it on the threshold CPU, and checks the return value against the value computed natively. The program retires in ~300 instructions and matches exactly. Stock rv32im toolchains (gcc, clang, rustc) emit this ISA; RNE rounding, gradual underflow, and FMA/FCVT for a full F extension are the remaining gaps.

The composed circuits evaluate in a leveled mode (NetlistEvaluator, one padded tensor op per topological level instead of one Python step per gate), ~18× faster on the FPU-scale netlists.

Threshold logic

A threshold gate computes a Boolean function by taking a weighted sum of binary inputs and comparing the result to a threshold; the output is 1 when the sum meets or exceeds the threshold and 0 otherwise. Equivalently, it is a neuron with Heaviside step activation, integer weights, and an integer bias.

Threshold gates are strictly more powerful than standard Boolean gates. A single threshold gate can compute any linearly separable Boolean function, which includes AND, OR, NAND, NOR, IMPLIES, and many others that require multiple levels of conventional gates. Functions that are not linearly separable (XOR, parity, mod-k for k not a power of two) require multiple layers.

Example gates:

AND: w=[1, 1], b=-2
  H(0+0-2) = 0     H(1+1-2) = 1

OR:  w=[1, 1], b=-1
  H(0+0-1) = 0     H(1+0-1) = 1

XOR: two layers (not linearly separable)
  layer 1: OR + NAND
  layer 2: AND of the two

A full adder is two half-adders plus a carry OR, around four threshold layers. An 8-bit ripple-carry adder is eight chained full adders, around 32 layers.

History

Warren McCulloch and Walter Pitts introduced the threshold neuron in 1943, proving that networks of such neurons can compute any Boolean function. Their work preceded both the perceptron and modern neural networks and established the theoretical foundation for neural computation.

The 1960s saw substantial work on threshold logic synthesis. Saburo Muroga, Robert McNaughton, and Michael Dertouzos developed algebraic methods for determining whether a Boolean function can be implemented as a single threshold gate, and if so, how to compute the appropriate weights. The focus was on individual gates rather than complete systems.

Frank Rosenblatt's Mark I Perceptron (1957–1960) implemented threshold neurons in hardware using potentiometers for weights, but it was a pattern classifier that learned its weights through training; the final configurations were not published. Bernard Widrow's ADALINE and MADALINE (1960–1963) similarly used adaptive threshold elements with weights learned via the LMS algorithm.

Hava Siegelmann and Eduardo Sontag proved in the 1990s that recurrent neural networks are Turing-complete. The construction relied on continuous sigmoid activations with infinite precision, not the discrete step function used in threshold logic. Other theoretical work on neural Turing machines and differentiable computers followed similar patterns: universality with continuous activations chosen to support gradient-based training.

Neuromorphic hardware

Modern neuromorphic processors implement large arrays of configurable threshold-like neurons in silicon:

Intel Loihi (2017): 128 neuromorphic cores with programmable synaptic weights, spike-based communication, and on-chip learning. Supports integer weights and configurable neuron dynamics.
IBM TrueNorth (2014): one million neurons and 256 million synapses across a 4096-core array. Each neurosynaptic core implements 256 neurons with configurable weights and thresholds.
BrainChip Akida (2021): edge-oriented event-based processing with integer weights.
SpiNNaker (University of Manchester): ARM cores simulating spiking networks at scale.

Published work on these platforms has focused on neural network inference, sensory processing, and pattern recognition. A 2024 paper demonstrated basic logic gates, adders, and decoders on SpiNNaker and Dynap-SE1 and described that work as "a first step toward the construction of a spiking computer"; that implementation lacked instruction fetch, a program counter, memory, and control logic.

The weights in this repository implement a complete CPU: registers, ALU with 16 operations, status flags, conditional branching, subroutine calls, stack operations, and memory access. Every logic component is a threshold neuron with integer weights; instruction decode and PC sequencing are fixed wiring between them.

Hardware compatibility

All weights are in {-1, 0, 1}, all activations are Heaviside step, and every gate is a single weighted sum followed by a sign test. This eliminates multipliers entirely: each gate evaluation is a popcount of +1-weighted inputs minus a popcount of -1-weighted inputs plus an integer bias. The circuits are intended to deploy directly on:

FPGA: every gate maps to a small LUT cluster (or a popcount tree of LUT4/LUT6 + carry chain). Ternary weight storage compresses to 2 bits per weight; routing collapses to bit selection.
Intel Loihi: integer weights and Heaviside threshold neurons are the native primitive. Ternary fits well within Loihi's 8-bit weight range.
IBM TrueNorth: configurable threshold per neurosynaptic core; ternary weights and small biases are within the supported range.
BrainChip Akida: edge-oriented integer-weight inference; ternary weights fit cleanly.

LLM integration

Threshold circuits can be embedded into transformer MLP layers to give a language model exact arithmetic. Standard LLMs fail at arithmetic because they interpolate over the training distribution rather than compute, so a 360M-parameter model trained on web text has seen 127 + 128 = 255 few times if at all and guesses based on pattern matching.

The integration freezes the circuits and trains only the interface layers that:

Extract operands from token embeddings.
Route computation through the appropriate circuit.
Inject the result back into the residual stream.

The model learns call dispatch; the arithmetic is already solved.

Architecture

x ──┬── MLP path ─────────────────┬── + ── output
    │                             │
    └── BitExtractor ── Circuit ──┴── BitInjector
                          │
                       Router (learned weighting)

Augmented MLP forward pass:

def forward(x):  # x: [batch, seq, d_model=960]
    mlp_out = self.down_proj(silu(self.gate_proj(x)) * self.up_proj(x))

    a_bits, b_bits = self.bit_extractor(x)              # [batch, seq, 8] each
    result_bits, carry = self.circuits.add_8bit(a_bits, b_bits)
    flags = self.compute_flags(result_bits, carry)
    circuit_delta = self.bit_injector(result_bits, flags)

    route_weights = self.router(x)                       # [batch, seq, 2] softmax
    return mlp_out + route_weights[..., 1:2] * circuit_delta

Target model

The reference integration uses HuggingFace's SmolLM2-360M-Instruct. See llm_integration/SMOLLM2_ARCHITECTURE.md for the full technical analysis.

Property	Value
Parameters	361.82 M
Hidden dimension	960 (matches the extractor input)
Layers	32 transformer blocks
Attention	15 query heads, 5 KV heads (GQA)
MLP	SwiGLU (960 → 2560 → 960)
Position encoding	RoPE (theta = 100k, max 8192)

Digits tokenize individually ("47 + 86" → ['4', '7', ' +', ' ', '8', '6'], with digit token IDs 32 + digit_value), which makes position-based operand extraction practical.

Gradient flow

Heaviside has zero gradient almost everywhere. The implementation uses a straight-through estimator:

class HeavisideSTE(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x):
        return (x >= 0).float()

    @staticmethod
    def backward(ctx, grad_output):
        return grad_output

At inference, Heaviside is the true step function; if the extractor identifies operands correctly, the circuit produces the correct result by construction.

Baseline

SmolLM2-360M-Instruct on randomized 8-bit arithmetic (2,000 cases, operands uniform on [0, 255], generous answer extraction):

Operation	Accuracy
Addition	35.92%
Subtraction	17.72%
Multiplication	1.25%
Greater than	14.37%
Less than	4.31%
Equality	0.28%
Overall	11.90% (238/2000)

Multiplication accuracy at 1.25% is essentially random over the output space. Comparison operations often echo the expression rather than evaluate it. Even addition fails roughly two-thirds of the time on full 8-bit operands. Performance degrades further as operand magnitude increases: edge cases like 127 + 128 are almost never correct.

The frozen threshold circuits reach 100% on the same task when given correctly formatted bit inputs (10,000 random cases, every operation). The integration challenge is therefore the extractor, not the arithmetic.

Trainable parameters (SmolLM2, hidden_dim = 960)

Component	Parameters	Description
AttentionPooling	~3.7 M	4-head attention over the sequence
MultiHeadBitExtractor (× 2)	~245 K each	8 per-bit MLPs for A and B
OpRouter	~246 K	960 → 256 → 6 MLP
Extractor total	~4.4 M	full extraction module

Alternative architectures: PositionExtractor (~~1.5 M, position-specific, no attention), DigitExtractor (~~1.2 M, predicts digits 0–9 instead of bits), HybridExtractor (digit lookup with MLP fallback for word numerals). With --unfreeze_layers 4 an additional ~39.3 M trainable parameters open up in the top four transformer layers.

Training

python train.py --mode router --epochs 100                          # sanity check
python train.py --mode llm --epochs 100 --batch_size 256            # frozen LLM
python train.py --mode llm --unfreeze_layers 4 --batch_size 4096    # fine-tune top layers

Loss components: BCE on output bits, BCE on extracted A and B bits (2× weight), and CE on operation classification. Curriculum runs 0–9 → 0–99 → 0–255. Optimizer is AdamW, lr 3e-4, gradient clipping 1.0.

Repository layout

neural_computer.safetensors         canonical model (32-bit, 64 KB, ~8.61M params)
variants/                           18 prebuilt configurations + neural_subleq8 + neural_rv32
build.py                            generator (one safetensors per invocation; also `subleq`, `rv32`)
build_all.py                        builds, quantizes (--ternary --strict), and verifies every profile
quantize.py                         min integer dtypes + ternary verification/repair
eval.py                             gate-level fitness suite, NetlistEvaluator, 64 KB reference CPU
eval_all.py                         variant-agnostic harness + manifest-sized threshold CPU
cpu_programs.py                     assembler + ten-program suite for CPU-level validation
test_cpu.py                         runs the program suite against a chosen variant
machines.py                         neural_subleq8 + neural_rv32 runtimes, references, assemblers,
                                    RV32 object loader, and both test suites (subleq / rv32 / rv32-c)
play.py                             interactive demo
prune_weights.py                    GPU-batched weight reduction with conflict resolution
llm_integration/                    SmolLM2 extractor + circuit wrapper + training code
  ├── circuits.py                   FrozenThresholdCircuits (loads safetensors, exposes
  │                                 add_8bit / sub_8bit / mul_8bit / compare_*)
  ├── model.py                      Extractor variants, ArithmeticModel
  ├── train.py                      router / interface / llm training modes
  ├── fitness.py                    randomized fitness function
  ├── baseline.py                   vanilla SmolLM2 baseline measurement
  ├── trained/                      checkpointed extractor weights
  └── smollm2/
      ├── SMOLLM2_ARCHITECTURE.md   architecture analysis
      ├── analyze_smollm2.py        analysis script
      └── smollm2_analysis.json     analysis output

Citation

@misc{8bit-threshold-computer,
  title={8bit-threshold-computer: A Turing-Complete Threshold Logic CPU},
  author={Norton, Charles},
  year={2026},
  howpublished={Hugging Face},
  url={https://huggingface.co/phanerozoic/8bit-threshold-computer}
}

License

MIT

References

McCulloch & Pitts (1943). A Logical Calculus of Ideas Immanent in Nervous Activity.
Muroga (1971). Threshold Logic and Its Applications.
Siegelmann & Sontag (1995). On the Computational Power of Neural Nets.
Bengio et al. (2013). Estimating or Propagating Gradients Through Stochastic Neurons.
Ma et al. (2024). The Era of 1-bit LLMs (BitNet b1.58).
HuggingFace (2024). SmolLM2: Small Language Models — model card.
Vaswani et al. (2017). Attention Is All You Need.
Su et al. (2021). RoFormer: Enhanced Transformer with Rotary Position Embedding.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support