AI IN COMBINATORS
Posted: Fri Sep 25, 2026 3:19 pm
Hi everyone! For the last few months I've been building something slightly unreasonable: a real GPT-style language model that runs entirely on Factorio 2.0 combinators.
What it is
A character-level transformer: 6.4M parameters, 8 layers, 8 attention heads, 256-character context
~105,000 entities: 53K arithmetic, 19K decider and 19K constant combinators, plus 8,800 lamps
No mods, no Lua — just combinators, wires and lamps
It chats in English and knows Factorio facts (biters, science, trains, Space Age planets), and it can tell a short story or a joke
How you talk to it
You type a question letter by letter into a row of constant combinators, put a ✓ after the last letter and flip the SEND switch. A progress bar shows the model reading your question, then the answer lights up letter by letter on a giant 5×7 pixel lamp board, about 7 seconds per letter at 60 UPS.
you: hi!
bot: hello! i am busy with my factory, but i always have time to talk.
The trick that made it possible
In 2.0, an arithmetic combinator can do each * each across the red and green wires and sum the result, so one combinator computes a whole dot product. Each row of a weight matrix lives in a constant combinator, and one constant plus one arithmetic combinator make a neuron with 256 or 1024 inputs. It also turns out a single wire can carry 2,169 different signals (items, fluids, recipes, even entities like the atomic bomb wave), which is how 256-number vectors travel through the network.
Softmax exp is a staircase of 134 deciders, LayerNorm's square root is Newton's method, and each layer keeps a 256-cell KV cache made of sample-and-hold latches. Everything is integer math and matches a Python reference bit for bit.
Try it yourself (needs Space Age)
Save file, blueprint and full source: https://github.com/RomOSTiny/factorio-transformer
Next step: a model several times bigger. Happy to answer any questions about how it works!
What it is
A character-level transformer: 6.4M parameters, 8 layers, 8 attention heads, 256-character context
~105,000 entities: 53K arithmetic, 19K decider and 19K constant combinators, plus 8,800 lamps
No mods, no Lua — just combinators, wires and lamps
It chats in English and knows Factorio facts (biters, science, trains, Space Age planets), and it can tell a short story or a joke
How you talk to it
You type a question letter by letter into a row of constant combinators, put a ✓ after the last letter and flip the SEND switch. A progress bar shows the model reading your question, then the answer lights up letter by letter on a giant 5×7 pixel lamp board, about 7 seconds per letter at 60 UPS.
you: hi!
bot: hello! i am busy with my factory, but i always have time to talk.
The trick that made it possible
In 2.0, an arithmetic combinator can do each * each across the red and green wires and sum the result, so one combinator computes a whole dot product. Each row of a weight matrix lives in a constant combinator, and one constant plus one arithmetic combinator make a neuron with 256 or 1024 inputs. It also turns out a single wire can carry 2,169 different signals (items, fluids, recipes, even entities like the atomic bomb wave), which is how 256-number vectors travel through the network.
Softmax exp is a staircase of 134 deciders, LayerNorm's square root is Newton's method, and each layer keeps a 256-cell KV cache made of sample-and-hold latches. Everything is integer math and matches a Python reference bit for bit.
Try it yourself (needs Space Age)
Save file, blueprint and full source: https://github.com/RomOSTiny/factorio-transformer
Next step: a model several times bigger. Happy to answer any questions about how it works!