AI IN COMBINATORS

[RomaPro227]

Hi everyone! For the last few months I've been building something slightly unreasonable: a real GPT-style language model that runs entirely on Factorio 2.0 combinators.

What it is

A character-level transformer: 6.4M parameters, 8 layers, 8 attention heads, 256-character context
~105,000 entities: 53K arithmetic, 19K decider and 19K constant combinators, plus 8,800 lamps
No mods, no Lua — just combinators, wires and lamps
It chats in English and knows Factorio facts (biters, science, trains, Space Age planets), and it can tell a short story or a joke
How you talk to it
You type a question letter by letter into a row of constant combinators, put a ✓ after the last letter and flip the SEND switch. A progress bar shows the model reading your question, then the answer lights up letter by letter on a giant 5×7 pixel lamp board, about 7 seconds per letter at 60 UPS.


you: hi!
bot: hello! i am busy with my factory, but i always have time to talk.
The trick that made it possible
In 2.0, an arithmetic combinator can do each * each across the red and green wires and sum the result, so one combinator computes a whole dot product. Each row of a weight matrix lives in a constant combinator, and one constant plus one arithmetic combinator make a neuron with 256 or 1024 inputs. It also turns out a single wire can carry 2,169 different signals (items, fluids, recipes, even entities like the atomic bomb wave), which is how 256-number vectors travel through the network.

Softmax exp is a staircase of 134 deciders, LayerNorm's square root is Newton's method, and each layer keeps a 256-cell KV cache made of sample-and-hold latches. Everything is integer math and matches a Python reference bit for bit.

Try it yourself (needs Space Age)
Save file, blueprint and full source: https://github.com/RomOSTiny/factorio-transformer

Next step: a model several times bigger. Happy to answer any questions about how it works!

[BHakluyt]

Video, or it didn't happen.

Very interesting and very cool.

I've been trying a few weeks with Claude AI free account to make a micro language model for me in game but the furthest we got is a mix of Eliza like reflection and a Markov generator...