Factorio Net Protocol Spec?

[smudj]

Hey there everyone.

I'm interested in making a version of the Factorio server that wouldn't be peer-to-peer (since we all know how annoying it is when there's that one 250ms ping person on..)
But to do so, I need a specification of the net protocol..
So I'm reaching out to the devs in this post (who I've heard frequent the forums) in hopes of finding a way to get the protocol without smashing it up with wireshark and decompilers.

If you're a dev and you don't wish to provide this information, just tell me and I'll bugger off.
If you're not a dev, please help me get in contact with them (as I don't know a good way to do so)

Thanks in advance for anyone who replies,

smudj.

[daniel34]

I don't know about the devs providing their net specifications (but I wouldn't bet on them doing so).

What I know is that even if you had the complete specifications you still wouldn't be able to write your own Factorio server. The server isn't just there to manage communications between players, it is also a peer itself and simulates the world. The only thing exchanged between peers is their keyboard inputs (character movement, inventory changes,...), every other thing is computed by all peers themself. If you wanted to write your own server you would need to essentially copy the complete code of Factorio.

[prg]

Let's assume you had access to the protocol spec and managed to reimplement the game logic, in which way would the "server" do things differently from a normal peer that would help with latency? The "clients" would still expect to run the simulation in lockstep with each other, you don't have control over them so you can't just have a "server" calculate everything for everyone and send only the results around. The best you could hope for would be the already implemented latency hiding.

[smudj]

I don't know about the devs providing their net specifications (but I wouldn't bet on them doing so).

What I know is that even if you had the complete specifications you still wouldn't be able to write your own Factorio server. The server isn't just there to manage communications between players, it is also a peer itself and simulates the world. The only thing exchanged between peers is their keyboard inputs (character movement, inventory changes,...), every other thing is computed by all peers themself. If you wanted to write your own server you would need to essentially copy the complete code of Factorio.

Yes, the server would need to do a lot of processing and handling of data, but it wouldn't need to be as comprehensive as if it were a fully-fledged client.
It could get away with minimal processing and handling

Let's assume you had access to the protocol spec and managed to reimplement the game logic, in which way would the "server" do things differently from a normal peer that would help with latency? The "clients" would still expect to run the simulation in lockstep with each other, you don't have control over them so you can't just have a "server" calculate everything for everyone and send only the results around. The best you could hope for would be the already implemented latency hiding.

A simple approach would be to compute, say, n ticks per second, where during each tick, every entity is updated (furnaces smelt; miners mine; etc).
This would be run separately to the client packet receiving, which would not be tethered to a time.
Every tick, packets are sent to each client stating what the client was last doing (according to their last packets).
If no packets were received between ticks, it would simply send the previous data again.

This way all connected clients see all other connected clients as doing stuff, even though they aren't.
This means all latency ends up with client -> server and server -> client, not client <-> client.
In other words, this would make it such that one client isn't impacted by another client's latency; only by their latency to the server.

I think this would substantially improve the multiplayer experience.

[daniel34]

Every tick, packets are sent to each client stating what the client was last doing (according to their last packets).
If no packets were received between ticks, it would simply send the previous data again.

It doesn't work that way. Each tick every peer updates all the entities on the map for that tick, and then computes a CRC checksum of that map. This checksum is then sent to all other peers and if even only one peer has the wrong checksum calculated this leads to a desync, requiring the desynced peers to redownload the map from the other peers.
That also is the reason you need to have the same version to play multiplayer and to replay a save, even tiny differences in the code can change the checksum and lead to a desync.

You could certainly write your server in a way that doesn't need a checksum, but then you'd also have to write your own client for that server, effectively leading to writing all the code of Factorio from scratch. It would certainly be easier for you to just write your own multiplayer game

Example log entry (Peer 0 is the server):

Code: Select all

691.510 Error NetworkInputHandler.cpp:343: Multiplayer desynchronisation: crc test(CheckCRCHeuristic) failed for mapTick(18246900) peer(3) testCrc(-749056919) testCrcPeerID(0) currentCrc(1870529033)
691.510 Info NetworkInputHandler.cpp:419: Desync specification: Reference CRC = 3545910377, reference peers = {0, 1 (daniel34), 2 (Bulisek)}; desynced CRC = 1870529033, desynced peers = {3 (TiTaN)}

[Oxyd]

There is no specification of the protocol, other than the source code. And we're not giving away the source code.

And, as others have mentioned here, what you are trying to do would require you to reimplement the whole game, because you'd have to change the clients as well to be able to work with your new server. So, have fun & good luck.

[smudj]

There is no specification of the protocol, other than the source code. And we're not giving away the source code.

And, as others have mentioned here, what you are trying to do would require you to reimplement the whole game, because you'd have to change the clients as well to be able to work with your new server. So, have fun & good luck.

Fair enough, haha.

Every tick, packets are sent to each client stating what the client was last doing (according to their last packets).
If no packets were received between ticks, it would simply send the previous data again.

It doesn't work that way. Each tick every peer updates all the entities on the map for that tick, and then computes a CRC checksum of that map. This checksum is then sent to all other peers and if even only one peer has the wrong checksum calculated this leads to a desync, requiring the desynced peers to redownload the map from the other peers.
That also is the reason you need to have the same version to play multiplayer and to replay a save, even tiny differences in the code can change the checksum and lead to a desync.

You could certainly write your server in a way that doesn't need a checksum, but then you'd also have to write your own client for that server, effectively leading to writing all the code of Factorio from scratch. It would certainly be easier for you to just write your own multiplayer game

Example log entry (Peer 0 is the server):

Code: Select all

691.510 Error NetworkInputHandler.cpp:343: Multiplayer desynchronisation: crc test(CheckCRCHeuristic) failed for mapTick(18246900) peer(3) testCrc(-749056919) testCrcPeerID(0) currentCrc(1870529033)
691.510 Info NetworkInputHandler.cpp:419: Desync specification: Reference CRC = 3545910377, reference peers = {0, 1 (daniel34), 2 (Bulisek)}; desynced CRC = 1870529033, desynced peers = {3 (TiTaN)}

Didn't know about that. Nevermind my idea then. Ain't gonna work.


Could someone please lock/delete this thread? I can't seem to figure out how to do so.

[daniel34]

Could someone please lock/delete this thread? I can't seem to figure out how to do so.

We usually don't close/delete such topics, as others might have the same question.
(Well, maybe not in this case, but it still is an interesting read for everyone who wants to understand how the server actually works.)