Factorio Forums

I play with a friend on a dedicated server. Often, I'll play for a bit by myself and then stop and go do something else for a while and come back later. When I rejoin the server I have to redownload the map again, even if the game was paused the entire time I was disconnected and nobody else connected at all. I was thinking that a possibile optimisation could be:

1. Save the map automatically (to a temporary save file or something) when disconnecting from a multiplayer server. Maybe this would only be done in the case that there is only 1 player on the server, since if there's not then the save would be outdated instantly and therefore pointless.
2. When connecting to the server, compare a timestamp/checksum/some other value with the server to see if the map that the client has is the same as the server's.
3. If it is, just load the save rather than downloading it first.

Obviously on larger servers this would be irrelevant, but would save a lot of time for those of us that play with only a few other people and not necessarily at the same time.

I have often thought that it would be useful to have multiplayer map downloads use something like rsync, which can compare the files and download only the differences. It's not an issue for small maps, but if you have a large save file with several people playing then it can take quite a while.

Hmm. And what should we do with the players that complain they mixed up the save with the server and now they played x hours but cannot use, what they built.

@ssilk You mean people that started the game as a solo game with the local save ?
You could just add a [local] flag and print a warning when it is started or you could hide them by default if really it is that much of a problem.
It mostly seems to be a non problem though as the workflow to connect to a multiplayer game isn't the same as the one to load a game ?

I have the same problem playing with friends. We don't have a dedicated server, but the problem is kind of the same.
We all play together and mostly stop together.

Here are the problems we encounter:
- We always need to redownload the whole factory and it takes 10 to 20 min for each player
- We mostly can't have multiple people downloading at the same time or it takes forever + the server drops people that already connected because all of the bandwidth is used and they have to restart the whole thing again...

After that, the game plays smoothly without any network problem, but we need like an hour to start really playing.

I'm one usecase, but to me it feels like a very important feature that can have a very easy first version.

Here is a proposition:
When you disconnect, you are already asked if you want to save the game. If you save it, then Factorio uses it as local cache.

The server sends the checksum of what will be sent.
The client recv the checksum. If there is a local version of the save, it checks the checksum against it.
To know if a save is a local version you need a way to link it with the server save you are trying to download.
The hackish way is to have the same name for both saves and the game you are trying to load. Would not cause problem even if you have a game with the same name but unrelated (you will just do a useless checksum)
The better way is to be able to know if it's the same world with some kind of metadata.
If you found the local version of the save with the same checksum as what you are trying to download, you take the local one and avoid the download.

@ssilk Is it possible to mod this ? I would be willing to add that kind of stuff.

ssilk wrote: ↑Sat Nov 28, 2020 3:27 am Hmm. And what should we do with the players that complain they mixed up the save with the server and now they played x hours but cannot use, what they built.

that is what checksums are for. this is how the game already knows the mods mismatch server... checksum the save, too. duh

ssilk wrote: ↑Sat Nov 28, 2020 3:27 am Hmm. And what should we do with the players that complain they mixed up the save with the server and now they played x hours but cannot use, what they built.

This could be completely transparent to the player since it would appear to them to work in exactly the same way as it does now for multiplayer - the machine hosting is responsible for maintaining the up to date game save, but players can choose to save locally if they wish. The change is purely to the method used to transfer the game file.

The point is that connecting to a multiplayer game forces the host to upload the entire save file to the connecting client. If the game is large, this could be upwards of 50-100MB or so. Not so bad if it's one person connecting, but if you have many people joining at once and don't have a particularly fast upload speed, it can take a long time to get everyone loaded in. If someone drops connection because of an internet interruption, they have to load the whole thing again.

If you used something like the rsync protocol to transfer saved games, the client could cache the most recent version it has of the saved game, and would only need to transfer what had changed since the last connection to that game. The player would not need to do anything to create the cached version, since the client would do it automatically.

Instead of having the host have to upload hundreds of MB of data each time people want to play, they'd only have to upload a minimal amount. Obviously the first connection to the game would require the complete upload, but subsequent connections would be much faster.

tankslapper wrote: ↑Sat Jan 30, 2021 12:55 am
If you used something like the rsync protocol to transfer saved games, the client could cache the most recent version it has of the saved game, and would only need to transfer what had changed since the last connection to that game. The player would not need to do anything to create the cached version, since the client would do it automatically.

it wouldn't work very well for factorio. it uses a simple 128bit rolling checksum to validate that two files are not the same. that's the magic of rsync. it's not much. it's not super accurate.

to be accurate, the client would need to be able to present its actual save state to the server somehow. or the server would need to have a list of saves to compare it to.

then the server could use its local copy of this save and do some diff routine between the two and transfer only the differences.

it's not unachievable but it's of dubious value. a lot of opportunity to break things and who knows what kind of gains. that diff operation isn't free.

better off adding things like zstd compression for save files. 30% performance penalty to lz4 for gzip-9 style compression performance. the game already has zstd built in, i've seen it crash in that lib before. come on Oxyd, doooo itttttt