Sync multiplayer saves to avoid downloading big maps / Synchronizing Maps / Avoid Multiplayer Download / Use local copy

schmetterling · Post by **schmetterling** » Sun Nov 17, 2019 9:47 am

TL;DR

Saving and immediately stopping the simulation in a multiplayer game would leave all players with exactly the same save file. This save could then be used to reload the game at the exact same point for everyone, removing the need to download the map.

What ?

It would require another kind of saving mechanics. The server pauses, saves, make all the player save, and immediately eject everyone from the game.
When starting a multiplayer game, the server loads the file and waits for other people to join with the same file. Checksums are exchanged to ensure that everyone has the exact same file. At some point the server decides to start the game, and everyone is immediately in the same game without any download. It's just a matter of loading everbody's local copy. It would also remove the catching up.
From a GUI perspective, it would need some "Save & Stop" in the in-game menu, and something like "Host synced saves" in the main menu.

Why ?

As maps get bigger, it gets more and more tedious to join a game with a weak internet connection. When playing with the same people, always at the same time (like a normal coop game), always downloading the same maps seems unnecessary.

(Thanks to all the team for making this really nice game)

Cyrilis · Post by **Cyrilis** » Wed Jan 27, 2021 3:01 pm

Me and a friend have been working on a factory for quite some time now. We only play on this map together, so we always have the same savegame. It is kind of exhausting to have to download the map every single time (and meanwhile it takes quite a bit).
So here is my idea:
Synchronized savegames:

- both host and player save the game on their local computer at the same time (host saves and triggers a save option for the other player. The savegame gets a unique ID, like a number combined from map seed, time strap and a random number of digits)
- next time both players load the game from local and as soon as that is done the Host can trigger the game to start from the "paused" mode.

what do you think?

Xarovin · Post by **Xarovin** » Fri Jan 29, 2021 12:28 pm

It's a well thought out concept, but I imagine it'd be way too much effort to implement for how niche it is.

Kyralessa · Post by **Kyralessa** » Fri Jan 29, 2021 2:48 pm

Better yet, a hash of the saved game contents. Then it can be confirmed that the contents are the same on both sides.

Of course, whether this would save time depends on whether the hashing function would be faster than saving a game.

asheiduk · Post by **asheiduk** » Fri Jan 29, 2021 4:02 pm

Xarovin wrote: ↑
Fri Jan 29, 2021 12:28 pm
It's a well thought out concept, but I imagine it'd be way too much effort to implement for how niche it is.

Well, my friends and myself have the same usecase and would benefit too!

asheiduk · Post by **asheiduk** » Sat Jan 30, 2021 1:40 pm

asheiduk wrote: ↑
Fri Jan 29, 2021 4:02 pm
Well, my friends and myself have the same usecase and would benefit too!

After some thinking I'd like to explain some of the benefits of that proposal:

Most of my friends and I have a highly asymmetrical DSL connection. In my case: 27MBit/s downstream, 2.7MBit upstream. For downloading huge maps (~240MB) Factorio's custom network protocol is inferior to TCP: The download is capped at ~2MByte/s because my upstream is saturated. (sidenote: Plain TCP-downloads usually can do ~3MByte/s while using only about 10% of the upstream.)
Having no huge up/download due to synced saves would just solve that cap.
The time to enter the game could be reduced considerably also because the download just does not happen.
Hosting a server in the AWS cloud (or something similar) causes charges for the CPU and for outbound network transfers. In the case of a ~240MB map the network cost could be reduced considerably with synced saves.

Xarovin · Post by **Xarovin** » Sat Jan 30, 2021 2:27 pm

Alternative idea which may require less implementation:

As a user, you can save the current state of the game.
The server holds an optional buffer of previous commands.
Upon connection, if you have the save file and it is within the buffer period, then load the save and fast forward to the current server state.

Cyrilis · Post by **Cyrilis** » Thu Feb 04, 2021 10:35 am

Xarovin wrote: ↑
Sat Jan 30, 2021 2:27 pm
Alternative idea which may require less implementation:

As a user, you can save the current state of the game.
The server holds an optional buffer of previous commands.
Upon connection, if you have the save file and it is within the buffer period, then load the save and fast forward to the current server state.

That sounds interesting... I'm not too familiar with this command buffer stuff. Could you please give a walkthrough how exactly that is done?
*EDIT* sorry, misread your post. Thought that was an already existing, implemented method... Forget about the walkthrough.

Taneeda · Post by **Taneeda** » Fri Feb 05, 2021 7:02 am

+1

Same usecase for my friends and me, I like the Idea

i-make-robots · Post by **i-make-robots** » Mon May 17, 2021 2:33 am

Hello and thank you for this great game. I'm currently into a long Space Exploration session over many weeks with friends 10 time zones away. Every time they connect now is a very long download and sync. This seems a little odd because they have saves and autosaves.

My suggestion is that reconnecting to the same MP server should check recent autosaves for the same map. if it is there then only sync the diff.

Again, thank you and stay awesome!

ptx0 · Post by **ptx0** » Mon May 17, 2021 2:40 am

i-make-robots wrote: ↑
Mon May 17, 2021 2:33 am
Hello and thank you for this great game. I'm currently into a long Space Exploration session over many weeks with friends 10 time zones away. Every time they connect now is a very long download and sync. This seems a little odd because they have saves and autosaves.

My suggestion is that reconnecting to the same MP server should check recent autosaves for the same map. if it is there then only sync the diff.

Again, thank you and stay awesome!

waaaayyy too complicated IMO you're better off just not playing SE over Multiplayer. I know, because I had LTE bandwidth capped at 50GB and couldn't do it for the longest time. what would help is zstd compression with dictionaries - there's a feature request. but this is really low priority to solve for Wube.

Post by **ssilk** » Tue May 18, 2021 7:12 am

joined with quite similar topic

In my eyes this makes sense. It could be checked quite fast, if two (tree...) saves have the same checksum.

Gully · Post by **Gully** » Sun Mar 20, 2022 4:19 pm

TL;DR

When logging into a MP server, check if the client already has the savefile, skip the download in that case.

What ?

If you could start a MP server paused and wait for all clients to login, the login could be faster. (provided all clients have an identical savefile). It should only be necessary to compare a checksum and load on all the machines, not wait for download and catch-up.

Why ?

Well, a friend and me are building a large factory. (Thanks to Earendel for the awesome Space Exploration mod). The save is approaching 120MB, and when we start, it downloads the game-state at a painfully slow bandwidth. (Yes we tried to remove the limit). It would be great if we just create identical savefiles (pause, both save to the same filename) and in the next game-session no download is needed. I don't know how much work I'm asking for, but if it's not much it would be great.

Post by **ssilk** » Mon Mar 21, 2022 6:49 am

joined thread with similar topic, joined also some older threads from 2021

Factorio Forums