Ways to speed up map transferring in multiplayer.

[ManselD]

Send data in chunks and store them on the disk (for each client). Then if there's been an updated chunk, send it to the client when they connect to the server again.
Maybe hash the chunk to check whether it's been updated?

[The Lone Wolfling]

You can do one better than that: Use a Merkle tree.

(Essentially, you have a n-ary tree, where each (non-leaf) node is labelled with the hash of the labels of its children nodes.)

(There's a time versus bandwidth efficiency tradeoff here, as well. For minimal bandwidth use, you want to send only the top node, have them respond if they have it or don't, if they don't have it then send children of the top node, repeat as necessary. But if you want it to be faster you can just send the tree in a top-down approach, stopping sending children of a particular node if you receive confirmation that they already have that node.)

Still doesn't speed up initial map transfer, though.

[ManselD]

Ooh very fancy...

[MF-]

Also note that large part of the save is the replay data,
which has a "log" structure - ie. there exists a point before which the replay data is good.
Solutions to that are thus far simpler - I suggested one at https://forums.factorio.com/forum/vie ... 377#p51377

[cube]

These are all great suggestions, but if we wanted to do any of this, we would use the binary diff mechanism that is used in updates. BUT... I think that our effort will be better spent if we get rid of desynchronizations (at least of most of them) instead of making the recovery faster.

[SHiRKiT]

These are all great suggestions, but if we wanted to do any of this, we would use the binary diff mechanism that is used in updates. BUT... I think that our effort will be better spent if we get rid of desynchronizations (at least of most of them) instead of making the recovery faster.

Just want to point it out that there will be a point where fixing those desyncs will take too long and speeding up the recovery may actually be the better solution at that point.

[Psycix]

For me it's more about joining than re-syncing.

I think most of us play the same game mostly with the same friends, so I'd love to see a "synced save" feature, where the game is saved at the same tick (and state) for everyone. Then, the next day or so, as soon as I load up the game it is paused, and people who join and happen to have the save do not need to download anything (other than doing a quick hashing check for validity)
This can be expanded to synchronised auto-saves, where if someone joins with an older save available of his client, provided my client has that autosave as well, I only have to send him the diff patch.

[cube]

This makes sense. I've added it into our list for 0.12.0.

[bobingabout]

I've not experienced a Desync myself yet, but during setting up a multiplayer game, it's basically.... oh look, someone is joining, time to go make a cup of tea.

Especially when transfering to literally the oposite side of the planet, with a really slow upload, and Oz net at the other end, it takes 5 mins+ even with the most basic small map.