Idea/Concept: Stream map data on join

[Tr1cKy]

This is a fairly simple proposal for a fairly simple problem ... but with a possible complex implementation. I certainly don't expect this idea to be implemented as is. There could be many things I haven't thought of etc. It is also possible that the devs have much better ideas in the backlog.

Purpose:
Reduce or completely eliminate the "pause-and-download" period that occurs when players join a game.

Basic concept:
- Stream the game world from the peers to the joiner, starting at the player's "starting position".
- Allow the joiner to play whilst the game world is still streaming in.

Details:
The joiner's world/mapview starts at the player's position and expands outward - much like how the map is generated - except that the data is gotten from the other players rather than from the seed data.

The status of the world is expanded one squared unit at a time while relying on the other nodes to ensure that events affecting "edge" squares are applied. This would be similar to how the world expands when exploring.

When fully synced, resume normal "fully-simulated" processes.

[bobingabout]

Due to the way that the game works in multiplayer, your idea is impossible to implement.

In short, here is why:
To minimise traffic between peer and client of possibly millions of different entities, the entire simulation is run on every machine simultaniously. Since all peers have all the same parts, they should all make the same descissions at the same time.
Because all parts are required to accurately reach the same result (Random values will be drasticly different if more, or less are calculated, even by 1 value), the entire map needs to be available, and identicle on all computers.

if something is detected to be different on any one peer, a Desync is triggered, forcing the redownload of the entire map to bring it back into sync.


This restriction becomes a necesity because of the way the devs chose to write their multiplayer code, to try and change this would require a complete re-design and re-write of the multiplayer system. Your sugestion would work in other games because they are usually server/client relationships, where the simulation is calculated on one end only, and minor syncs are sent by the server to make sure the client is looking at the same thing. to keep trafic to a minimum in these situations, the behaviour is predicted. This wouldn't work in factorio, because even the slightest thing can cause massive diferences via the "Butterfly effect".



take for example a low power situation. On one peer you are producing more power than the other due to a section of the map that has power plants not being receaved yet. due to having less power, mining is slower, so less resources are transfered to your factories. Both due to having less power, and less resources, your factories produce less ammo, when in turn provide less ammo to your turrets.
Before you know it a swarm of biters are attacking your base. Player A manages to kill everything with the gun turrets, Player B has a deficiency of Ammo, and the biters eat the walls... the turrets... and start rampaging through your base.


This could be the result of simply 1 less boiler, or steam engine, if you give it enough time.
now imagine half the map missing... just doesn't work in this game.

[SHiRKiT]

The issue will always be borders. Let's say that the client that joined loaded Chunk X only. The other clients knows about Chunk Y, and they are computing stuff. So now, some product of the Chunk Y will be sent to Chunk X (let's iron plates on a belt from Chunk Y to Chunk X). Now the recently joined player doesn't know about Chunk Y, and the iron Chunk X will start moving, but it won't receive additional input. There are thousands of ways for everything to mess up with the current multiplayer code. The main issue in this game is that, even the two furthest away chunks from each other can have an instantaneous impact at the gameplay.

[keyboardhack]

Streaming everything would require you to send everything that happens in the game to the client 60 times/sec(update rate of factorio).

Lets keep it simple and just focus on the items on belts.
Lets say we have 200.000 items on belts moving(standard factory size) and lets say that one items position size 4 bytes(it's probably more).
The bandwith you require to update one client with only the belt items would require a bandwidth of: (200.000 * 4bytes * (60/sec)) / 1.000.000 = 48megabytes/s. That's 384 megabits/s.

You would need a ridiculous internet connection to stream part of the game to one client. Streaming the whole game to multiple people would just be downright impossible.


So lets not send all that information because we can do better. we can just tell the client to update the belts and then the client will make sure to update the items positions. Now we are already simulating part of the game on the client so why not simulate everything on the client and save your internet connection?

There really is no way out of simulating the game on every computer.

[Tr1cKy]

Here's an idea. Use it - or don't. I'll defend the concept if I can. The devs might have better ideas. So far I just hear excuses - but not coming from the devs.

Due to the way that the game works in multiplayer, your idea is impossible to implement.

I didn't say it would be easy But it certainly is not impossible.

This restriction becomes a necesity because of the way the de

Only edge tiles would need to be streamed - and it would be temporary - only while the player is joining. There are many possible measures that could be used to ensure the environments remain identical. A simple one would be a an occasional status checksum for game regions. BUT - when the entire map is synced completely, there is no need for further "streaming". This isn't a proposal to drop the existing multiplayer system. Who knows, maybe that simple check could be used to help prevent a desync for already-joined players. Instead of an error and dropping the connection, you get a warning message.

The issue will always be borders.

Exactly!:

while relying on the other nodes to ensure that events affecting "edge" squares are applied

Streaming everything

Why stream everything? You only need to stream the edges (and I suppose, occasionally, mismatched data - per the checksum idea mentioned above). In theory you need only a 5-10% higher throughput capacity than the game normally uses. The faster your connection, the faster your game gets into "full sync".

[quinor]

Even if it is possible, it requires the game engine to be entirely rewritten from scratch. Yeah, it works, for example in minecraft, but 1) minecraft is client-server and server forces its game state into the player 2) there are terrible desyncs in minecraft 3) minecraft engine is not deterministic, it uses heuristics. It is not possible in factorio due to mass of updating entities (have you ever tried to play minecraft server with massive redstone? It's terrible and often does not work as predicted) which
Does not happen in minecraft.

[Tr1cKy]

Even if it is possible, it requires the game engine to be entirely rewritten from scratch.

I'm a developer and I don't see why. In fact I know it would require a lot of work - but it doesn't require the game engine to be entirely rewritten (unless the game engine was written really badly ... which I doubt).

Maybe the resistance is that this looks difficult - that the cost-benefit isn't so good. I don't see it that way*, but an opinion is not always so easily changed or understood.

*
Benefits:
- easier to implement desync self-fix (wasn't even my idea!)
- no waiting for 15 minutes while players try to join from their slow internet connection

Detractors:
- Complex implementation
- Dev time that would otherwise work on other more urgent fixes or features

[ssilk]

I think that is a good idea.

My thought about that:
- New player joins, and a small part of the map (a region, https://forums.factorio.com/wiki/inde ... tle=Region) is shared (1024x1024 tiles for example), plus: from now on the game logs not only the player interaction, it logs also everything, which comes from outside this region into this area and vice versa. So the map-data is relatively small, but the replay-log might get big relatively fast.
- The new player needs to download this map-part/region and then the engine needs to replay the replay log as fast as possible. This means: The game can still continue for the others, the other players don't need to wait, all is done in background, it might of course influence the performance a bit, but that is not really a huge impact.
- When that runs smooth and the replay-log is in the same position, the new player can download the next part of the map. In this case, we need only to log the changes for the whole area, that the new player has already downloaded. (*)
- And so on.

In other words:
- The new player downloads a part of the map and replays the replay-log (which includes the changes on the borders), until his game is in sync with the others.
- Then he downloads the next part of the map and replays the log again.

At some time, he has downloaded the whole map and he can also do changes himself. Until then he can only run around into the already downloaded blocks and look around.
The other players don't recognize that (or only, if the bandwith is too low to keep the replay-log updated or the new players computer is too slow to replay the log fast enough). For that case it would be enough to reduce the game speed, make only an update each 1/30 secs or so...

For the worst case I think we can think of situations, where we have for example 50 belts running in an out of a block. So we have 100 belts with each 30 items per second (blue belts), that is 3000 changes per second, each with 16 byte we have: 48 kByte per second. I think that is at the edge, but in a LAN or good internet connections it would work with no problems.


(*) The interesting fact here is: The area grows quadratic, but the border of that area only linear.
Example:
First region is 1024x1024 tiles = 1048576 tiles
Border is: 4 * 1024 = 4096 tiles

After downloading the second region:
Area: 2048 x 1024 tiles = 2097152 tiles (of course just twice as much)
Border: 2 x 2024 + 2 * 1024 = 6144 (which is only factor 1.5 and not 2!)

After downloading the third area (which looks then like so:

Code: Select all

    XXXX
    XXXX
XXXXXXXX
XXXXXXXX

Area: 2048 x 1024 +1024 x 1024 tiles = 3145728 (3 times as much)
Border: 2048 + 2048 + 4 x 1024 = 8192 (Only factor 2, not 3)

So it looks like the border grows only with 2/3 of the area.

But for the next step (4 times the area), the border is still 8192 (4 * 2048)!! Still only 2 times bigger, than at the start.

I think this is, why this idea might really work in practice.

[quinor]

I agree with ßilk that it's possible and not that hard to allow non-blocking map download, way he described is similar to my idea of non-blocking savegame. The hard part (at least as far as I know) is allowing the peer to actually play and alter map while he is still downloading it. I'd even suggest not to allow him to run but rather view the map in some kind of ghost-mode, as running could alter the game state (biters, collisions with trains etc.)

[Tr1cKy]

some kind of ghost-mode

Good catch - I had not thought of that. Much less work to do if the idea is used.

[ssilk]

Well, this is really complex. And it is not really clear, if it is really possible. Take for example the pollution. That spreads over big parts of the map. Which means: The local pollution is dependent on the total pollution. Which we don't know. And it will make problems, if the algorithm, which calculates pollution is dependent on the map around that local part and not possible to split this up into smaller parts.

But that is really relatively theoretical. More practical is, that we have global stuff, that influences local stuff, like electric/circuit network or logistic network, that might make problems, too.

[Martc]

For the worst case I think we can think of situations, where we have for example 50 belts running in an out of a block. So we have 100 belts with each 30 items per second (blue belts), that is 3000 changes per second, each with 16 byte we have: 48 kByte per second. I think that is at the edge, but in a LAN or good internet connections it would work with no problems.

But good internet connection and LAN don't need this. Slow connection 10-20 kB/s (according to 15 min download time) will be probably overloaded with updates packets. Instead of downloading new parts of world, it will be trying to catch up with update packets.

I think much better way is save size reduction, now significant part is replay information and I think there will be more thinks that are not essential for player to be able join. And there are many other ways how to reduce save size, like storing differences instead of exact positions and so on.
I think this mechanism is overkill and don't worth the time to implement it.

[judos]

The idea is great. Sounds to me like nobody here want's to have non-blocking mp connecting.

One solution to the problem would also be:
- When the player joins the server saves the game state which is then transmitted. The other players can continue. When the initial game state is transmitted, the server pauses for the player to join but only sends a diff version from the already transmitted game state (much faster operation, while the other players are blocked)
- Also extendable as following: when the player disconnects, the server and the player stores 1 copy of the same map state. When the player reconnects, instead of a full map transfer, only a diff is sent (saves a lot of data)
- Non-blocking join: Split the download task into two steps: Downloading static data (environment, no blocking for the other players), download dynamic content (everything else.)