Parrallel processing in games & applications

Post all other topics which do not belong to any other category.
User avatar
boskid
Factorio Staff
Factorio Staff
Posts: 1785
Joined: Thu Dec 14, 2017 6:56 pm
Contact:

Re: Parrallel processing in games & applications

Post by boskid »

blazespinnaker wrote:
Sat May 07, 2022 8:45 pm
Wube limits it to user input for reasons that they have never made clear.
I am feeling like being trolled by responding to this but i will answer anyway to have something to link to in the future.

Determinism in context of factorio is basically a property that when you have game state S at tick t (lets call this S_{t}) and set of input actions to be applied in that tick (lets call those I_{t}), a result of the game update function Update is same for all players:

Code: Select all

S_{t+1} = Update(S_{t}, I_{t})
When a player joins the game, server saves the game to a save file (which is basically entire S_{t}), sends it to another player so both players have the same starting point. If server and client also have the same input actions, under assumption of determinism they will get exactly the same state in the next tick S_{t+1}. Similar to mathematical induction, since the base case holds (due to save transfer) and induction step holds (due to input action broadcasting and deterministic update), client and server will stay in sync for entire duration of client connection to a server as long as all the I_{t+x} are guaranteed to be the same. That means when you want to get a data from a game state, there is no need to transfer anything through a network because server and every client has those values and they are equal.

You can read slightly more about "part of game state" in my other post in this topic

Since there is literally no good reason sending anything being "part of game state" through a network (it would not be carrying any new information as every client already knows about those), only reason to send an input action is literally to introduce data available only one of the machines (which means those are not part of game state) and which are supposed to affect state of the game.

Player interactions are the most obvious source of the input actions, but those are not the only ones.

All server console commands issued through RCON are also broadcasted through InputActions to all clients because if such command changes game state, scripts on all clients have to apply exactly same changes. If a server command says "add 100 iron plates to entity X because they are being sent from another clusterio node", all clients that are already connected need to know that the amount of items in a given inventory is to be changed and this cannot be deduced from game state in previous tick alone - those are broadcasted as input actions.

There is also third source of input actions, it is really subtle but it is one which is under mods control: it is a LuaPlayer::request_translation call. As it is implemented, all game instances not related to the given player are simply ignoring those LuaPlayer::request_translation calls, but the one instance which is of related player does the translation of LocalisedString into a regular string and then this string is sent through an InputAction so all other script instances will get the same value. In theory it should be relatively easy to abuse this mechanism by requesting translation of complex localised string with the data to be broadcasted and simply ignoring the translation result: in that case instance of a mod running on one client, even when being in a desync state (having some state differences to other clients) could properly introduce those data to other clients.

Now when all the important technical bits were explained, i can give you 2 reasons why we do not allow mods to send input actions:

1/ Throughput of internet connections and scaling. If your mod would need to send 100 bytes of input actions every tick for every client, that is basically a 100 * 60 * N bytes of data incoming to the server every second. Since the server is doing a broadcast of those input actions to all connected players, now server has to send a total stream of 100 * 60 * N * N bytes every second. Lets assume N = 100 players (even when i saw games where N was going up to 500). That gives 60MB of data to send by the server every second. Its basically 480Mbps of upload from the server side and its only 100 bytes every tick. This is not acceptable, by having N in square it becomes large really fast. Because of that we try to send as few data through input actions as possible and only in reaction to some reasonable events and when events are going more often (even player moving cursor from one entity to another is sent through an input action as a selection changed input action) they can take down to 3 bytes (where 1 is input action type, 1 is player index and 1 is encoded position in a coarse grid around last selection position).

2/ Design of scripting support assumes scripts should not have anything in their state that is not part of game state. That means we never expose anything not part of game state through a Lua-API and as such we never give any reasons for mods to need to send an InputAction. It is a lot easier to write a correct mod that is only based on a data which are part of game state compared to what it would be if you would have access to non game state data. In case of access to non game state data, literally every Lua-API read/write/call would also need to have sets of remarks which would be only useful for advanced mod developers and incorrect usage of them would almost always mean a super hard to understand or reproduce desync event.
Those remarks would need to touch details such as:
- remark if given access is causing game state changes - there are even some places where a read operation is causing game state changes.
- remark if given variable is part of game state - usage of values which are not part of game state to make a direct game state change would immedaitely cause a desync as they violate our assumption of deterministic update (mod script update is part of the update). Those values would need to be sent through input actions to be correctly introduced for every script instance.
- remark if given variable is accessible at all on every instance - for example a headless server will not have any local players.

I am aware it would be powerful to write scripts which would be aware of local player and doing some computation only on one instance (first example i could think of is a helmod which could find a solution using CPU power of the player requesting the computation and then sending the results through input actions - that would make other players with slower CPU not be punished by other player using this tool), or they could access some data that were previously not accessible (like local clock or last tick time measurements), or even interact with latency state (absurdly non realistic, that would be a separate API to maintain which we do not want to go into). I am simply not sure if it makes sense to introduce scripting API which would be hard to correctly use, hard to debug and having only a tiny benefits (maybe there are some with huge benefits).

blazespinnaker
Filter Inserter
Filter Inserter
Posts: 532
Joined: Wed Sep 16, 2020 12:45 pm
Contact:

Re: Parrallel processing in games & applications

Post by blazespinnaker »

Thanks, great clarity there. Nothing but respect.

"add 100 iron plates to entity X because they are being sent from another clusterio node"

fwiw, this was precisely what I described above. And now looks to me that it is fully supported via lua comms. sigh.

I think there is some arbitrary json serdes going with clusterio rather than basic rcon, I can imagine different approaches to supporting that.
OptimaUPS Mod, pm for info.

Post Reply

Return to “General discussion”