Blacklist from checksum

[Soul]

https://forums.factorio.com/viewtopic.php?p=145166#p145166

• Changes
  • Mod checksums are calculated when the game starts and are compared with other peers when joining a multiplayer game. This is to ensure everyone has exactly the same mod in order to prevent desyncs caused by local changes made to mod files.

So Factorio 0.12.30 partially broke a mod called Red Alerts which is an audio event mod. This mod is designed to be customized (config.lua) and can be different per-player depending on their personal preferences (if someone wants a different sound pack than other players). Because of the forced checksums if you have a different config file than the server you can't join the server.

It would be nice to have a blacklist of files that are not checksummed by the server, so that such customizations can once again exist. So by default you can still checksum all of the files, but give the host the ability to specify exceptions.

[bobucles]

Wouldn't that be a whitelist, since it's allowing things through?

[ssilk]

Black- or whitelist: That would mean to break determinism. It very, very sure won't gonna happen!

Instead, the mod needs to be changed to make it multiplayer-safe.

For example in a way, that all configurations of all players needs to be known by every client (global state) and it just plays only the configuration of the current player. It's a bit more complicated then I describe it here.

[Soul]

Wouldn't that be a whitelist, since it's allowing things through?

A Whitelist means "I only want these items"

A Blacklist means "I want to exclude these items"

[mattj256]

Instead, the mod needs to be changed to make it multiplayer-safe.

For example in a way, that all configurations of all players needs to be known by every client (global state) and it just plays only the configuration of the current player. It's a bit more complicated then I describe it here.

Is there an example of an existing mod that does this? This sounds tricky to implement.

[keyboardhack]

Black- or whitelist: That would mean to break determinism. It very, very sure won't gonna happen!

Instead, the mod needs to be changed to make it multiplayer-safe.

For example in a way, that all configurations of all players needs to be known by every client (global state) and it just plays only the configuration of the current player. It's a bit more complicated then I describe it here.

That's not possible to do with such things as sounds/graphics and such as they have to be defined in a data.lua which means the hash would be different with different configs.
In a control.lua you could have different configs for different users, but you wouldn't be able to use them for anything super specific as the other players will still see/hear the same as you do.

[Zeblote]

Black- or whitelist: That would mean to break determinism. It very, very sure won't gonna happen!

Instead, the mod needs to be changed to make it multiplayer-safe.

For example in a way, that all configurations of all players needs to be known by every client (global state) and it just plays only the configuration of the current player. It's a bit more complicated then I describe it here.

That's not possible to do with such things as sounds/graphics and such as they have to be defined in a data.lua which means the hash would be different with different configs.
In a control.lua you could have different configs for different users, but you wouldn't be able to use them for anything super specific as the other players will still see/hear the same as you do.

Clearly the mod was working before, so what's your point?

[ssilk]

Hm.
Please try to understand that determinism in Factorio means, that every player runs an exact copy of the same game. The game cannot distinct, if the difference comes because users want to hear different sounds or if it influences game-logic, cause the config-value is more or less global and even if it might be only used for different sound, it could also be used for something else - there is nothing, which disables that, but this check.

A blacklist for mods, doesn't help, cause other mods could also use that config-value.

Hmmmm... Think of two parallel-universes. If in one universe a butterfly makes a an extra flap the universes differ.


Here the difference is then between Player1 has a hurricane, while Player2 has non. The game checks this different state then (not the flap, but the hurricane) and that will lead to a desync. This is very bad. To avoid this, this check was implemented.
...

So to come back to the question: I think the point is, that Factorio is currently not able to handle this and needs to be exended in some way to enable that, cause I think it is useful.

And I don't know exactly the background, but I would say that it formerly worked was the error - that is fixed now.



[Off-topic: I think this is something which must run in a limited environment, where only operations/function-calls allowed, that doesn't influence the determinism. In bigger database-engines this works equally; there they distinguish between non-deterministic and deterministic functions. Here it must be distinguished between functions that cannot influence the determinism, like different colors, different graphics, different sounds, and more...]

[Zeblote]

You can have determinism with different actions on each client as long as they have no other effects (like playing a sound printing a message) or as long as you synchronize the game state again when it becomes relevant for other clients (like client-sided gui code).

The problem isn't that it's not possible, the problem is that it's not exposed to the mod api. Factorio uses tons of client-sided code internally.

[Rseding91]

You can have determinism with different actions on each client as long as they have no other effects (like playing a sound printing a message) or as long as you synchronize the game state again when it becomes relevant for other clients (like client-sided gui code).

The problem isn't that it's not possible, the problem is that it's not exposed to the mod api. Factorio uses tons of client-sided code internally.

Indeed. Doing it proper is something that I'd like to look into for 0.14: client side mods.

[seronis]

You can have determinism with different actions on each client as long as they have no other effects (like playing a sound printing a message) or as long as you synchronize the game state again when it becomes relevant for other clients (like client-sided gui code).

The problem isn't that it's not possible, the problem is that it's not exposed to the mod api. Factorio uses tons of client-sided code internally.

Indeed. Doing it proper is something that I'd like to look into for 0.14: client side mods.

Best situation. Disable (or at least allow server to disable) the entire hash check. People should be able to join a server they know will work. Mandatory hash check when its preventing valid situations is inexcusable

[bobingabout]

Client side mods would be useful.

I mean, my clock mod, all it does is add a button, which displays the game world's time of day. Clicking it does nothing, it contributes absolutely nothing to the state of the game, if only one person had it, and the others didn't, it wouldn't actually break anything, yet, because it runs a script, everyone needs that script.

[Zeblote]

You can have determinism with different actions on each client as long as they have no other effects (like playing a sound printing a message) or as long as you synchronize the game state again when it becomes relevant for other clients (like client-sided gui code).

The problem isn't that it's not possible, the problem is that it's not exposed to the mod api. Factorio uses tons of client-sided code internally.

Indeed. Doing it proper is something that I'd like to look into for 0.14: client side mods.

Now that's nice to hear

0.14 has been confirmed for best update.

[ssilk]

I mean, my clock mod, all it does is add a button, which displays the game world's time of day. Clicking it does nothing, it contributes absolutely nothing to the state of the game, if only one person had it, and the others didn't, it wouldn't actually break anything, yet, because it runs a script, everyone needs that script.

You as modder cannot know that. The game (or your mod) can be extended so that it suddenly does.

We had here for example a discussion about why the real-time (the computers clock) was removed from the game. One would think: How?
But this is simple: If you have a value, you can do things with it. As long as it is used in parts of the game that don't need to be deterministic (like just printing it into the frame-buffer), no problem.

But it breaks determinism, if you then built in an alarm. Let's say you make a game mode for multiplayer that creates new resource patches every full hour. Cool idea. Now on one desktop it is 13:55 but on the other it is at the same tick 14:00.
Very unlikely, it's just an example, but I think it explains the problem a bit.

[Zeblote]

I mean, my clock mod, all it does is add a button, which displays the game world's time of day. Clicking it does nothing, it contributes absolutely nothing to the state of the game, if only one person had it, and the others didn't, it wouldn't actually break anything, yet, because it runs a script, everyone needs that script.

You as modder cannot know that. The game (or your mod) can be extended so that it suddenly does.

Mods are sandboxed from each other. Honestly, you guys need to stop thinking modders are stupid, we know just fine whether something is relevant for other clients or not. If it doesn't work in a future version of the game, the mod can be updated aswell.

[ssilk]

Sorry, but if you look into the mod-board I see countless proofs. Not that they are stupid (that said you ), but that they are not aware of this determinism stuff. Or much better: They are aware of this and use it as a game-element.

The devs must see that as a problem, cause that will decrease the stability of the game, even if they are not responsible for that. The logical decision is for them just: We cannot be sure, we need to forbid it, cause it lowers the value of our game, if we allow it, cause then it will be used, no matter how clever the modders are.

[Zeblote]

Sorry, but if you look into the mod-board I see countless proofs. Not that they are stupid (that said you ), but that they are not aware of this determinism stuff. Or much better: They are aware of this and use it as a game-element.

The devs must see that as a problem, cause that will decrease the stability of the game, even if they are not responsible for that. The logical decision is for them just: We cannot be sure, we need to forbid it, cause it lowers the value of our game, if we allow it, cause then it will be used, no matter how clever the modders are.

Well, of course people aren't aware of it, if you don't explain it to them in the modding tutorials. Some also might just not care, that's fine, simple mods pretty much "just work"

There's no reason to hide the more advanced mechanics from people that want to/can use them, just because someone else might do something wrong.

[bobucles]

There are a few things that can be clearly different between clients without any obvious issues. Most of it is UI related such as graphical assets, sounds and keybindings. Macro-like effects can also work as long as it only uses functions a player normally has in the base game. I don't think the simulator cares about how many smoke trails your smelter has or how flashy your rocket explosions get.

The moment a simulation behavior changes such as with biter AI or how inserters/assemblers work, that's pretty much an automatic desync.

[ssilk]

Yes, and yes.

It doesn't matter. If I would be a developer and need to decide
A) We write a documentation of how to take care with the determinism and try to tell all modders of how to use that,
B) I force the modders to take care of determinism so deep as possible by checking that in the game,

I would always choose B, cause A will never work for all and so I put just a lot of time into a nice idea, that won't work. But B does.
And as you see, that is also the way they (the devs) go, so I could not be completely wrong with this statement.

[Zeblote]

A) We write a documentation of how to take care with the determinism and try to tell all modders of how to use that,
B) I force the modders to take care of determinism so deep as possible by checking that in the game,

C) Implement B) for general mods and have a lower level API for modders that want to go further. They will figure out how to use it properly.