Client-side scripts to send custom events to sim; readfile()

[Zeblote]

There's a makefile method to write a file and store data, but no matching readfile method to access the data later.

This would be really useful to persistently store mod settings and similar things, shared between multiple save games.

[ratchetfreak]

not gonna happen because it leads to desyncs when the files on each client aren't synced

[Zeblote]

not gonna happen because it leads to desyncs when the files on each client aren't synced

How would this lead to a desync? Other clients don't even need to know whether the file exists or that it has been read at all.


For example you could load settings in a client sided gui. Other clients don't even need to know whether you have this gui open, it's irrelevant for them.
When you apply the changes a notice is sent to other clients how settings have changed so they can reproduce the changes. For many things there's no need to duplicate all actions on different clients, only the end result.

[ratchetfreak]

not gonna happen because it leads to desyncs when the files on each client aren't synced

How would this lead to a desync? Other clients don't even need to know whether the file exists or that it has been read at all.


For example you could load settings in a client sided gui. Other clients don't even need to know whether you have this gui open, it's irrelevant for them.
When you apply the changes a notice is sent to other clients how settings have changed so they can reproduce the changes. For many things there's no need to duplicate all actions on different clients, only the end result.

In the current state of affairs all clients will run the exact same code and there is no way to only run some code on a single client.

It would be nice to have a on_gui_create event to fill a gui that while it is being handled disallows game state changes (only reads from the game state is allowed).

[Zeblote]

not gonna happen because it leads to desyncs when the files on each client aren't synced

How would this lead to a desync? Other clients don't even need to know whether the file exists or that it has been read at all.


For example you could load settings in a client sided gui. Other clients don't even need to know whether you have this gui open, it's irrelevant for them.
When you apply the changes a notice is sent to other clients how settings have changed so they can reproduce the changes. For many things there's no need to duplicate all actions on different clients, only the end result.

In the current state of affairs all clients will run the exact same code and there is no way to only run some code on a single client.

It would be nice to have a on_gui_create event to fill a gui that while it is being handled disallows game state changes (only reads from the game state is allowed).

In that case it would be nice for mods to have lower level control over how they work in multiplayer. Duplicating all actions seems like quite the brute force approach that doesn't really make sense when it comes to user interfaces.

We should be able to send/receive custom data to other clients and sync certain things ourselves.

[Adil]

Welp, to some people multiplayer brings only despair.

[Rseding91]

In that case it would be nice for mods to have lower level control over how they work in multiplayer. Duplicating all actions seems like quite the brute force approach that doesn't really make sense when it comes to user interfaces.

We should be able to send/receive custom data to other clients and sync certain things ourselves.

As far as mods are aware they never know they're being run in a MP game. To them you're just running a single player game with a bunch of simulated players doing stuff.

Mods do no syncing of data and they don't need to - they simply respond to the events generated by the game and do what ever logic they see fit based off those events. No extra data has to be sent between peers with 1 mod vs. 100 mods using this method and there's no chance for a mod to send the wrong data or interpret the data in the wrong way.

It's highly unlikely this will ever change - that goes against the entire model that Factorio is built on: input actions are synced (player clicked GUI, player pressed key) and the game simulation is run deterministically by all peers in the game.

[Zeblote]

As far as mods are aware they never know they're being run in a MP game.

Why not? This may be important for some things.

Mods do no syncing of data and they don't need to - they simply respond to the events generated by the game and do what ever logic they see fit based off those events. No extra data has to be sent between peers with 1 mod vs. 100 mods using this method and there's no chance for a mod to send the wrong data or interpret the data in the wrong way.

It's highly unlikely this will ever change - that goes against the entire model that Factorio is built on: input actions are synced (player clicked GUI, player pressed key) and the game simulation is run deterministically by all peers in the game.

That makes a lot of sense, but in some situations it just doesn't work. The mod I'm thinking about at the moment is a blueprint library.

For this I suggest a new sub-type of mod/script: client side scripts. These don't change the game state by themselves and run on one client only - no synchronization. Other clients don't know what you're doing with them or whether they exist at all (and it doesn't matter as they don't change the game state). However they are able to send custom inputs (events) to the simulation - much like the examples you gave above. That would then allow all clients to handle the changes exactly the same.

How I imagine it works is with a client-side script that handles saving a blueprint to the local library (on my client, inside script-output folder), and loading these blueprints from in-game on any server that runs the mod.

The client script runs the gui with the blueprint library. Using it I can, for example, save the blueprint I have currently selected to a local file and list all blueprints in the library. There's nothing that needs to be synchronized so far. Loading a blueprint however, modifies the game state so the client script can't do it. It would send the structure of the blueprint as an 'input' to the simulation, so all clients can reproduce the change of my current blueprint. Perfectly deterministic for the part that matters.

I'm sure more modders could come up with more and maybe even better uses for client-sided scripts that are able to send custom events to the simulation (these are then called on all clients just like you described). It would allow for much more complex and even more awesome mods.

[Rseding91]

Right now the blueprint saving/loading mods that exist simply take the input through a GUI text field. The user pastes the blueprint text into the GUI and the mod reads the GUI.

If there was some way to guarantee that a mod wasn't going to touch the game state then it might be feasible to implement such a method for mods to use. As it is right now there's no good way to guarantee that a mod will or won't touch the game state.

[Zeblote]

Right now the blueprint saving/loading mods that exist simply take the input through a GUI text field. The user pastes the blueprint text into the GUI and the mod reads the GUI.

Yes but this isn't user friendly and annoying to use. You have to minimize factorio and open a text editor. (https://www.factorio.com/blog/post/fff-106) The one I described would be completely seamless and well integrated within the game.

If there was some way to guarantee that a mod wasn't going to touch the game state then it might be feasible to implement such a method for mods to use. As it is right now there's no good way to guarantee that a mod will or won't touch the game state.

You don't have to do the work of modders, if a mod causes a desync because it is badly programmed then that's the modders fault, not yours! More options that may be dangerous if used wrong is better than less options. All it takes is adding a client.lua that is executed on the client side and telling modders to not change the game state in it or things will go bad. Both the client and the server part could be in the same mod folder by using control.lua like usual and client.lua for the client sided scripts/guis/whatever

I imagine the method would work similar to this, in client.lua:

Code: Select all

local params = {test="blah", num=4.5} 
game.transmit_event("transmitBlueprint", params)

It can call any event by its name.

And in control.lua:

Code: Select all

game.on_event("transmitBlueprint", function(event)
	//This is called on all clients like a regular event, change blueprint in here
	//event.params.test contains "blah", event.params.num contains 4.5
end)

The params would be serialized and sent to all clients along the normal inputs. For simplicity let's say there is a size limit of 500kB or however big your packet size is so it doesn't have to be fragmented. The modder could implement this himself if it is really necessary to fragment and send large data. On the other side you'd hook the event like usual and access the sent parameters.

[Rseding91]

You don't have to do the work of modders, if a mod causes a desync because it is badly programmed then that's the modders fault, not yours!

It doesn't work that way in the real world every mod that breaks gets blamed on the game engine and not the mod.

[Zeblote]

You don't have to do the work of modders, if a mod causes a desync because it is badly programmed then that's the modders fault, not yours!

It doesn't work that way in the real world every mod that breaks gets blamed on the game engine and not the mod.

But you're not breaking mods, they just won't work in the first place. The modder has to ensure it'll work before he releases it.

At the moment I'm blaming your engine for not supporting client-side scripts

[Zeblote]

I don't know how your lua integration works in detail, but can't you just forbid changing the game state in client scripts? Let client.lua only read from the game state and access local things that won't change the game state like guis, chat or files. To change things it needs to send an event to the simulation to the other part of the mod in control.lua.

[DaveMcW]

A better model is the text box gui. You can paste a block of text into it, and it automatically copies to all clients.

We need a similar object that can read a file on a specific player's computer, and copy the contents to all clients.

[Zeblote]

A better model is the text box gui. You can paste a block of text into it, and it automatically copies to all clients.

We need a similar object that can read a file on a specific player's computer, and copy the contents to all clients.

That would solve the problem for blueprint loading. It's kinda what I meant, except they would implement the specific thing in C++ instead of a more generic version where you'd implement the sync logic in lua. I guess this can be implemented a lot easier

To go with this, we could then use functions to list all files in a folder and copy the result to all clients aswell.

[Zeblote]

This topic is kind of a mess now, so I'll try to summarize it. This is the mod interface we need at this point:


player.write_file(path, contents) - returns boolean
Works just like the existing makefile, except only on one clients computer. Returns true or false depending on whether the file could be written.


player.read_file(path) - returns string or false
Reads a file from script-output on one clients computer and replicates the contents to other clients. Returns false if the file couldn't be read.


player.delete_file(path) - returns boolean
Deletes a file from script-output on one clients computer. Returns false if the file doesn't exist or couldn't be deleted.


player.delete_path(path) - returns boolean
Deletes a folder with all subfolders and files from script-output on one clients computer. Returns false if the folder doesn't exist or couldn't be deleted.


player.find_files(mask) - returns a table of file paths, file sizes and modified dates
Searches for files matching the mask inside script-output and replicates the table to other clients.

For example, it could be used like this: player.find_paths("blueprints/*.dat") and might return the following:

Code: Select all

blueprints/big_station.dat           |  938 | 04/10/2015 14:51:43
blueprints/station.dat               |  472 | 08/10/2015 08:33:12
blueprints/somefolder/powerplant.dat | 2134 | 14/09/2015 19:16:58

player.find_paths(mask) - returns a list of folders
Searches for folders matching the mask inside script-output and replicates the table to other clients.

Works similar to the above method, but searches for folders (including empty ones!). player.find_paths("blueprints/*") might return

Code: Select all

blueprints/somefolder/
blueprints/sometolder/anotherfolder/

With all these functions together, it should be possible to implement a fancy local blueprint library and all sorts of other things like saving favorite mod settings and whatnot.

[ratchetfreak]

There is one thing missing

How do you specify which client to write/read the file to/from

Or what about headless servers can they provide files for this as well?

[Zeblote]

There is one thing missing

How do you specify which client to write/read the file to/from

They're methods on the player class. Based on other mods, that's what identifies a specific client. Gui events pass along the player who clicked buttons. Or did I miss something?

About headless servers, I've no idea. They would need a seperate set of methods.

[ratchetfreak]

They're methods on the player class. Based on other mods, that's what identifies a specific client. Gui events pass along the player who clicked buttons. Or did I miss something?

About headless servers, I've no idea. They would need a seperate set of methods.

I noticed that a bit later. was too lazy to clear the text.

Also on the more technical side; the replay file must contain the file that was pulled in. That can cause the replay file to grow quite a bit if a mod is dumb and pulls a file every tick.

[Zeblote]

Also on the more technical side; the replay file must contain the file that was pulled in. That can cause the replay file to grow quite a bit if a mod is dumb and pulls a file every tick.

I see no reason for a mod to do that. If a file is needed regularly, it can be cached in lua.