Client-side scripts to send custom events to sim; readfile()

[Rseding91]

You don't have to do the work of modders, if a mod causes a desync because it is badly programmed then that's the modders fault, not yours!

It doesn't work that way in the real world every mod that breaks gets blamed on the game engine and not the mod.

[Zeblote]

You don't have to do the work of modders, if a mod causes a desync because it is badly programmed then that's the modders fault, not yours!

It doesn't work that way in the real world every mod that breaks gets blamed on the game engine and not the mod.

But you're not breaking mods, they just won't work in the first place. The modder has to ensure it'll work before he releases it.

At the moment I'm blaming your engine for not supporting client-side scripts

[Zeblote]

I don't know how your lua integration works in detail, but can't you just forbid changing the game state in client scripts? Let client.lua only read from the game state and access local things that won't change the game state like guis, chat or files. To change things it needs to send an event to the simulation to the other part of the mod in control.lua.

[DaveMcW]

A better model is the text box gui. You can paste a block of text into it, and it automatically copies to all clients.

We need a similar object that can read a file on a specific player's computer, and copy the contents to all clients.

[Zeblote]

A better model is the text box gui. You can paste a block of text into it, and it automatically copies to all clients.

We need a similar object that can read a file on a specific player's computer, and copy the contents to all clients.

That would solve the problem for blueprint loading. It's kinda what I meant, except they would implement the specific thing in C++ instead of a more generic version where you'd implement the sync logic in lua. I guess this can be implemented a lot easier

To go with this, we could then use functions to list all files in a folder and copy the result to all clients aswell.

[Zeblote]

This topic is kind of a mess now, so I'll try to summarize it. This is the mod interface we need at this point:


player.write_file(path, contents) - returns boolean
Works just like the existing makefile, except only on one clients computer. Returns true or false depending on whether the file could be written.


player.read_file(path) - returns string or false
Reads a file from script-output on one clients computer and replicates the contents to other clients. Returns false if the file couldn't be read.


player.delete_file(path) - returns boolean
Deletes a file from script-output on one clients computer. Returns false if the file doesn't exist or couldn't be deleted.


player.delete_path(path) - returns boolean
Deletes a folder with all subfolders and files from script-output on one clients computer. Returns false if the folder doesn't exist or couldn't be deleted.


player.find_files(mask) - returns a table of file paths, file sizes and modified dates
Searches for files matching the mask inside script-output and replicates the table to other clients.

For example, it could be used like this: player.find_paths("blueprints/*.dat") and might return the following:

Code: Select all

blueprints/big_station.dat           |  938 | 04/10/2015 14:51:43
blueprints/station.dat               |  472 | 08/10/2015 08:33:12
blueprints/somefolder/powerplant.dat | 2134 | 14/09/2015 19:16:58

player.find_paths(mask) - returns a list of folders
Searches for folders matching the mask inside script-output and replicates the table to other clients.

Works similar to the above method, but searches for folders (including empty ones!). player.find_paths("blueprints/*") might return

Code: Select all

blueprints/somefolder/
blueprints/sometolder/anotherfolder/

With all these functions together, it should be possible to implement a fancy local blueprint library and all sorts of other things like saving favorite mod settings and whatnot.

[ratchetfreak]

There is one thing missing

How do you specify which client to write/read the file to/from

Or what about headless servers can they provide files for this as well?

[Zeblote]

There is one thing missing

How do you specify which client to write/read the file to/from

They're methods on the player class. Based on other mods, that's what identifies a specific client. Gui events pass along the player who clicked buttons. Or did I miss something?

About headless servers, I've no idea. They would need a seperate set of methods.

[ratchetfreak]

They're methods on the player class. Based on other mods, that's what identifies a specific client. Gui events pass along the player who clicked buttons. Or did I miss something?

About headless servers, I've no idea. They would need a seperate set of methods.

I noticed that a bit later. was too lazy to clear the text.

Also on the more technical side; the replay file must contain the file that was pulled in. That can cause the replay file to grow quite a bit if a mod is dumb and pulls a file every tick.

[Zeblote]

Also on the more technical side; the replay file must contain the file that was pulled in. That can cause the replay file to grow quite a bit if a mod is dumb and pulls a file every tick.

I see no reason for a mod to do that. If a file is needed regularly, it can be cached in lua.

[ratchetfreak]

Also on the more technical side; the replay file must contain the file that was pulled in. That can cause the replay file to grow quite a bit if a mod is dumb and pulls a file every tick.

I see no reason for a mod to do that. If a file is needed regularly, it can be cached in lua.

As I said "if a mod is dumb" and there will be dumb mods.

Doing that in multiplayer for each player will be even more hazardous because the bandwidth cost will explode.

So a rate limiter on it would be nice.

[Zeblote]

As I said "if a mod is dumb" and there will be dumb mods.

Doing that in multiplayer for each player will be even more hazardous because the bandwidth cost will explode.

So a rate limiter on it would be nice.

Well, just don't install dumb mods. There shouldn't be a limit just because someone might be unable to use it correctly.

[DaveMcW]

I assume text box synchronization happens between ticks, so doing something like:

Code: Select all

str = player.read_file(path)

would wait for the next tick, which would wait for the script to finish, which would wait for the next tick. Infinite loop!

It would be need to be at least slightly asynchronous, like:

Code: Select all

player.readfile(path, eventHandler)

And since it is asynchronous anyway, it is no problem to cap the file transfer speed to prevent it from eating all the bandwidth.

[Zeblote]

Hmm. That is a good point. Loading a larger file might also take many ticks to sync.

Now the real question is, how long does it take to implement these and can we get them in 12.11 or 12.12?

[Zeblote]

Seems like you've found a way to prevent a script from changing the game state, so I don't see what's stopping client sided scripts.

Not running everything on all clients makes way for many client sided functions in the future, especially regarding guis and files.

[Rseding91]

Seems like you've found a way to prevent a script from changing the game state, so I don't see what's stopping client sided scripts.

Not running everything on all clients makes way for many client sided functions in the future, especially regarding guis and files.

By not allowing access to the game state you can't change the game state. Allowing reading files in while you had no access to the game state or the global table would also prevent modifying the game state but it would be useless as you couldn't *do* anything with the file.

GUIs are part of the game state. If the mod is able to read from the GUI then it is changes the game state.

[Zeblote]

By not allowing access to the game state you can't change the game state. Allowing reading files in while you had no access to the game state or the global table would also prevent modifying the game state but it would be useless as you couldn't *do* anything with the file.

That's why you send a custom event to the game that can then be processed deterministically. What you're doing with a gui and files and whatever does not matter to other clients until you actually want to do something with it that changes the game state. Client sided scripts are required for many advanced APIs like files, maybe internet access, etc...

GUIs are part of the game state. If the mod is able to read from the GUI then it is changes the game state.

They shouldn't be, it makes no sense. If I open the escape menu and then the Save game gui, do other client's know what I'm doing with them? Does it send the preview images to other clients when I select a save game in the list? Do other client's know whether I'm typing a chat message until I've actually sent it? If not, why do you want to force modders to use this system?

[Zeblote]

Has any more thought been put into this?

Looked around a bit at how the default guis work, and your internal system is a lot more advanced than the few parts you expose to modding. I can only assume this is because it is (obviously) client-sided and doesn't have to deal with synchronizing everything.

The high level gui system is fine for many simple things, but it quickly reaches its limit. I'd like to make more advanced mods with seperate game/sim sided and client sided (ui, keybinds, files, network, etc) parts but can't because there's no such system available.

[Rseding91]

They shouldn't be, it makes no sense. If I open the escape menu and then the Save game gui, do other client's know what I'm doing with them? Does it send the preview images to other clients when I select a save game in the list? Do other client's know whether I'm typing a chat message until I've actually sent it? If not, why do you want to force modders to use this system?

Mods have read/write access to the GUI elements they and other mods create. Because they have read access the GUIs mods create have to be persisted between saves.

Without persisting the GUI elements mods create you easily end up with a desync: Create GUI element through mod, MP peer connects, GUI element wasn't persisted between save/load and now peer 1 has the GUI element and peer 2 does not. The mod is identical on both ends and does different stuff based on the GUI existing on one peer and not on the other.

[Zeblote]

I think you're misunderstanding what I'm asking for. It's not a change to the existing system, but rather a new system. Lower level than the existing one.

Currently, we have one script: control.lua. This runs as part of the simulation and is fully deterministic. However, certain actions simply are not deterministic or don't have to be, because they don't affect anyone else.


I'm asking for a second script: client.lua. This runs on a client and is local to that client only.

It's executed/called at these times:
- Started single player game, before (!) initializing the control.lua scripts
- Loaded existing save
- Started multiplayer game, before (!) initializing the control.lua scripts
- Loaded existing save as multiplayer game
- Joined multiplayer game

This script does not need to be synchronized with other clients, or store any persistent data in the save files.
It would introduce a new namespace "client", similar to the existing "script" and "game".

Because it's local to each client, it can't change the game state on it's own. (It does have read access, though. As in, you would be able to run find_entities but not create_entity.)
That means there needs to be a way to communicate between these two scripts. It could be as simple as client.transmit_event(name, args). The sent events will be processed deterministically by the control.lua script, which applies the game state change we need.

Now that we have the client script framework, what's the point? For example, you can now implement full file IO under the client namespace. You don't have to worry about synchronization, because the file is handled on the client side and sent to the server using transmit_event when it's actually needed. You can implement all of the internal gui elements under the client namespace, because you don't have to worry about synchronization. Once a client does something in the gui that should trigger a game state change, it is simply sent to control.lua with transmit_event and replicated on all clients. I'm sure you can think of more.

By offering lower level control over handling multiplayer you let modders make much more advanced features that otherwise have to be part of the engine.
For example, the blueprint gallery. I'm imagining a fancy gui that has all your blueprints sorted in folders and even with preview images. From there, you can add network functions to the client namespace so we can make an online blueprint sharing site. There's really no limit!