Access to prototype dictionaries via script table

[aubergine18]

My mod defines a remote interface allowing other mods to register extensions to its various features (eg. add new bridges, pipes, character classes, etc).

In the API I want to validate any items/entities used as they need specific values to be set in their prototypes, which might cause errors later on if not set, and also determine some settings based on the values in the prototype.

The prototype dictionaries, being on the `game` table, aren't available via the `on_init` event. As such, any mod that wants to extend features of my mod is going to need to do so in the first invocation of the `on_tick` event which is somewhat grim, or I'm going to have to defer the data validation and completion which is also somewhat grim.

Would it be possible to get dictionaries such as `entity_prototypes`, `item_prototypes`, etc., available on the `script` table (or some other table - maybe a global `prototypes` table?) so that they are always available at any stage of control.lua lifecycle? If not, would it be possible to get an `on_game` event that fires when the `game` table becomes available to a control.lua instance?

[Klonan]

My mod defines a remote interface allowing other mods to register extensions to its various features (eg. add new bridges, pipes, character classes, etc).

In the API I want to validate any items/entities used as they need specific values to be set in their prototypes, which might cause errors later on if not set, and also determine some settings based on the values in the prototype.

The prototype dictionaries, being on the `game` table, aren't available via the `on_init` event. As such, any mod that wants to extend features of my mod is going to need to do so in the first invocation of the `on_tick` event which is somewhat grim, or I'm going to have to defer the data validation and completion which is also somewhat grim.

Would it be possible to get dictionaries such as `entity_prototypes`, `item_prototypes`, etc., available on the `script` table (or some other table - maybe a global `prototypes` table?) so that they are always available at any stage of control.lua lifecycle? If not, would it be possible to get an `on_game` event that fires when the `game` table becomes available to a control.lua instance?

game.item_prototypes and game.entity_prototypes and what not are available on script.on_init, and script.on_configuration_changed for these exact situations

They are not available on script.on_load.

[aubergine18]

Can we get an `on_game` event so we can do stuff as soon as they become available (without needing to do stuff in first on_tick)?

EDIT: Misread your comment. I'll wire in to those two events.

[Therenas]

Digging this up again, because I think it would be useful to have prototype read access during on_load, if it doesn't cause desync or similar issues.

In my use case, I have a rather large global table. The vast majority of that table are condensed and organised versions of game prototypes, like recipes, items, machines, etc. This table provides me with great performance and useability advantages elsewhere, with the downside that saves and loads take forever. As an example, if you play with modpacks like pyanodons, there will be like 8k+ of those prototypes, which will make the global table 6+MB. In any case, the save times are 3x slower than vanilla, which is annoying to have to deal with every quicksave.

Now, what having prototype access on_load will allow me to do is to save only bare-bones versions of those tables to global, and rebuild them to local variables on load. The only kind-of workable alternative that I can think of is doing it on the first tick, but I'm not even 100% sure if this would cover all cases. It 100% is a bad way to do this, though. So, if possible, it would be really neat to have this access granted on_load.

(Feel free if you want more specifics on my use case, trying to keep it short)

[DaveMcW]

The vast majority of that table are condensed and organised versions of game prototypes, like recipes, items, machines, etc.

Sounds like a perfect fit for on_configuration_changed and on_init.

[Therenas]

Sounds like a perfect fit for on_configuration_changed and on_init.

Except that I have to save it all in global then, which is slow when saving/loading. I currently do it on_init and on_configuration_changed, and parts of it would still be there, but this is to reduce the total size saved to global. Basically, I don't want to have to save protoype attributes to global, it would be better to just save their name or whatever to global, and load the needed attributes during on_load.

[Boodals]

I tried to implement a "prototypes" object for this purpose when I first got source access, but it was denied for this reason:
"I don't consider this worth the sheer amount of code/mods it will break. Additionally mods still wouldn't be allowed to keep any prototype references alive outside of the global table because doing that will still cause desyncs. If a prototype was available during on_load it would give a false sense of security making people think it was ok to keep it around in local variables when it's not."

Initially I wanted to say that this argument is stupid (in fancier words), but after thinking about it for a while, there is a definite truth to it. But I think the problem is bigger than this quote lets on. I'm not totally sure, but I think that LuaObjects are part of the gamestate, so they can't be created from within on_load at all, and it just so happens (read: it was intentionally made this way) that LuaGameScript is the only way to get access to new LuaObjects, so it was made inaccessible.

More recently, I've been trying to come up with a solution for this, but I don't really understand everything that goes on with on_load. One thing I did theorize that could be useful if you are storing data extracted from the prototypes, is that mods could store each prototype they're interested in inside global, then in on_load read through them and build the cached data. I think you could even store copies of the prototypes themselves in local variables (although the quote implies you couldn't, I think it would be fine as long as the variable in global existed, or maybe indefinitely), but I wouldn't try it without extensive testing because I don't totally understand where/how LuaObjects need to be synced.
You'd have to check whether saving and parsing the prototypes would actually take longer to load than just saving the cached data, and the save file size differences, etc. But a library mod could be made which handles all of this for other mods, so you don't end up with 5 mods each saving the same prototypes.

From that, my next line of thinking was "could I just make this idea built into the API somehow?", optimized/overhauled of course, there's no reason to actually save the prototype references from the game's perspective. Maybe if I made lua prototypes not be LuaObjects?
But I haven't made any progress beyond that. Regardless of the solution, it would likely break all mods that use the current methods, so for that reason alone it might still get rejected.

Its 3am so I probably made some oversights, but hopefully this clears things up or helps or something.

[Therenas]

Thanks for the reply Boodals. It wouldn't be great, but having a prototypes table specifically on_load, and keeping the rest of the API the same (ie not removing the prototype access from LuaGameScript) would not break any mods, or would it?
Also, if we are thinking about this on an abstract level, prototypes do not need to be synced to the gamestate, because they are guaranteed to be the same for every player. This is what you said, and it would be cool to have them not be LuaObjects, but that is probably a ton of work. A workaround again could be to have a tailor-made solution, just for on_load, and keep the rest of the API the same. But this is not a very clean solution, I'll admit.

I just ran a little experiment, and you can actually get the prototypes inside on_load if you save the references to the LuaObjects in global previously. To illustrate, I run this snippet in on_init():

Code: Select all

global.items = {}
for name, item in pairs(game.item_prototypes) do
    global.items[name] = item
end

You can't save game.item_prototypes to global directly, because CustomTable can't be serialized.
Then, in on_load, you can use those prototypes like you would normally, like so:

Code: Select all

log(global.items["iron-plate"].localised_name)

Now if you update your references on_configuration_changed, this should continue to work I think.
So you can take this in two ways: Either, you see this as a workaround for our problem, and are happy with it, or you see it as 'proof' that prototypes should be available during on_load, because they kind of are anyways. Not sure on which side I fall yet. (There also might be issues with this approach, I only tested it briefly)

Edit: I didn't test whether this saves/loads faster than manually created tables, but as I understand it, it doesn't actually save a table, only a reference to a LuaObject, so that should be much more compact. Is this correct?

[Boodals]

It wouldn't be great, but having a prototypes table specifically on_load, and keeping the rest of the API the same (ie not removing the prototype access from LuaGameScript) would not break any mods, or would it?

That should work, but yeah, it isn't very clean.

Edit: I didn't test whether this saves/loads faster than manually created tables, but as I understand it, it doesn't actually save a table, only a reference to a LuaObject, so that should be much more compact. Is this correct?

That is correct, all it saves is a table containing the userdata variable "__self", which is read by the game when loading to reestablish the connection to the real object. You can see this if you run game.print(serpent.line(game.player)). But if all you want to cache is a table such as { ["iron-plate"] = 2, etc } then it would probably be better to just save the table, not the prototypes.