data stage "events" for new and modified recipe prototypes

[slippycheeze]

G'day. I was recently politely horrified to see someone catching recipe creation in `data-fixes-final` by setting the metatable of the data.raw recipes table. They then used the __newindex event to catch recipe creation, and take their own actions. That'll work, demonstrably, but it really seemed off.

On the other hand, I was noodling with an idea recently and found myself in the same position: I wanted to catch any change to the ingredients of a recipe, as well as creation, including in the `data-fixes-final` stage, without having to coordinate with other mods.

Which is a subbing an empty object in place, with events like __index, __newindex, and __pairs, so you can intercept reads and writes in your proxy, and forward to/from the real table. Untested, but ... unless the C++ side uses raw get operations, it should work, and without it should at least theoretically be possible to make the horror work.

At the end of days these, and I suspect many uses of `data-fixes-final`, are really about ensuring I can run some code in response to these events. If I could do that it'd be a trivial declaration at the start of `data` and I'd never have to care again, because I know it'll do the right thing.

So: I suggest y'all consider adding events, ike the ones during runtime, in the data phase. If I can do this, I don't need dream of horrible hacks, or petition for a `data-fixes-final-2-ex` phase.

Code: Select all

-- I typed this fresh in the post, so syntax errors everywhere probably
data.on_event({defines.on_recipe_prototype_created, defines.on_recipe_prototype_modified}, function(event)
  if event.prototype.name == "landfill" then
    -- obviously contrived example is obviously contrived
    event.prototype.hidden = true
  end
end)

Obvious thing: recursion might happen. I'd suggest breaking the loop after a limit of cycles, and reporting incompatible a mod incompatibility for everything involved. Like, someone responds to my hiding landfill by showing it.

Alternatively: just stop events after the first 20 or something, and let the last one win. This is probably not deterministic without lots of care, though, so refusing to work at all seems smart.

I'd probably *not* treat any reentrancy as a problem, just excessive, because not all reentrance will cycle forever. A huge kindness would be to detect actual change rather than just assignment, and only trigger the next round if something did change.

If I fight over a field with someone, boom, but if I only change a field once each time they change prototype , and they change an unrelated field when I change the prototype, if we assigned the same value it'd terminate the loop quickly and correctly.

Alternative: ban metatable stuff

The following will make it impossible to mutate the metatable of a table every again, completely forbidding anyone from doing this. Which is the other reasonable response to the situation, I think. Once you make that setmetatable call (from C or lua) it becomes impossible to get the metatable, because you get the value of __metatable instead from getmetatable.

The spec or lua wiki have more details about how this works, but the principal is trivial enough: once __metatable is present in the actual metatable, you only get that value back. Without the real metatable table, no adding metatable stuff. Cruel persons might consider tormenting people with __metatable = {}, a fresh table, so it *looks* like you can set methods, but they just don't work.

Code: Select all

local so_controversial_yet_so_brave= { __metatable = "this is too awful to live" }

data.raw = setmetatable({}, so_controversial_yet_so_brave)
data.raw.armor = setmetatable({}, so_controversial_yet_so_brave)
-- etc, etc

[eradicator]

Adding events would probably murder performance (startup-time), not sure. Adding more stages would probably be trivial code-wise, but wouldn't solve the problem as every inexperienced mod author would still use the_really_final_ultimate_data_stage.lua.

Cruel persons might consider tormenting people with __metatable = {}, a fresh table, so it *looks* like you can set methods, but they just don't work. :)

(Disregarding the general insanity of basing a mod on data stage metatables in the first place...) If someone is actually intelligent enough to at least try and preserve the previous metatable then that approach will simply force them to do the same dumb thing everyone else does: overwrite the whole thing and give no fucks about other peoples metatables. And if i ever notice that too many mods start sticking their dirty metas on everything i'll proabably start including a function that scrubs data.raw and all of it's content clean at the start of every mod i own :p.

[slippycheeze]

Adding events would probably murder performance (startup-time), not sure.

Why? Serious question, because maybe you know something I don't, but ... lua function calls - even from C - are pretty inexpensive, and the total amount of work being done is likely to be low. Writing code in an "event" handler is pretty clearly not the normal way to do it - can't define anything in there, just mutate stuff - and the actual changes are minor.

"Did it actually change" is theoretically maybe a fraction costly, but in practice full userdata object - familiar from the runtime era protoype objects, etc - make it relatively inexpensive overall. Even the pure lua version of dirty tracking via a proxy table (hello, self) turns out pretty inexpensive.

Adding more stages would probably be trivial code-wise, but wouldn't solve the problem as every inexperienced mod author would still use the_really_final_ultimate_data_stage.lua.

Well, I wouldn't specify the amount of experience here. All it really takes is either (a) need to catch all recipes to create derivatives of them (eg: reverse factory) or (b) want to modify all things in ${category} (eg: electric power for vehicles mods) in a way that doesn't force other mods to integrate by hand.

(Also usable: actually *support* other mods changing things, so you can reflect that change with your addition or enhancement, such as updating your stuff when someone, eg, changes chest slot count.)

(Disregarding the general insanity of basing a mod on data stage metatables in the first place...) If someone is actually intelligent enough to at least try and preserve the previous metatable then that approach will simply force them to do the same dumb thing everyone else does: overwrite the whole thing and give no fucks about other peoples metatables. And if i ever notice that too many mods start sticking their dirty metas on everything i'll proabably start including a function that scrubs data.raw and all of it's content clean at the start of every mod i own :p.

I think disregarding the fact that this sort of metatable hacking is ... shall we say, undesirable to have more than one implementation of, here? Anyway, I think disregarding that would miss the point of the post.

PS: you can't ignore metatables, because I can make it literally impossible for you to change them in any way, and that is both trivial to write, and very well documented. Roughly the second "metatables do stuff" entry in any lua guide, in fact. The third one is how to control all reads and writes, not just new ones. By default even the C API won't ignore them.

Unless data is full userdata or something, the first mod loaded wins the war, and has absolute control over what anyone else can do. So clearly AAI Programmable Metatables is the winner.

[eradicator]

Adding events would probably murder performance (startup-time), not sure.

Why? Serious question

Because currently data.raw is just a plain table, with plain assignments. Intercepting each and every assignment to any value of the table is obviously more expensive. Ofc i have no clue *how* expensive, but i'd speculate at least two times as expensive ("assign value" vs "function call + assign value"). And seeing that the devs seem to like saving a few seconds on startup whereever they can...

I think disregarding the fact that this sort of metatable hacking is ... shall we say, undesirable to have more than one implementation of, here? Anyway, I think disregarding that would miss the point of the post.

That "disregard" was meant for that one part of your argument where you want to punish only people that actually try to do the right thing while letting the ignorant people continue to do whatever they want.

PS: you can't ignore metatables, because I can make it literally impossible for you to change them in any way, and that is both trivial to write, and very well documented.

The word "impossible" is not in my dictionary: debug.setmetatable(infested_table,nil)
And if you fuck up debug too then you're writing malicious code ;).

[Klonan]

I don't really see why it would be needed

Also metatables are cool and powerful, I don't want to stop people using them.

[slippycheeze]

The word "impossible" is not in my dictionary: debug.setmetatable(infested_table,nil)
And if you fuck up debug too then you're writing malicious code .

haha, lemme just figure that one out. but also, debug is available in the data phase. I see no way I could possibly abuse this.

I don't really see why it would be needed
Also metatables are cool and powerful, I don't want to stop people using them.

OK. Legit. Thanks for considering it.