Importing any basic data from protoype stage to runtime

Osmo
Long Handed Inserter
Long Handed Inserter
Posts: 94
Joined: Wed Oct 23, 2024 12:08 pm
Contact:

Importing any basic data from protoype stage to runtime

Post by Osmo »

It is common for mods' scripts to need some data that is only availible at protoype stage. Some of that could and should be exposed in the relevant LuaProtoype. But that is not always the case.
There exist hacky solutions, like creating a hidden item with data encoded into localised_description, but that is very unintuitive and needs an unnecesserily complicated way to compact the data into a localised_string with its lenght limits.
What would be useful instead, is prototype type designed specifically to hold any data. There already exists a concept in the API called Any Basic, as clearly functions and metatbles shouldn't be allowed to be imported from ptotoype stage to runtime stage.
It could be a protoype type that allows any data, so it can be indexed by name in data.raw, or be limited to a single instance and have custom properties like GuiStyle, but instead be of type string -> AnyBasic. And then the protoype(s) could be accesed at runtime through the protoypes global as usual, and it would return the same AnyBasic data as was constructed in prototype stage, like what utility_constants does already.
Loup&Snoop
Inserter
Inserter
Posts: 37
Joined: Wed Dec 04, 2024 4:33 am
Contact:

Re: Importing any basic data from protoype stage to runtime

Post by Loup&Snoop »

+1

If devs want to send new types of data from data stage to control stage, they can add a new prototype type. eg Any prototype with only 1-2 fields is simialr to passing yourself a hashset/dictionary from data to control stage. Modders do not have this option.

As Osmo said, we can already smuggle data. A lack of support doesn’t stop modders from doing it. The current system just forces modders to spend more effort to implement hacky solutions. This is counterproductive if the whole point of not supporting data smuggling is to prevent issues.

———

EDIT: In Discord, the main arguments against this seems to be “People have abused this in the past to store massive data.” and “Making it difficult filters out bad users.”

Part of the issue is that we don’t readily have/use tools to check data usage (specifically the total data cost of all the prototypes a specific mod defined in data stage). If you can see “X mod is using 6 GB of RAM”, then that would follow the same chain of events as “X crashes the game.” or “X consumes too much UPS!”. Which normally leads to the mod getting 1) fixed, 2) suppressed in trending, 3) people accept the poor optimization and go on anyway. As of right now, if my mod smuggled 100 copies of data.raw, good luck figuring out it was me :lol: .
Last edited by Loup&Snoop on Sat Jun 21, 2025 4:28 pm, edited 1 time in total.
raspberrypuppies
Inserter
Inserter
Posts: 48
Joined: Tue Mar 26, 2019 4:43 am
Contact:

Re: Importing any basic data from protoype stage to runtime

Post by raspberrypuppies »

+1, also commented my usecase on the duplicated feature request
Loup&Snoop
Inserter
Inserter
Posts: 37
Joined: Wed Dec 04, 2024 4:33 am
Contact:

Re: Importing any basic data from protoype stage to runtime

Post by Loup&Snoop »

If the interest in actually doing this is real, and not trolling, here is how I would request the implementation look like with an example. Imagine the mod "Rubia" wants to define a "wind restriction" data object, to define how control-stage scripts should operate on a given entity.

Rubia's data.lua:

Code: Select all

data:extend({
    {
        --Mandatory
        type = "mod-data",
        subtype = "rubia-wind-restriction",
        name = "fast-transport-belt",

        --The actual custom data
        wind_behavior = "force-not",
        orientation = {defines.direction.west},
    },
    ... more of them
})
Rubia's control stage working with the data:

Code: Select all

script.on_event(defines.events.on_built, function(event)
    if not event.entity.valid then return end
    local wind_restriction = prototypes.mod_data["rubia-wind-restriction"][event.entity.name]
    if wind_types[wind_restriction.wind_behavior] then
        ... go do stuff based on that data
    end
end)
Now someone makes a cheat mod: "No Rubia Wind Restrictions". In their data-final-fixes.lua :

Code: Select all

---Go delete all of the mod_data prototypes of this type
if data.raw.mod_data["rubia-wind-restriction"] then
    local to_delete = util.table.deepcopy(data.raw.mod_data["rubia-wind-restriction"])
    for name, _ in pairs(to_delete) do
        data.raw.mod_data["rubia-wind-restriction"][name] = nil
    end
end
The main points:
  • Things that conceptually should be defined in data stage are defined in data stage--not control stage.
  • Custom mod data is organized by a subtype for interoperability. It will save on a lot of prefix searching and convention nonsense.
  • Other mods get to add/delete/modify in data stage without other modders explicitly adding remote interfaces.
———-

One bit that would be helpful (but idk if this would be too intrusive on the game’s staging) would be the ability to run file(s) to validate the values of the custom mod objects at the end of data stage, right after data.raw has been locked into its immutable form.

Example: data-validation.lua

Code: Select all

for name, entry in pairs(prototypes.mod_data["rubia-wind-restriction"]) do
    assert(entry.orientation, "No orientation found for " .. name)
    assert(table_size(entry.orientation) > 0, "Orientation table was empty for " .. name)
    assert(type(entry.wind_behavior) == "string", "Type of rubia-wind-restriction needs a string for wind_behavior: " .. name)
end
No data saving/altering. Just make sure everything is good at data stage. You could do this in control stage, but this is a data stage task that does not belong in control stage.
protocol_1903
Filter Inserter
Filter Inserter
Posts: 270
Joined: Fri Sep 09, 2022 4:33 pm
Contact:

Re: Importing any basic data from protoype stage to runtime

Post by protocol_1903 »

Loup&Snoop wrote: Sat Jun 21, 2025 10:57 pm ...
The main points:
  • Things that conceptually should be defined in data stage are defined in data stage--not control stage.
  • Custom mod data is organized by a subtype for interoperability. It will save on a lot of prefix searching and convention nonsense.
  • Other mods get to add/delete/modify in data stage without other modders explicitly adding remote interfaces.
...
100% need subtypes.(No offense) I'm tired of having to run a handful of checks every single time I want to know if a specific entity/item/thing is part of my mod (or something extended from it). This is really apparent in the numerous pipe mods, where some entities, like the ducts added by Fluid Must Flow (no offense meant, just a prominent example) do not conform to the usual naming standards and as such must be specifically looked for/validated on every check. I'd much prefer some way to correlate that kind of information when it's generated instead of trying to guess it later.
Loup&Snoop wrote: Sat Jun 21, 2025 10:57 pm ...
One bit that would be helpful (but idk if this would be too intrusive on the game’s staging) would be the ability to run file(s) to validate the values of the custom mod objects at the end of data stage, right after data.raw has been locked into its immutable form.

Example: data-validation.lua

Code: Select all

for name, entry in pairs(prototypes.mod_data["rubia-wind-restriction"]) do
    assert(entry.orientation, "No orientation found for " .. name)
    assert(table_size(entry.orientation) > 0, "Orientation table was empty for " .. name)
    assert(type(entry.wind_behavior) == "string", "Type of rubia-wind-restriction needs a string for wind_behavior: " .. name)
end
No data saving/altering. Just make sure everything is good at data stage. You could do this in control stage, but this is a data stage task that does not belong in control stage.
Actually I disagree here, I feel like this could be abused a lot to enforce specific playstyles or game mechanics without allowing players to remove them from their game. In either case, a failed assert would correlate with a bug report, so it would probably be more useful as a development tool than a release feature.
If you need to reach me, message me on discord.

I make qol mods. Check them out, maybe.
https://mods.factorio.com/user/protocol_1903
If you have a mod idea, I can look into it.
Loup&Snoop
Inserter
Inserter
Posts: 37
Joined: Wed Dec 04, 2024 4:33 am
Contact:

Re: Importing any basic data from protoype stage to runtime

Post by Loup&Snoop »

protocol_1903 wrote: Sun Jun 22, 2025 4:52 am
Loup&Snoop wrote: Sat Jun 21, 2025 10:57 pm data-validation.lua
Actually I disagree here, I feel like this could be abused a lot to enforce specific playstyles or
Consider: The alternative is running the exact same code in control stage. I can’t stress this enough:
Data stage tasks do not belong in control stage

This data stage task belongs with normal prototypes validation. eg Technology prototypes can’t make prerequisite loops. Items need names to be strings…

Before you say: “Just don’t do these checks then.” Not doing those checks means that every time you access custom mod data in control stage, you need to perform nil checks etc on the prototype data. Otherwise, you risk crashes from someone else setting a field nil or the wrong type etc.
curiosity
Filter Inserter
Filter Inserter
Posts: 667
Joined: Wed Sep 11, 2019 4:13 pm
Contact:

Re: Importing any basic data from protoype stage to runtime

Post by curiosity »

I agree. You can already enforce specific playstyles by checking in runtime stage (as, for example, Space Exploration does), so it won't make things worse. But validation makes more sense during game startup, along with the similar native checks.
protocol_1903
Filter Inserter
Filter Inserter
Posts: 270
Joined: Fri Sep 09, 2022 4:33 pm
Contact:

Re: Importing any basic data from protoype stage to runtime

Post by protocol_1903 »

Loup&Snoop wrote: Sun Jun 22, 2025 11:53 am Consider: The alternative is running the exact same code in control stage. I can’t stress this enough:
Data stage tasks do not belong in control stage

This data stage task belongs with normal prototypes validation. eg Technology prototypes can’t make prerequisite loops. Items need names to be strings…

Before you say: “Just don’t do these checks then.” Not doing those checks means that every time you access custom mod data in control stage, you need to perform nil checks etc on the prototype data. Otherwise, you risk crashes from someone else setting a field nil or the wrong type etc.
That's fair. Under that assumption then, I support both requests
If you need to reach me, message me on discord.

I make qol mods. Check them out, maybe.
https://mods.factorio.com/user/protocol_1903
If you have a mod idea, I can look into it.
Bilka
Factorio Staff
Factorio Staff
Posts: 3415
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: Importing any basic data from protoype stage to runtime

Post by Bilka »

I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.
Post Reply

Return to “Implemented mod requests”