Custom prototypes for inter-mod expansion

Place to ask discuss and request the modding support of Factorio. Don't request mods here.
Post Reply
Boodals
Fast Inserter
Fast Inserter
Posts: 129
Joined: Sun Feb 11, 2018 7:10 pm
Contact:

Custom prototypes for inter-mod expansion

Post by Boodals »

I would like to be able to define custom prototype types which can then be used by other mods to expand my mod.

Eg:
I make a mod that adds a way to go to other planets. My mod adds a few basic planets, but players want more.
Another mod is created which adds several new planets, and modifies the ones in the base mod.
Somewhere in my mod, I parse through all the planet prototypes, and populate my own data structures with them.

Usage:

Code: Select all

data:extend(
{
  {
    type = "custom",
    custom_type = "planet",
    name = "Mars",

    mass = 1000,
    radius = 5,
    terrain_generation = { ... },

    --Alternatively, the custom fields could be within a table stored in custom_data, if that is easier for implementation.
    custom_data =
    {
      mass = 1000,
      etc
    },
  }
})

Code: Select all

for planet_prototype in game.get_custom_prototypes("planet") do
  make_planet(planet_prototype)
end
Perhaps a more formal system could be put into place, where a mod can register a custom prototype type mapped onto a function (or script) that gets called with each prototype. This would allow two things:
1. Prevent other mods from reacting for prototypes created for a specific mod (probably throwing an error if two mods try to register to the same custom prototype type). This could cause an issue where one of the two mods could have to rename their prototype type, which would then also need to be updated in any dependency mods. Perhaps a better solution would be to specify the mod name that each prototype is to be used for?
2. Allow access to the custom prototypes in the data phase, after data-final-fixes. This way, the base mod could create additional (regular) prototypes that are required for each of the custom prototypes. Eg. in my planet mod, I might want to create a new sprite for each planet that has the correct color tinting. This would probably require notably more work to implement though, as it would add an additional stage to loading mods.

The reason I went for a function instead of a read-only dictionary is to further prevent other mods from accessing data that my mod defines. A function that takes the prototype type as a string means that mods can't parse over every single prototype type, which would most likely be done following some bad practice.


I proposed the idea to Rseding in the discord. His main concern was that people could include data.raw (or any prototype) into one of the custom prototypes, then in the control phase access it and assume it is accurate, when in reality the values could have been modified by another mod.
I don't think this will be a serious issue, as all the prototypes are available in the control phase anyway via game.X_prototypes, and if anyone were to do it, they would eventually run into an error that would be fairly easy to track down.
If deemed that it is problem, a slightly severe solution could be to disallow populating values with tables. This initially sounds harsh, but modders could use other custom prototypes to store nested data. In the above example, terrain_generation could instead be the name of a custom "planet_generation" prototype that defines the behaviour desired. Because of this, it wouldn't completely solve the issue either, but it would complicate the steps enough that modders would question what they are doing.

The potential uses for inter-mod interaction are numerous. Mods that previously had to rely on checking if dependent mods existed to add features from that mod into their own can now just rely on the other mod adding the required custom prototypes.

Code: Select all

if mod_exists("dependency_mod") then
  add_feature(feature_from_dependency_mod)
end
if mod_exists("dependency_mod_2") then
  add_feature(feature_from_dependency_mod_2)
end
becomes

Code: Select all

--In base mod:
for feature in game.get_custom_prototypes("feature") do
  add_feature(feature)
end

--In dependency mods:
data:extend({{ type = "custom", custom_type = "feature", name = "feature from mod", feature = { ... } }})

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 5206
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: Custom prototypes for inter-mod expansion

Post by eradicator »

Boodals wrote:I proposed the idea to Rseding in the discord[/url]. His main concern was that people could include data.raw (or any prototype) into one of the custom prototypes, then in the control phase access it and assume it is accurate, when in reality the values could have been modified by another mod.
You can tell him that for as long as factorio modding existed, ppl have tried and "succeded" in pushing arbitrary data.raw data into the control phase, by abusing order strings, prototype names, serialization, etcpp. You can't really prevent it. Only make it difficult. And there's so many other sources of bugs, that this isn't really much of an issue imho. And besides some people might actually do it correctly. I mean...the ZAdventure guy has supposedly succeeded in building whole function libraries for control in data stage. Supposedly to shift the burden of generating all the stuff out of map-loading. (He said that.)

So +1 from me.
Author of: Belt Planner, Hand Crank Generator, Screenshot Maker, /sudo and more.
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

Rseding91
Factorio Staff
Factorio Staff
Posts: 13209
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Custom prototypes for inter-mod expansion

Post by Rseding91 »

eradicator wrote:
Boodals wrote:I proposed the idea to Rseding in the discord[/url]. His main concern was that people could include data.raw (or any prototype) into one of the custom prototypes, then in the control phase access it and assume it is accurate, when in reality the values could have been modified by another mod.
You can tell him that for as long as factorio modding existed, ppl have tried and "succeded" in pushing arbitrary data.raw data into the control phase, by abusing order strings, prototype names, serialization, etcpp. You can't really prevent it. Only make it difficult. And there's so many other sources of bugs, that this isn't really much of an issue imho. And besides some people might actually do it correctly. I mean...the ZAdventure guy has supposedly succeeded in building whole function libraries for control in data stage. Supposedly to shift the burden of generating all the stuff out of map-loading. (He said that.)

So +1 from me.
While I know someone will always find a way to do it I don't want to make it easy for them. They *shouldn't* do it and so far nobody has convinced me otherwise.

"Making it easy for mods to do whatever" is what modded Minecraft is and I've seen all too well how that ends up performance wise.

If I could implement a system where mods could define X as a prototype type they want and be sure they aren't just filling it will data.raw I could see utility in that. But I can also see mods abusing that and adding loads of different types bogging down the game.
If you want to get ahold of me I'm almost always on Discord.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 5206
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: Custom prototypes for inter-mod expansion

Post by eradicator »

Rseding91 wrote:"Making it easy for mods to do whatever" is what modded Minecraft is and I've seen all too well how that ends up performance wise.
*nods*
Rseding91 wrote:While I know someone will always find a way to do it I don't want to make it easy for them. They *shouldn't* do it and so far nobody has convinced me otherwise.
Yea, i know your usual stance. I just have a different opinion on the size and shape of the windmills you're fighting here :). I.e. i think it's a) not really difficult to do it and b) not really that useful that it would be widely used.
Hm...also i could've sworn there were several mods on the portal that do this for the inclined modder...i can only find one deprecated example.

As for "checking that ppl don't dump data.raw"...hm... the only possibility i see would be limiting creationg of custom prototypes to a new loading stage that doesn't have access to data. But having no access at all is a severe limitation for any concept that needs dynamic generation. :|
Author of: Belt Planner, Hand Crank Generator, Screenshot Maker, /sudo and more.
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

Boodals
Fast Inserter
Fast Inserter
Posts: 129
Joined: Sun Feb 11, 2018 7:10 pm
Contact:

Re: Custom prototypes for inter-mod expansion

Post by Boodals »

eradicator wrote:limiting creationg of custom prototypes to a new loading stage that doesn't have access to data. But having no access at all is a severe limitation for any concept that needs dynamic generation. :|
I actually think that's a fantastic idea. It would be harder to implement, and it could be a limitation for a few mods, but it would very simply solve the problem, and for 9 mods out of 10 you don't need to see the other prototypes. As long as you can add regular prototypes at the same time as custom ones, and modify custom prototypes at some point, there shouldn't be a problem.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 5206
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: Custom prototypes for inter-mod expansion

Post by eradicator »

Boodals wrote: As long as you can add regular prototypes at the same time as custom ones, and modify custom prototypes at some point, there shouldn't be a problem.
Which is exactly the things i'd expect to not be possible with that approach.
Author of: Belt Planner, Hand Crank Generator, Screenshot Maker, /sudo and more.
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

User avatar
Therax
Filter Inserter
Filter Inserter
Posts: 470
Joined: Sun May 21, 2017 6:28 pm
Contact:

Re: Custom prototypes for inter-mod expansion

Post by Therax »

I may be misunderstanding, but what is the value proposed here? What does it permit that isn't already possible? Since you're defining entirely new prototype types, there can by definition be nothing useful that can be done in the core C++ engine with these prototypes.

If you intend to create base game prototypes based on these custom prototypes, you can already do this in data-updates or data-final-fixes. All data phases run in a single shared state, and there is nothing special about data.raw. A hosting mod can create a global "planet_data" in its data phase, then an "extending mod" can put any arbitrary data into that table during its own data phase, and the "hosting" mod can read out any changes during its own data-updates phase. You don't even have to use a new global. You could put arbitrary data in data.raw["planet"] and the Factorio engine will just ignore the extra information during prototype creation.

If the concern is being able to pass arbitrary data from the data phase to the control phase that would be in your custom_data table, there are existing techniques that eradicator alluded to.
Miniloader — UPS-friendly 1x1 loaders
Bulk Rail Loaders — Rapid train loading and unloading
Beltlayer & Pipelayer — Route items and fluids freely underground

Boodals
Fast Inserter
Fast Inserter
Posts: 129
Joined: Sun Feb 11, 2018 7:10 pm
Contact:

Re: Custom prototypes for inter-mod expansion

Post by Boodals »

Therax wrote:A hosting mod can create a global "planet_data" in its data phase, then an "extending mod" can put any arbitrary data into that table during its own data phase, and the "hosting" mod can read out any changes during its own data-updates phase. You don't even have to use a new global. You could put arbitrary data in data.raw["planet"] and the Factorio engine will just ignore the extra information during prototype creation.
That's actually a really smart idea that I didn't consider, however as you guessed, the main problem is getting data from the data-phase to the control-phase.
Therax wrote:there are existing techniques that eradicator alluded to.
The existing techniques are being prevented in 0.17 (by limiting string length, Rseding said in a stream), and they are ugly workarounds at best.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 5206
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: Custom prototypes for inter-mod expansion

Post by eradicator »

Boodals wrote:
Therax wrote:there are existing techniques that eradicator alluded to.
The existing techniques are being prevented in 0.17 (by limiting string length, Rseding said in a stream), and they are ugly workarounds at best.
I certainly didn't mention any "techniques". I only spoke of abuse :p.
@Boodals Do you have a link to the stream?
Author of: Belt Planner, Hand Crank Generator, Screenshot Maker, /sudo and more.
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

Boodals
Fast Inserter
Fast Inserter
Posts: 129
Joined: Sun Feb 11, 2018 7:10 pm
Contact:

Re: Custom prototypes for inter-mod expansion

Post by Boodals »

eradicator wrote: @Boodals Do you have a link to the stream?
Rseding said that he had to disable stream VODs, otherwise he wouldn't be allowed to stream the game's code. So you only have my word unless Rseding/another dev confirms (although I may have misremembered, he might have said it was planned, or that he wanted to do it but hasn't gotten permission, or something).

Rseding91
Factorio Staff
Factorio Staff
Posts: 13209
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Custom prototypes for inter-mod expansion

Post by Rseding91 »

Boodals wrote:
eradicator wrote: @Boodals Do you have a link to the stream?
Rseding said that he had to disable stream VODs, otherwise he wouldn't be allowed to stream the game's code. So you only have my word unless Rseding/another dev confirms (although I may have misremembered, he might have said it was planned, or that he wanted to do it but hasn't gotten permission, or something).
It's planned.
If you want to get ahold of me I'm almost always on Discord.

User avatar
bobingabout
Smart Inserter
Smart Inserter
Posts: 7352
Joined: Fri May 09, 2014 1:01 pm
Contact:

Re: Custom prototypes for inter-mod expansion

Post by bobingabout »

I can see the game's code o3o

I just spent 6 hours yesterday reading the code. now I'm at work, so...
Creator of Bob's mods. Expanding your gameplay since version 0.9.8.
I also have a Patreon.

Post Reply

Return to “Modding interface requests”