Type-specific load order definitions and dependencies

[sparr]

I've recently come back to Factorio modding after a break, prompted by someone reporting a "bug" in one of my mods that was caused by another mod creating new entities in data-final-fixes. I'm doing the necessary dance to resolve the crash, but I don't want to keep doing that forever, so here's my idea to resolve at least some of this...

Allow mods to describe an arbitrary number of data scripts, each with their own potential dependency list of scripts that need to run before or after them, and each with a simple(?) description of what data it needs to already exist, what types of data it reads, and what types of data it writes, including whether the writes will be new data, editing existing data, or removing data. Then use a dependency resolver across all such scripts to figure out which scripts to run in which order.

While this would make things a little more complicated for developers of complex mods, it would go a long way to resolving ongoing problems with the current script loading order and just three data stages, which seems to be trending over time to people just moving functionality into later stages.

[DaveMcW]

Given that modders don't even use the 3 data stages properly, I don't see how a more complex system could work.

[sparr]

I won't say most, but many of the places where people use the current system "improperly" are them trying to get around limitations of the current system.

[Rseding91]

Realistically; this is never going to happen. Sorry.

You just need to add optional dependencies on other mods to make your mod load after them. Or mark them as incompatible so your mod doesn't run when they're installed.

There is no solution to this other than manually forcing a given mod to run after others by doing the above.

[Optera]

1 or 2 years ago I stopped trying to appease other mods, and did the silly dance required to work alongside mods defining prototypes in final fixes.
Now I just flag such mods as incompatible, leave a bug report on that mod and am done with it.

Mod authors just need to learn to treat final fixes as if it was read only.

[Creidhne]

Hi,
I'm just dropping here out of curiosity as someone who started modding quite recently (I don't currently have any need of changes in the data stage, but I can understand that others have).

Given that modders don't even use the 3 data stages properly, I don't see how a more complex system could work.

Where is the proper usage of the 3 data stages defined ? As I see in the API doc or wiki modding tutorial, it's just recommended to do changes as early as possible, but no restriction at any stage.

You just need to add optional dependencies on other mods to make your mod load after them.

Let's say that in a parallel universe, Factorio's base game doesn't generate barreled items/fluids itself, but it was done by a 3rd party mod. This mod would have a few possibilities to work:

• Add every other mod that declares a fluid prototype to the game as optional dependency (= a lot).
• Use a later stage in the data phase, and hope nobody adds a new fluid after that.
• Mark every mod that declare a fluid after your data script as incompatible.
• Write a core mod (adding the empty barrel & support of vanilla fluids), plus compatibility mods for each other mod declaring fluids.

Solution 1 is a massive pain for the mod developer.
Solution 2 will necessarily go wrong whenever 2 mods wants to access the "final" list of items/fluids to generate more items/fluids.
Solution 3 has the massive workload of 1, while resulting in a worse mod.
Solution 4 is more work than 1, for less user-friendliness (they have to not forget all the compatibility mods).

Factorio can get with solution 2 because as the dev they can just one-sidedly declare "If you want our auto-barreling feature, declare your fluid before our data-update script. We don't support the others". Major mods might also be able to do so. But the average mod has no easy way out.

Mod authors just need to learn to treat final fixes as if it was read only.

What is the point of a data phase that doesn't create/modify/delete any prototype ?

The official doc doesn't restrict prototype creation to specific stages in any way. The base game even creates items/recipes in the 2nd pass. So we even have an "official" precedent that if you want to auto-generate new prototypes from all defined prototypes, N+1 escalation to the next data phase is a valid practice. Obviously when 2 mods are racing in such a way to be the last one loaded, there'll be trouble. No amount of data stages would solve that.

TL;DR: There doesn't seem to be any obvious or documented "valid usage" of data stages (if I missed some important documentation, please point me to it). The current system of 3 data stages (or any fixed number of data stages) will always make writing generic (generate prototype Y from every prototype X) & inter-compatible mods extremely tedious.

[DaveMcW]

Where is the proper usage of the 3 data stages defined ? As I see in the API doc or wiki modding tutorial, it's just recommended to do changes as early as possible, but no restriction at any stage.

As early as possible is a good rule, the rest can be deduced from it.

Add new entity - data.lua
Modify a specific mod's entities - data.lua with depenency
Add a derived set of entities - data-updates.lua
Modify a general set of entities - data-updates.lua (if this conflicts with the previous line, add a dependency)
Duplicate a general set of entities - data-final-fixes.lua

[sparr]

The problem comes when you want to make derived entities from derived prototypes from derived prototypes, without declaring hundreds of mod dependencies.

This idea was intended to combat that. If mods could more verbosely describe what they were trying to do, a dependency resolver could fix many examples of this.

My fluid spill mod and existing barreling mods could all describe themselves as "reads in fluids, creates new items and recipes", and then the resolver could try to run them after anything that creates or edits fluids, and before anything that reads in items and recipes. Sure, there would be some dependency loops, but that wouldn't be any worse than what we have now, and it would allow resolving a lot of situations we can't currently resolve (at all or without significant ongoing developer intervention).

[sparr]

Tangentially... a boon for this would be for a script to be able to flag itself as idempotent, so that it could be run multiple times at multiple points in the dependency tree.

[ickputzdirwech]

Since mods load in alphabetical order you can also skip some dependancies by giving your mod a name that starts with eg „zz“.

Afaik it is also possible to add hidden optional dependencies so they are bot displayed in the mod portal and players don’t get the feeling they have to play with certain mods.